I have a Milvus collection with both dense and sparse vector fields for text retrieval. I also want to store a list of preprocessed keywords for each text chunk, for example:
tags = ["milvus", "array", "filter", "hybrid-search"]
My schema has an ARRAY field for these keywords:
schema.add_field(
field_name="tags",
datatype=DataType.ARRAY,
element_type=DataType.VARCHAR,
max_capacity=20,
max_length=512,
nullable=True, )
I also created a scalar index on this field:
index_params.add_index(
field_name="tags",
index_type="AUTOINDEX",
index_name="tags_index", )
Now I want to run a vector search with client.search() and only return records where the tags array contains a required subset of words. For example:
required_tags = ["milvus", "filter"]
I want to return only records where the tags field contains both "milvus" and "filter".
I tried this filter expression:
filter_expr = 'json_contains_all(tags, ["milvus", "filter"])'
res = client.search(
collection_name="my_collection",
data=[query_vector],
anns_field="dense_vector",
limit=10,
filter=filter_expr,
output_fields=["tags", "text"], )
If I need to add more scalar conditions, I plan to build the expression like this:
and_filters = [
'json_contains_all(tags, ["milvus", "filter"])',
'source == "docs"', ]
filter_expr = " AND ".join(and_filters)
I am using MilvusClient.search() for vector search.
The collection has a dense vector field named dense_vector.
Each entity also has a metadata field named tags.
The tags field is defined as ARRAY.
I created an AUTOINDEX scalar index on the tags field.
I want to combine vector search with an array filter such as json_contains_all(tags, [...]).
I want client.search() to return vector search results only when the tags ARRAY field contains all required words.I am not sure whether json_contains_all(tags, [...]) is the correct filter expression for an ARRAY field, or whether it is only valid for JSON fields.
What I checked:
The tags field is defined as DataType.ARRAY.
The array element type is DataType.VARCHAR.
The field has an AUTOINDEX scalar index.
I want to use the filter inside client.search(), not only in client.query().
I may need to combine this condition with other scalar filters using AND.
so, in Milvus client.search(), how should I write a filter expression to return vector search results where an ARRAY field contains all words from a required subset?