is it possible to create collection per user?

[JenuelDev]

So, for example with half a million users. Is it possible to create collection per user? any advice for this one? or should I add a metadata? or something?

[yhmo]

Each time you create a new collection, milvus will add more threads to manage the collection. With a larger number of collections, milvus could run into trouble or slow response. So, it is strongly recommended to keep the number of collections under 1000.

Partition key is a good solution for handling millions of users: https://milvus.io/docs/multi_tenancy.md#Partition-oriented-multi-tenancy

For example, we create a collection with a partition key field: "username", is_partition_key=True

fields=[
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=False),
    FieldSchema(name="vector", dtype = DataType.FLOAT_VECTOR, dim=dim),
    FieldSchema(name="username", dtype = DataType.VARCHAR, max_length=100, is_partition_key=True),
]
schema = CollectionSchema(fields=fields)

Then declare a collection, set the number of partitions:
collection = Collection(name="xxx", schema=schema, num_partitions=500)
num_partitions=500 means we will create 500 partitions inside this collection.

When we insert data, we assign a user name for each record.
The records are automatically distributed to different partitions, according to the hash value of the username.

When we search, we use filtering expression to search the records of a user:
collection.search(expr="username=='Mike'", ......)

With username=='Mike', milvus can quickly know which partition to be searched according to the hash value of "Mike". Then it only search the records of "Mike" inside the partition.