How to inspect memory usage?

[zhaown]

Our experiment showed memory footprint 4x larger than the number in theory: with disk-ann index of 40M 768d float vectors, the collection should be 768 * 4Bytes * 40M ≈ 120GB, and the disk-ann index should be ≈ 30GB.But when the collection is loaded, our milvus process used 131.1GB memory.

Is there any tool that can be used to check the memory usage of each part, like PQ/cache, etc.?

We are using milvus 2.2.8, standalone.

[xiaofan-luan]

Our experiment showed memory footprint 4x larger than the number in theory: with disk-ann index of 40M 768d float vectors, the collection should be 768_4Bytes_40M ≈ 120GB, and the disk-ann index should be ≈ 30GB.But when the collection is loaded, our milvus process used 131.1GB memory.

Is there any tool that can be used to check the memory usage of each part, like PQ/cache, etc.?

We are using milvus 2.2.8, standalone.

From my experience, for HNSW
40M 768 take ~240GB memory
DiskANN might take ~64GB memory.

Index build , Datanode Flush may take extra memory. My guess is you can limit your pod to 64GB and check if it can work. if not try 80GB. Sometimes there are some memory allocator issue which prevent milvus from return memory to the system.

30GB is definitely possible, graph index usually takes 1.5-2 times memory than raw data

[zhaown]

I read DiskANN takes 1/4 memory, compare with RAW DATA: https://www.modb.pro/db/619651. Is that should be 30GB for 40M 768?

[zhaown]

@xiaofan-luan It's there any info towards memory requirements of DiskANN? Thanks.

[yhmo]

Deploy a monitoring system to observe: https://milvus.io/docs/monitor.md

Did you set the replica_number for collection.load()?

[zhaown]

Thanks, didn't set replica_number, using default replica_number = 1.