How to search faster?

[iswangl]

大家好，我现在有10，000，000条256维的向量，然后我搜索一条向量的top10用时300ms，怎样才能更快呢？想达到一亿向量秒级，这个速度不太能满足，谢谢大家的指导！

[iswangl]

version: 0.5
cluster:
enable: false
role: rw

storage:
path: /var/lib/milvus
auto_flush_interval: 1
wal:
enable: true
recovery_error_ignore: false
buffer_size: 256MB
path: /var/lib/milvus/wal
cache:
cache_size: 24GB
insert_buffer_size: 1GB
preload_collection:
gpu:
enable: true
cache_size: 4GB
gpu_search_threshold: 500
search_devices:
- gpu0
build_index_devices:
- gpu0
fpga:
enable: false
search_devices:
- fpga0
logs:
level: debug
trace.enable: true
path: /var/lib/milvus/logs
max_log_file_size: 1024MB
log_rotate_num: 0
log_to_stdout: false
log_to_file: true

这是server_config.yaml的一些信息

[yhmo]

Let do some testing.
I did't know which kind of index you select, and what search parameters you choose.

I start a milvus server(v1.1.1) on my machine, with the config:

cache:
  cache_size: 40GB
  insert_buffer_size: 1GB

The machine environment:
Ubuntu18.04
Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz 12 cores
64GB memory

The following is a script, the script do these things:

1. create collection, index_file_size=2048MB, dimension=256
2. insert 10M vectors(random)
3. create index IVF_FLAT, with nlist=8192
4. load index data into cache(to get best performance)
5. single thread to search 1000 times, with nq=1, topk=10, nprobe=32
6. use 20 threads do parallel search, each thread execute 50 search(), totally execute 1000 search(), with nq=1, topk=10, nprobe=32

import random
import time
import threading
import numpy as np
from milvus import Milvus, IndexType
from milvus.client.types import MetricType

SERVER_ADDR = "127.0.0.1"
SERVER_PORT = '19530'
COLLECTION_NAME = "TEST"
COLLECTION_DIMENSION = 256
COLLECTION_PARAM = {'collection_name': COLLECTION_NAME,
                    'dimension': COLLECTION_DIMENSION,
                    'index_file_size': 2048,
                    'metric_type':MetricType.L2}
INDEX_TYPE = IndexType.IVF_FLAT
INDEX_PARAM = {'nlist': 8192}
NQ = 1
TOPK = 10
SEARCH_PARAM = {'nprobe': 32}

MILVUS = Milvus(host=SERVER_ADDR, port=SERVER_PORT)

def gen_vec_list(nb, seed=np.random.RandomState(1234)):
    xb = seed.rand(nb, COLLECTION_DIMENSION).astype("float32")
    vec_list = xb.tolist()
    return vec_list

def create_collection(name = COLLECTION_NAME):
    param = COLLECTION_PARAM
    param['collection_name'] = name
    status = MILVUS.create_collection(param)
    print("create", name)

def load_collection(name=COLLECTION_NAME):
    MILVUS.load_collection(collection_name=name)
    print("load collection", name)

def flush_collection(name=COLLECTION_NAME):
    MILVUS.flush(collection_name_array=[name])
    status, count = MILVUS.count_entities(collection_name=name)
    print("flush collection, row count:", count)

def create_partition(tag, name = COLLECTION_NAME):
    status = MILVUS.create_partition(collection_name=name, partition_tag=tag)
    print("create partition", tag)

def drop_collection(name = COLLECTION_NAME):
    status = MILVUS.drop_collection(collection_name=name)
    print("drop", name)

def get_collection_stats(name = COLLECTION_NAME):
    status, stat = MILVUS.get_collection_stats(collection_name=name)
    print(stat)

def create_index(name = COLLECTION_NAME):
    time_start = time.time()
    print("begin create index...")
    status = MILVUS.create_index(collection_name=name, index_type=INDEX_TYPE, params=INDEX_PARAM)
    time_end = time.time()
    print("create index time cost:", time_end - time_start, "sec")

def insert(name = COLLECTION_NAME):
    time_start = time.time()
    batch = 10000
    gen_cost = 0.0
    insert_cost = 0.0
    for i in range(1000):
        time_1 = time.time()
        vectors = gen_vec_list(batch, seed=np.random.RandomState(i+13))
        time_2 = time.time()
        status, ids = MILVUS.insert(collection_name=name, records=vectors)
        time_3 = time.time()
        print("insert", i, name, len(ids), status)
        gen_cost = gen_cost + (time_2 - time_1)
        insert_cost = insert_cost + (time_3 - time_2)
    time_end = time.time()
    print("insert totally time cost:", time_end-time_start, "sec")
    print("generate random vectors cost:", gen_cost, "sec")
    print("insert cost:", insert_cost, "sec")

def single_thread_search(repeat = 1000, calc_qps = True):
    time_start = time.time()
    for i in range(int(repeat)):
        q_records = [[random.random() for _ in range(COLLECTION_DIMENSION)] for _ in range(NQ)]
        status, results = MILVUS.search(collection_name=COLLECTION_NAME, query_records=q_records, top_k=TOPK,
                                        params=SEARCH_PARAM)
    if calc_qps:
        time_end = time.time()
        print("single thread total time cost:", time_end - time_start, "sec", "QPS:", repeat/(time_end - time_start))

def multi_threads_search(repeat = 1000):
    threads = []

    time_start = time.time()
    thread_count = 20
    for i in range(thread_count):
        x = threading.Thread(target=single_thread_search, args=(repeat/thread_count, False))
        threads.append(x)
        x.start()

    for th in threads:
        th.join()

    time_end = time.time()
    print("multi threads total time cost:", time_end - time_start, "sec", "QPS:", repeat/(time_end - time_start))

if __name__ == "__main__":
    drop_collection()
    create_collection()

    insert()
    flush_collection()
    create_index()
    get_collection_stats()

    load_collection()
    search_count = 1000
    single_thread_search(search_count, True)
    multi_threads_search(search_count)

[yhmo]

On my machine, the result is:

single thread total time cost: 8.559851169586182 sec QPS: 116.82446110197311
multi threads total time cost: 3.299497604370117 sec QPS: 303.07644372146854

For single thread search, each search() cost about 10 milliseconds.
For 20 threads search, each search() cost about 3 milliseconds.

[yhmo]

The "nprobe" will affect search performance, if we set nprobe=128, the result will be:

single thread total time cost: 13.492020845413208 sec QPS: 74.11788133576471
multi threads total time cost: 8.435431003570557 sec QPS: 118.54758809321292

[iswangl]

感谢您的解答！我把参数设置为和您的脚本一样后，使用c++ sdk获得了一样的搜索结果 之前使用的参数为nlist=4096 nprobe=128。是不是nprobe越大准确率越高但速度会变慢？

[yhmo]

感谢您的解答！我把参数设置为和您的脚本一样后，使用c++ sdk获得了一样的搜索结果 之前使用的参数为nlist=4096 nprobe=128。是不是nprobe越大准确率越高但速度会变慢？

是的，选择参数的时候要权衡性能和召回率。

[iswangl]

好的了解，再次感谢