pgvector improvements #98
Conversation
| raise IncompatibilityError(f"Unsupported distance metric: {distance}") | ||
|
|
||
| cls.conn.execute("SET max_parallel_workers = 128") | ||
| cls.conn.execute("SET max_parallel_maintenance_workers = 128") |
There was a problem hiding this comment.
This will use all available cores to build the index (let me know if it should do something different)
There was a problem hiding this comment.
If you want to utilize all cores, would not it be better to get the number of cores from OS rather than to hardcode it to 128?
There was a problem hiding this comment.
Setting a high upper limit should be simpler than trying to get the exact number of cores from the server (and won't make a difference in the number of cores used).
| # maintenance_work_mem should be ~65% | ||
| command: postgres -c shared_buffers=2GB -c maintenance_work_mem=5GB -c max_connections=200 | ||
| # shm_size should be shared_buffers + maintenance_work_mem | ||
| shm_size: 7g |
There was a problem hiding this comment.
These settings should be updated based on the hardware used to run the benchmark
Edit: Updated the parameters for 25 GB of memory in a follow-up commit
| "search_params": [ | ||
| { "parallel": 1, "search_params": { "hnsw_ef": 128 } } | ||
| { "parallel": 8, "search_params": { "hnsw_ef": 128 } } |
There was a problem hiding this comment.
Updated the parallel options for pgvector-default to be the same as qdrant-default in qdrant-single-node.json
There was a problem hiding this comment.
It's not used for running the experiments. But okay :)
| @@ -3,13 +3,17 @@ version: '3.7' | |||
| services: | |||
| pgvector: | |||
| container_name: pgvector | |||
| image: ankane/pgvector:v0.5.1 | |||
| image: pgvector/pgvector:pg16 | |||
There was a problem hiding this comment.
New Docker image as of v0.6.0
Edit: Updated to use the versioned tag 0.6.0-pg16
| with cls.cur.copy( | ||
| "COPY items (id, embedding) FROM STDIN WITH (FORMAT BINARY)" | ||
| ) as copy: | ||
| copy.set_types(["integer", "vector"]) |
There was a problem hiding this comment.
Binary format requires set_types in psycopg 3
| @@ -7,6 +7,7 @@ | |||
| def get_db_config(host, connection_params): | |||
| return { | |||
| "host": host or "localhost", | |||
| "port": PGVECTOR_PORT, | |||
There was a problem hiding this comment.
PGVECTOR_PORT was previously unused
| cls.conn.execute( | ||
| f"CREATE INDEX ON items USING hnsw (embedding {hnsw_distance_type}) WITH (m = {cls.upload_params['hnsw_config']['m']}, ef_construction = {cls.upload_params['hnsw_config']['ef_construct']})" | ||
| ) |
There was a problem hiding this comment.
Hm, might be a bit unfair to the other competitors, which also support loading in a "split-mode", however everyone is welcome to come and improve the setup if they think it is not optimised 🤷♂️
|
@KShivendu could you make a couple of test runs to make sure that everything works fine? |
| @@ -3,13 +3,17 @@ version: '3.7' | |||
| services: | |||
| pgvector: | |||
| container_name: pgvector | |||
| image: ankane/pgvector:v0.5.1 | |||
| image: pgvector/pgvector:0.6.0-pg16 | |||
There was a problem hiding this comment.
Can you please use the official image?
There was a problem hiding this comment.
This is the official image starting with 0.6.0. https://github.com/pgvector/pgvector#docker-1
| "search_params": [ | ||
| { "parallel": 1, "search_params": { "hnsw_ef": 128 } } | ||
| { "parallel": 8, "search_params": { "hnsw_ef": 128 } } |
There was a problem hiding this comment.
It's not used for running the experiments. But okay :)
|
Rebased to fix merge conflict from #100 |
* pgvector improvements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updated Postgres parameters * Use versioned Docker image * Updated pgvector to 0.6.2 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
Thanks |
Hi, thanks for creating this framework! This PR makes a number of improvements to pgvector performance.