copy with BINARY FORMAT fails with cannot find a dumper for type vector

[alfer-edb]

Hi all,

I'm getting an error while I'm using the COPY command with BINARY FORMAT. I'm using the latest pgvecto.rs version (0.3.0) and psycopg3.

from pgvecto_rs.psycopg import register_vector
import psycopg

url = "postgresql://{username}:{password}@{host}:{port}/{db_name}".format(
            port=os.getenv("DB_PORT", "5432"),
            host=os.getenv("DB_HOST", ""),
            username=os.getenv("DB_USER", "postgres"),
            password=os.getenv("DB_PASS", ""),
            db_name=os.getenv("DB_NAME", "ann1")
        )
        with psycopg.connect(url) as conn:
            conn.execute("CREATE EXTENSION IF NOT EXISTS vectors;")
            register_vector(conn)
            conn.execute("CREATE TABLE items (id int, embedding vector(3))")
            conn.execute("ALTER TABLE items ALTER COLUMN embedding SET STORAGE PLAIN")
            conn.commit()
            try:
                print("copying data...")
                with conn.cursor() as cur, cur.copy((
                        "COPY items (id, embedding) FROM STDIN (FORMAT BINARY)"
                        ) as copy:
                    copy.set_types(["int4", "vector"])
                    for i, embedding in enumerate(X):
                        copy.write_row(i, embedding)
                conn.commit()
          [...]      

The error:

  File "/home/app/ann-benchmarks/ann_benchmarks/algorithms/pgvecto-rs/module.py", line 45, in fit
    copy.set_types(["int4", "vector"])
  File "/usr/local/lib/python3.10/dist-packages/psycopg/copy.py", line 144, in set_types
    self.formatter.transformer.set_dumper_types(oids, self.formatter.format)
  File "psycopg_binary/_psycopg/transform.pyx", line 180, in psycopg_binary._psycopg.Transformer.set_dumper_types
  File "psycopg_binary/_psycopg/transform.pyx", line 342, in psycopg_binary._psycopg.Transformer.get_dumper_by_oid
  File "/usr/local/lib/python3.10/dist-packages/psycopg/_adapters_map.py", line 256, in get_dumper_by_oid
    raise e.ProgrammingError(msg)
psycopg.ProgrammingError: cannot find a dumper for type vector (oid 25233) format BINARY

Checking psycopg in pgvecto_rs repo i see the register_vector function is registering loaders and dumpers, hence there should be a dumper type registered already.

Any ideas why is this happening?

[usamoi]

0.3.0 is not released yet.

Binary format is supported since 0.3.0.

[alfer-edb]

My bad, I'm using 0.3.0-alpha.1 since 0.2 doesn't seem to be supporting binary format for copy.

Thanks for the answer, will keep an eye for new releases

[VoVAllen]

We do have support at https://github.com/tensorchord/pgvecto.rs/blob/main/bindings/python/src/pgvecto_rs/psycopg/__init__.py#L50-L54. But the new version of the sdk haven't be released yet.