0.1a3
Pre-releaseThis alpha changes the way embedding vectors are stored. Previously they were stored on the table itself, as columns with names like emb_text_embedding_3_large_256
. #3
They are now instead stored in a separate table, with the name _embeddings_TABLE
where TABLE
is the name of the original table.
If you have previously calculated embedding vectors stored in columns on an existing table, you can migrate them to the new table structure by following these steps: #11
-
Create a new shadow table to store your embeddings. This table should have the name
_embeddings_TABLE
whereTABLE
is the name of your source table, and the same primary key (or compound primary keys). It should then have aemb_xxx
column for each stored embedding vector.One way to create this table is to select a single row and embed just that - this will create the table for you while only costing you one embedding run.
-
Run the following SQL query to copy across your vectors:
insert or replace into _embeddings_TABLE (id, emb_xxx) select id, emb_xxx from TABLE where emb_xxx is not null
-
Drop the old column:
alter table TABLE drop column emb_xxx