feat: port create_table to the async python API and the remote rust API #1031
Conversation
python/python/lancedb/table.py
Outdated
| >>> table = db.create_table("my_table", data=[{"vector": [1.1, 1.2], "b": 2}]) | ||
| >>> table.head() | ||
| pyarrow.Table | ||
| vector: fixed_size_list<item: float>[2] | ||
| child 0, item: float | ||
| b: int64 | ||
| ---- | ||
| vector: [[[1.1,1.2]]] | ||
| b: [[2]] | ||
|
|
||
| Can append new data with [Table.add()][lancedb.table.Table.add]. | ||
|
|
||
| >>> table.add([{"vector": [0.5, 1.3], "b": 4}]) | ||
|
|
||
| Can query the table with [Table.search][lancedb.table.Table.search]. | ||
|
|
||
| >>> table.search([0.4, 0.4]).select(["b"]).to_pandas() | ||
| b vector _distance | ||
| 0 4 [0.5, 1.3] 0.82 | ||
| 1 2 [1.1, 1.2] 1.13 |
There was a problem hiding this comment.
These examples probably need await now, right?
There was a problem hiding this comment.
Or is this left as a TODO?
There was a problem hiding this comment.
I'm going to leave this as a TODO. My rationale is that some of these examples need other capabilities (e.g. this one needs add and so it'll be easier to do the doc tests all at once at the end.
python/src/connection.rs
Outdated
| let mode = if mode == "create" { | ||
| CreateTableMode::Create | ||
| } else if mode == "overwrite" { | ||
| CreateTableMode::Overwrite | ||
| } else { | ||
| // TODO: allow users to pass in open options | ||
| CreateTableMode::exist_ok(|builder| builder) | ||
| }; |
There was a problem hiding this comment.
nit: seems like a good use for match?
| let mode = if mode == "create" { | |
| CreateTableMode::Create | |
| } else if mode == "overwrite" { | |
| CreateTableMode::Overwrite | |
| } else { | |
| // TODO: allow users to pass in open options | |
| CreateTableMode::exist_ok(|builder| builder) | |
| }; | |
| match mode { | |
| "create" => CreateTableMode::Create, | |
| "overwrite" => CreateTableMode::Overwrite, | |
| _ => CreateTableMode::exist_ok(|builder| builder) | |
| } |
There was a problem hiding this comment.
Switched to match and put in a helper function to share with create_empty_table. I also updated the node version to behave more similarly to the python version.
| .post(&format!("/v1/table/{}/create", options.name)) | ||
| .body(data_buffer) | ||
| .header(CONTENT_TYPE, ARROW_STREAM_CONTENT_TYPE) | ||
| .header("x-request-id", "na") |
There was a problem hiding this comment.
This is for tracing purposes?
There was a problem hiding this comment.
It was a UUID. It's no longer used by LanceDb Cloud. However, LDBC returns a deserialize error if the field is missing. We should fully remove it from LDBC and then we can remove it here.
… async in python. Added mode/existOk to typescript create_table. Changed countRows in TS to return number instead of bigint
…ts do) and it uses too much space for CI
westonpace
force-pushed
the
feat/new-api-create-table
branch
from
6a92fe2
to
ec80d9b
Compare
… be rewritten soon, so that doctest passes
…PI (lancedb#1031) I've also started `ASYNC_MIGRATION.MD` to keep track of the breaking changes from sync to async python.
…PI (lancedb#1031) I've also started `ASYNC_MIGRATION.MD` to keep track of the breaking changes from sync to async python.
…PI (#1031) I've also started `ASYNC_MIGRATION.MD` to keep track of the breaking changes from sync to async python.
I've also started
ASYNC_MIGRATION.MDto keep track of the breaking changes from sync to async python.