Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Initial remote table implementation for rust #1024

Merged
merged 9 commits into from
Feb 29, 2024

Conversation

westonpace
Copy link
Contributor

This will eventually replace the remote table implementations in python and node.

rust/lancedb/Cargo.toml Show resolved Hide resolved
headers.insert(
"Host",
HeaderValue::from_str(&host).map_err(|_| Error::Http {
message: format!("non-ascii table name '{}' provided", db_name),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
message: format!("non-ascii table name '{}' provided", db_name),
message: format!("non-ascii database name '{}' provided", db_name),

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@westonpace you marked this as resolved, but looks like it's still "table name". Is it supposed to be "table name"?

rust/lancedb/src/remote/client.rs Outdated Show resolved Hide resolved
api_key: &str,
region: &str,
db_name: &str,
has_host_override: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this supposed to be used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user has overridden the host then we set the x-lancedb-database header. I copied this from the existing python client. I believe the rationale is that we normally get the db header from the URI. However, when testing locally, we don't have DNS setup and we need to fallback to some alternate method.

It would probably be simpler, and I think it would be safe, if we just always set this header too. CC @albertlockett @chebbyChefNEQ for second opinion.

rust/lancedb/src/remote/client.rs Show resolved Hide resolved
.client
.get("/v1/table/")
.query(&[("limit", 10)])
.query(&[("page_token", "")])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess pagination is TODO?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. It wasn't part of the Table abstract class, so it wasn't documented at all and there was no implementation for the local table. I figure I can add it back in, with support for both tables, at a later date. I'll make a issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just referring to the fact we only request the first page here (first 10).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's pretty broken. Still, I think I'd rather defer. I've added this to #926 and I'm starting to track all the cleanup tasks as part of #926

Copy link
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but have a question about the earlier comment I made.

@westonpace westonpace merged commit 294c33a into lancedb:main Feb 29, 2024
11 of 13 checks passed
raghavdixit99 pushed a commit to raghavdixit99/lancedb that referenced this pull request Apr 5, 2024
This will eventually replace the remote table implementations in python
and node.
raghavdixit99 pushed a commit to raghavdixit99/lancedb that referenced this pull request Apr 5, 2024
This will eventually replace the remote table implementations in python
and node.
westonpace added a commit that referenced this pull request Apr 5, 2024
This will eventually replace the remote table implementations in python
and node.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants