Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Python batch document read & writes #41

Closed
ashvardanian opened this issue Aug 28, 2022 · 0 comments
Closed

Fixing Python batch document read & writes #41

ashvardanian opened this issue Aug 28, 2022 · 0 comments
Assignees
Labels
invalid This doesn't seem right

Comments

@ashvardanian
Copy link
Contributor

At this point, the batch document reads and document writes are semantically wrong. They are internally replacing a batch request with a number of separate requests, which changes the expected behavior.

Worse than that, the implementation parses/converts all inputs at once, but serializes and writes them one-by-one. A proper implementation would create something like a growing_tape_t and parse+convert+serialize entries one-by-one, but would submit those for write just once!

Bringing back the ukv_format_docs_internal_k would allow us to immediately choose the optimal underlying serialization format to avoid following conversions in the backend.

@ashvardanian ashvardanian added the invalid This doesn't seem right label Oct 3, 2022
@ashvardanian ashvardanian added this to the 0.4.0 milestone Oct 3, 2022
ashvardanian added a commit that referenced this issue Oct 26, 2022
Fix: Python batch operations #41
DarvinHarutyunyan pushed a commit that referenced this issue Dec 9, 2022
Fix: Python batch operations #41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right
Projects
No open projects
Development

No branches or pull requests

2 participants