We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LanceFragment.to_batches
filter
It looks like to_batches isn't respecting the filter kwarg
to_batches
Repro
import lance ds = lance.dataset(path) fragments = ds.get_fragments() for batch in fragments[0].to_batches( batch_size=1, filter="split == 'test'", columns=["image", "split"], with_row_id=True, batch_readahead=8, ): break print(batch) >>> pyarrow.RecordBatch image: binary split: string ---- image: ... split: ["train"]
The text was updated successfully, but these errors were encountered:
dataset.filter
This seems to happen only under the legacy storage format. In stable this seems to be working correctly atm
legacy
stable
Sorry, something went wrong.
westonpace
wjones127
No branches or pull requests
It looks like
to_batches
isn't respecting the filter kwargRepro
The text was updated successfully, but these errors were encountered: