Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client: counting number of matching entries when data_returned is not available #1924

Closed
ml-evs opened this issue Jan 9, 2024 · 0 comments · Fixed by #1925
Closed

Client: counting number of matching entries when data_returned is not available #1924

ml-evs opened this issue Jan 9, 2024 · 0 comments · Fixed by #1925
Labels
client Issues/PRs relating to the OPTIMADE client. enhancement New feature or request

Comments

@ml-evs
Copy link
Member

ml-evs commented Jan 9, 2024

Since we added a default timeout to the data_returned field (which can require a whole COLSCAN for every page of results), it is now tricky to count the number of results returned by a query unless the whole database fits in memory. This was already the case for some other databases, since data_returned is optional.

The client should specialise the --count method in this case and perform a binary search for the last accessible page matching a query. This method should require significantly fewer requests than exhaustive pagination.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Issues/PRs relating to the OPTIMADE client. enhancement New feature or request
Projects
None yet
1 participant