Skip to content
This repository has been archived by the owner on Apr 3, 2019. It is now read-only.

Add support for large dataset results #5

Open
bradgessler opened this issue Feb 10, 2016 · 0 comments
Open

Add support for large dataset results #5

bradgessler opened this issue Feb 10, 2016 · 0 comments
Milestone

Comments

@bradgessler
Copy link
Contributor

Queries with very large results can timeout Datagram. Run the queries in a background job and store them in a file or object store.

Issue #3 is relevant for tracking a query that's "in progress" via versioning. If the results are stores in a file or object store the versioning mechanism could easily be used to deal with large async queries.

Currently we're just sticking a LIMIT 100; at the end of a query if a LIMIT isn't specified. Ideally we'd do this pagination transparently/lazily if you run a query from the web view and just want to see the results (e.g. load query results in 100 row chunks).

For larger/longer queries the server just times out.

@bradgessler bradgessler added this to the 2.0 milestone Feb 10, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant