Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dot not assume Table._rows will always be in memory #45

Open
5 tasks
turicas opened this issue Sep 2, 2015 · 1 comment
Open
5 tasks

Dot not assume Table._rows will always be in memory #45

turicas opened this issue Sep 2, 2015 · 1 comment

Comments

@turicas
Copy link
Owner

turicas commented Sep 2, 2015

Currently some operations assume all rows are in memory (such as order_by). We may move all the code to something lazy.
For order_by specifically we could sort in-disk instead of in-memory, like csvsort does.

  • Implement LazyTable class
  • Add lazy parameter to rows.plugins.utils.create_table
  • Change all plugins to force (or not) lazyness (example: in HTML
    lazy=False, in CSV lazy=True by default)
  • Change default sample size to 1000 (or any other non-arbitrary number)
  • Create documentation about it
@turicas
Copy link
Owner Author

turicas commented Jun 23, 2017

Working on branch feature/lazyness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant