Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of memory error when using queryset iterator #256

Open
lucasmoeskops opened this issue May 7, 2024 · 0 comments
Open

Out of memory error when using queryset iterator #256

lucasmoeskops opened this issue May 7, 2024 · 0 comments

Comments

@lucasmoeskops
Copy link

lucasmoeskops commented May 7, 2024

What happened?

A search engine indexer module uses a queryset iterator to iterate over all the records to index. This sends a sql query to the database to fetch all data, but also to iterate over it and not to actually fetch all of this. Cachalot intercepts the request and seems to ignore the fact that it is going to iterate, instead fetching all the data from the table. This caused on out of memory error on the server.

What should've happened instead?

Cachalot should understand that the query was sent by an iterator and just let it query the database the way it did OR have support for this iteration process and get the right data from the cache somehow.

Steps to reproduce

OS -> Ubuntu
Django version -> 4.2.9
Database -> Postgres
django-cachalot -> 2.6.2

@lucasmoeskops lucasmoeskops changed the title Bug when using queryset iterator Out of memory error when using queryset iterator May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant