Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For some forms, v1.8 uses an unacceptable amount of memory #595

Closed
1 of 2 tasks
lognaturel opened this issue Mar 14, 2022 · 4 comments · Fixed by #596
Closed
1 of 2 tasks

For some forms, v1.8 uses an unacceptable amount of memory #595

lognaturel opened this issue Mar 14, 2022 · 4 comments · Fixed by #596

Comments

@lognaturel
Copy link
Contributor

lognaturel commented Mar 14, 2022

A 200kb Excel doc was measured using 1.5GB of memory.

The most likely culprit is openpyxl, introduced in #575 by @sheppard

From https://openpyxl.readthedocs.io/en/stable/performance.html:

Memory use is fairly high in comparison with other libraries and applications and is approximately 50 times the original file size, e.g. 2.5 GB for a 50 MB Excel file.

What we're seeing is higher by two orders of magnitude.

@lognaturel
Copy link
Contributor Author

lognaturel commented Mar 14, 2022

openpyxl read only mode appears to bring mem usage back to v1.7 levels. Hurray for an easy fix. 😮‍💨

@lognaturel lognaturel changed the title v1.8 uses an unacceptable amount of memory For some forms, v1.8 uses an unacceptable amount of memory Mar 14, 2022
@lognaturel
Copy link
Contributor Author

lognaturel commented Mar 14, 2022

Looks like it's a specific form running into something like what this post describes: https://stackoverflow.com/questions/47582274/iterate-through-columns-in-read-only-workbook-in-openpyxl

There are memory gains to using read-only mode so we should still do it but this is not as critical as it originally seemed.

@lognaturel
Copy link
Contributor Author

lognaturel commented Mar 14, 2022

For whatever reason, the specific form this was seen on had a huge number of extra columns. Deleting those makes the form convert quickly with minimal memory usage with v1.8.0. I still would like to do #596 but it's not critical. CC @aurdipas

@lognaturel
Copy link
Contributor Author

lognaturel commented Mar 16, 2022

More users have been reporting this so grateful for your help getting it addressed, @lindsay-stevens 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant