High memory usage while creating a xlsx file and xlsx.UseDiskVCellStore not working #596
Comments
|
Do you have some code that you can share - it's hard for me to debug it based on what you've told me. |
|
Sure, I have removed extra code (like error checking) to make it small and simple. |
|
OK, I did some profiling and it looks like there might be a leak in the NewCell call, I'm looking into it. |
|
Perhaps leak is too strong a term. At the moment it seems to rest on the following: We always hold an entire row of cells in memory at once. If you're using a spreadsheet with many columns, this can use quite a significant chunk of memory. Originally I wrote the backend to work around this by only keeping a single cell in memory at a time, but this required adding a "Save" call to the Cell which needed to be called every time you modified a cell, and was rather slow. However, I think something along this lines might be required to fix this issue. I'll think some more. |
|
Okay. |
|
I ran into the same problem, When I try to open an excel file (more than 300,000 rows) used |
|
Stale issue message |
Note, DiskVCellStore is designed to deal with the specific corner case of dealing with very large spreadsheets. To that end, I've favoured enforcing behaviours on developers over modifying the public API of this library to support those cases. This manifests itself as the following rule: When using dhe DiskVCellStore you can only update the row or cell most recently returned from the cell store. In effect this means that if you keep multiple cells or rows in temporary variables, that's fine, so long as you're referencing them only, but should you try to set a value on an old cell or row you'll get a deliberate panic from the library.
Fix #596. DiskVCellStore persists with cell level granuality.
Note, DiskVCellStore is designed to deal with the specific corner case of dealing with very large spreadsheets. To that end, I've favoured enforcing behaviours on developers over modifying the public API of this library to support those cases. This manifests itself as the following rule: When using dhe DiskVCellStore you can only update the row or cell most recently returned from the cell store. In effect this means that if you keep multiple cells or rows in temporary variables, that's fine, so long as you're referencing them only, but should you try to set a value on an old cell or row you'll get a deliberate panic from the library.
Fix tealeg#596. DiskVCellStore persists with cell level granuality.
I have large data (160K+ rows & 39 columns) and while creating excel file it consumes 4GB+ memory and shows out of memory
As you have mentioned in [#487 ] use
UseDiskVCellStoreas a file option.I have passed in the
xlsx.NewFile(xlsx.UseDiskVCellStore)but it's not working, please check why it's not working and give example how to use it or any other solution.The text was updated successfully, but these errors were encountered: