Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AddComment used too many memory ( more than 10GB for only 100 comments) #1310

Closed
pqx0703 opened this issue Aug 9, 2022 · 8 comments
Closed
Projects

Comments

@pqx0703
Copy link

pqx0703 commented Aug 9, 2022

we added about 100 comments in 5 batch, but it use too many memory, like more than 10GB.

code and files are in the attachment

test1.zip

@xuri
Copy link
Member

xuri commented Aug 9, 2022

Thanks for your issue. I have test with your attachment and get runtime memory usage by printBenchmarkInfo function, and it just cost about 11MB, please make sure your code and real memory usage.

@xuri xuri added the wontfix This will not be worked on label Aug 9, 2022
@pqx0703
Copy link
Author

pqx0703 commented Aug 9, 2022

@xuri 你这个函数可能不太准,开个 top 看一下,处理100条就很慢,而且内存明显飙升

我的操作系统是 centos 7.9 , Go 版本 1.6 / 1.8/ 1.9 都试过,都有问题

@xuri
Copy link
Member

xuri commented Aug 9, 2022

The Getrusage uses the low-level operating system primitives, which get the same resident memory usage as the top command. Version 2.6.0 of this library requires the Go language to be 1.15 later. If you're using 1.6, 1.8, or 1.9, you couldn't build successfully. Please check your code or service, and you can also diagnose logic and performance problems by profiling tools.

@xuri
Copy link
Member

xuri commented Aug 9, 2022

I closed this issue. If you have any questions, please let me know to reopen this anytime.

@xuri xuri closed this as completed Aug 9, 2022
@pqx0703
Copy link
Author

pqx0703 commented Aug 9, 2022

@xuri 是 1.16 / 1.18/ 1.19 , 上面打错了

@pqx0703
Copy link
Author

pqx0703 commented Aug 9, 2022

@xuri I think i've found what's wrong.

https://github.com/qax-os/excelize/blob/master/comment.go#L236

It's ok when you create a new file, add some comments, and save it. But when you reopen that file, and append some comments, it loads the existing shapes from xl/drawings/vmlDrawing1.vml everytime, and append it to the current vml.

so, the ploblem is : The existing shapes should only load once !

Dont modify and just run this code: test2.zip , then unzip the 20220808.xlsx file, you would get a 2GB vml file, which is crazy for just 100 comment !!!

image

@pqx0703
Copy link
Author

pqx0703 commented Aug 9, 2022

I've made a fix for this, tested locally and works fine

@xuri xuri removed the wontfix This will not be worked on label Aug 10, 2022
@xuri xuri added this to Performance in v2.6.1 Aug 10, 2022
@xuri
Copy link
Member

xuri commented Aug 10, 2022

Thanks for your feedback, I've accepted pull request #1311, and this patch will be released in the next version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
v2.6.1
Performance
Development

No branches or pull requests

2 participants