Attachments causes report generation to take forever

[v84a]

For large sized execution with several steps and attachments, report generation takes longer than execution itself
We have really long end to end test cases, with complex and hierarchial steps which can go in thousands. Each step has one or more attachment. When full regression tests are executed which takes say 5 hours to execute, report generation is taking more than 12 hours.

Primary reason appears to be that allure creates small sized files for each attachment, which happens to run into more than 300k files in our case. Then during report generation, these files are moved from result folder to report folder. This takes forever.

Expected behavior
I can only say that report generation should be quicker and we shouldn't have to wait for reports to be produced.

Screenshots

Environment (please complete the following information):

Allure version	2.9.3
Test framework	testng@7.4.0
Allure adaptor	allure-testng@2.19.0
Generate report using	allure-commandline@2.19.0

Additional context
Since we realized this problem, we did an override on allure attachments. Started a directory listing server, updated our code to place files on this server instead of attaching to allure steps, created a link to attachment and attached as parameter to allure step. This reduced our report generation time from 12 hours to 2 minutes. However, they look very ugly in the report generated.

Questions/ Suggestions
Currently, allure tries to create attachment file to every attachment, even links and strings for a test step. Is there a way to keep this data in results.json only and not create an external attachment file?
We also tried to create an html string of links and add it as parameter to test step. However, allure will show this html string as plan string (sort of escaping html). Is there a way to disable html escaping for parameter values, or just specific parameter if not all as a whole?

[baev]

You can use TestLifecycleListener to remove unused attachments (for example, you can remove all the attachments from passed tests) before report generation.

[v84a]

Hi @baev unfortunately I cannot, for regulatory reasons, we need to keep evidences for passed scenarios as well.

[nbdnnm]

Hi @baev and @anilverma-070
We have encountered the same problem.
In our case, we use it in pytest, and as result, we have about 2.7 GB of raw data (results+attachments) which should be generated then into a report.
There are about 15k tests, 46k attachments, 90k containers.
It takes around 50min to generate the report in the pipeline, around 30min on my mac m1.
So we dived into that and came to a quick and raw patch that gives us about 10min on mac m1.
The CPU usage jumped 10%->80%.
The resulting report seems to be identical (still need to figure out how to automatically compare it, just did it manually checking numbers in different places).
But I am not familiar enough with all the allure internals and potential corner cases and it is just an idea after a day of experimenting.
So it would be great if @baev can have a look and estimate do such changes make sense or we are missing something significant and it would not work for other cases.
Then I can have a look what else can pe optimized the same way.
And @anilverma-070 would be great to test it on your report and see the numbers and if it generates the same content.
UPDATED 05.04-19.11: https://github.com/nbdnnm/allure2/releases/tag/05.04-18.09

[Vladislav-Bartalevich]

Hi!
@baev I faced this issue, too.

There are about ~2k test tests with logging only (so there are no attachments for except captured log output). The average report size is ~104mB and it takes extra 7-10 minutes for allurectl to process all the files after tests are finished.

It's kinda weird since whole the test run is finished in 4-5mins :/

[nbdnnm]

Hi @Vladislav-Bartalevich
could you try my version on your report? https://github.com/nbdnnm/allure2/releases/tag/05.04-18.09