Process gets killed after a while when parsing a large xcactivitylog

[apstygo]

Our project contains a ton of targets and dependencies. The generated xcactivitylog file is around 4.2MB in size. For comparison xcactivitylog for a newly created project is around 7KB in size.

Whenever I try parsing my project's xcactivitylog using HTML reporter xclogparser's process gets killed by the system after about a minute of waiting. It works just fine on smaller projects/targets though. Also it succeeds in dumping the log into a JSON file.

Seems to me XCLogParser is just not able to process a file this big in a reasonable amount of time.

All builds were made in Xcode 12.5.1 (12E507).

Unfortunately, I cannot provide the xcactivitylog file, as it contains sensitive info. Is there a way to enable verbose output for the parse command, so that I could provide you with some more information on what's going on?

[apstygo]

I have found the exact line causing the problem – line 192 in HtmlReporter.writeTargetFiles. Encoding the flattened target into a JSON representation seems to eat through my machines RAM. RAM usage rises steadily until the process is eventually killed by the system.

[apstygo]

I've found the culprit. One of the targets contained around 40000 notes. Seems like JSONEncoder just can't handle such a big amount of data at one time.

I filtered out substeps with 10000 notes or more and XCLogParser was able to create the HTML report. Yay!

Do we even need the notes when dumping into an HTML file? Maybe we could just get rid of them altogether. Or introduce some kind of culling/filtering logic. As of right now, it seems projects with very verbose build phases are incompatible with XCLogParser.

[ecamacho]

Hi, yes JSONEncoder has memory issues with large documents. I added a flag to truncate Notes (and Warnings and Errors) from the final JSON #151
We also have the --omit_notes flag that you can pass to completely skip notes.