processing .incbin with offset/count uses excessive memory

[szllzs]

compile error log

build i386 wine:
winegcc output shell32.dll LLVM out of memory from wine-8.2

compile on termux-app_v0.118.0+github-debug_x86.apk
clang-16.0.1 i686

but x86_64 wine can build shell32.dll ok on termux-app_v0.118.0debug_x86_64.apk

[efriedma-quic]

A 32-bit compiler is limited to allocating 4GB of memory; probably the file in question just requires more memory than that for some reason.

[szllzs]

Even with 8GB memory settings, there is still a memory overflow issue. I feel like it has nothing to do with the memory size, and Clang 15.x doesn't have this problem, it can compile successfully. With Clang 16, I can reproduce this bug stably even when I set the make thread to 1.

[efriedma-quic]

Even if you have more than 4GB memory, a 32-bit process can't use more than 4GB of address-space. So you can "run out of memory" even if there's memory available.

It's hard to say more without access to the file in question.

[szllzs]

.incbin causes memory explosions，can it be solved by adding compilation parameters？
https://bugs.winehq.org/show_bug.cgi?id=54889

[efriedma-quic]

That looks like a bug. AsmParser::processIncbinFile loads the whole file into memory, not just the specific bits that are necessary.

[llvmbot]

Hi!

This issue may be a good introductory issue for people new to working on LLVM. If you would like to work on this issue, your first steps are:

1. Assign the issue to you.
2. Fix the issue locally.
3. Run the test suite locally.
   3.1) Remember that the subdirectories under test/ create fine-grained testing targets, so you can
   e.g. use make check-clang-ast to only run Clang's AST tests.
4. Create a git commit
5. Run git clang-format HEAD~1 to format your changes.
6. Submit the patch to Phabricator.
   6.1) Detailed instructions can be found here

For more instructions on how to submit a patch to LLVM, see our documentation.

If you have any further questions about this issue, don't hesitate to ask via a comment on this Github issue.

@llvm/issue-subscribers-good-first-issue

[tfsingh]

@llvmbot assign