[CGData] Make an option to skip reading Names into StableFunctionMap #142095
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uilds Names are used for debugging purpose and have no impact on codegen. For a non-trivial project, reading them consumes a lot of memory and slows down the compilation significantly. This patch adds a field in the serialized CGData to remember the total size of Names, and skips reading Names by advancing the pointer when deserializing for the Use mode in non-assertion builds.
kyulee-com
reviewed
Jun 4, 2025
| : DataBuffer(std::move(DataBuffer)) {} | ||
| IndexedCodeGenDataReader(std::unique_ptr<MemoryBuffer> DataBuffer, | ||
| Options Opts) | ||
| : DataBuffer(std::move(DataBuffer)) { |
There was a problem hiding this comment.
Suggested change
| : DataBuffer(std::move(DataBuffer)) { | |
| : DataBuffer(std::move(DataBuffer)), Opts(Opts) { |
There was a problem hiding this comment.
I decide to fully remove Opts for simplicity.
llvm/lib/CGData/CodeGenData.cpp
Outdated
| @@ -155,7 +155,14 @@ CodeGenData &CodeGenData::getInstance() { | |||
| // Instead, just emit an warning message and fall back as if no CGData | |||
| // were available. | |||
| auto FS = vfs::getRealFileSystem(); | |||
| auto ReaderOrErr = CodeGenDataReader::create(CodeGenDataUsePath, *FS); | |||
| CodeGenDataReader::Options Opts; | |||
| #ifdef NDEBUG | |||
There was a problem hiding this comment.
I'd prefer to have consistent behavior between debug and release builds.
Also we can allow users to have explicit control regardless of build types.
It'd be simpler and maintainable without any confusion as the constructor sets it as true.
Can we define a command line flag, and set it based on that? Something like:
static cl::opt<bool> CodeGenDataReadFunctionMapNames(
"codegen-data-read-function-map-names", cl::init(true), cl::Hidden,
cl::desc("Read function map names in CodeGenData (disable to save memory and time)"));
...
Opts.ReadStableFunctionMapNames = CodeGenDataReadFunctionMapNames;
There was a problem hiding this comment.
Actually this is exactly what I initially thought. Maybe I misunderstood your response in the RFC thread.
ellishg
reviewed
Jun 4, 2025
ellishg
reviewed
Jun 4, 2025
nocchijiang
changed the title
|
@kyulee-com Could you please merge this PR for me? |
rorth
pushed a commit
to rorth/llvm-project
that referenced
this pull request
Jun 11, 2025
…lvm#142095) Names are used for debugging purpose and have no impact on codegen. For a non-trivial project, reading them consumes a lot of memory and slows down the compilation significantly. This patch adds a field in the indexed CGData to remember the total size of Names, and creates a command-line option to skip reading Names by advancing the pointer when deserializing the indexed CGData.
tomtor
pushed a commit
to tomtor/llvm-project
that referenced
this pull request
Jun 14, 2025
…lvm#142095) Names are used for debugging purpose and have no impact on codegen. For a non-trivial project, reading them consumes a lot of memory and slows down the compilation significantly. This patch adds a field in the indexed CGData to remember the total size of Names, and creates a command-line option to skip reading Names by advancing the pointer when deserializing the indexed CGData.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Names are used for debugging purpose and have no impact on codegen. For a non-trivial project, reading them consumes a lot of memory and slows down the compilation significantly. This patch adds a field in the indexed CGData to remember the total size of Names, and creates a command-line option to skip reading Names by advancing the pointer when deserializing the indexed CGData.