[llvm-nm] Improve performance while faking symbols from function starts #162755
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-llvm-binary-utilities Author: Daniel Rodríguez Troitiño (drodriguez) ChangesBy default The implementation that looked for those Instead of the nested loop, exchange time for memory and add all the addresses of the symbols into a set that can be checked then for each of the Fixes #93944 Full diff: https://github.com/llvm/llvm-project/pull/162755.diff 1 Files Affected:
diff --git a/llvm/tools/llvm-nm/llvm-nm.cpp b/llvm/tools/llvm-nm/llvm-nm.cpp
index ff07fbbaa5351..1a0d045d8daa3 100644
--- a/llvm/tools/llvm-nm/llvm-nm.cpp
+++ b/llvm/tools/llvm-nm/llvm-nm.cpp
@@ -16,6 +16,7 @@
//===----------------------------------------------------------------------===//
#include "llvm/ADT/StringSwitch.h"
+#include "llvm/ADT/SmallSet.h"
#include "llvm/BinaryFormat/COFF.h"
#include "llvm/BinaryFormat/MachO.h"
#include "llvm/BinaryFormat/XCOFF.h"
@@ -1615,12 +1616,11 @@ static void dumpSymbolsFromDLInfoMachO(MachOObjectFile &MachO,
}
// See if these addresses are already in the symbol table.
unsigned FunctionStartsAdded = 0;
+ SmallSet<uint64_t, 32> SymbolAddresses;
+ for (unsigned J = 0; J < SymbolList.size(); ++J)
+ SymbolAddresses.insert(SymbolList[J].Address);
for (uint64_t f = 0; f < FoundFns.size(); f++) {
- bool found = false;
- for (unsigned J = 0; J < SymbolList.size() && !found; ++J) {
- if (SymbolList[J].Address == FoundFns[f] + BaseSegmentAddress)
- found = true;
- }
+ bool found = SymbolAddresses.contains(FoundFns[f] + BaseSegmentAddress);
// See this address is not already in the symbol table fake up an
// nlist for it.
if (!found) {
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
By default `nm` will look into `LC_FUNCTION_STARTS` for binaries that have the flag `MH_NLIST_OUTOFSYNC_WITH_DYLDINFO` set unless `--no-dyldinfo` flag is passed. The implementation that looked for those `LC_FUNCTION_STARTS` in the symbol list was a double nested loop that checked the symbol list over and over again for each of the `LC_FUNCTION_STARTS` entries. For binaries with couple million function starts and hundreds of thousands of symbols, the double nested loop doesn't seem to finish and takes hours even in powerful machines. Instead of the nested loop, exchange time for memory and add all the addresses of the symbols into a set that can be checked then for each of the `LC_FUNCTION_STARTS` very quickly. What took hours and hours and did not seem to finish now takes less than 10 seconds. Fixes llvm#93944
drodriguez
force-pushed
the
nm-dyldinfo-slow
branch
from
5aa6b87 to
87e7ec3
Compare
ellishg
reviewed
Oct 10, 2025
- Inline `found` boolean. - Improve comment wording - Use foreach loop instead of classic index loop.
ellishg
approved these changes
Oct 10, 2025
There was a problem hiding this comment.
Just wanted to point out that this optimization only works if all elements in FoundFns is unique, otherwise we will need to update SymbolAddresses here:
llvm-project/llvm/tools/llvm-nm/llvm-nm.cpp
Line 1664 in 0aef9eb
After talking offline, I'm convinced this is the case, although it may be worth adding a comment on this.
LC_FUNCTION_STARTSis delta encoded list of addresses from the start of__TEXT, which finishes when 0 is found. Because of that, all the addresses should be unique.
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/72/builds/15439 Here is the relevant piece of the build log for the reference |
DharuniRAcharya
pushed a commit
to DharuniRAcharya/llvm-project
that referenced
this pull request
Oct 13, 2025
…ts (llvm#162755) By default `nm` will look into `LC_FUNCTION_STARTS` for binaries that have the flag `MH_NLIST_OUTOFSYNC_WITH_DYLDINFO` set unless `--no-dyldinfo` flag is passed. The implementation that looked for those `LC_FUNCTION_STARTS` in the symbol list was a double nested loop that checked the symbol list over and over again for each of the `LC_FUNCTION_STARTS` entries. For binaries with couple million function starts and hundreds of thousands of symbols, the double nested loop doesn't seem to finish and takes hours even in powerful machines. Instead of the nested loop, exchange time for memory and add all the addresses of the symbols into a set that can be checked then for each of the `LC_FUNCTION_STARTS` very quickly. What took hours and hours and did not seem to finish now takes less than 10 seconds. Fixes llvm#93944
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By default
nmwill look intoLC_FUNCTION_STARTSfor binaries that have the flagMH_NLIST_OUTOFSYNC_WITH_DYLDINFOset unless--no-dyldinfoflag is passed.The implementation that looked for those
LC_FUNCTION_STARTSin the symbol list was a double nested loop that checked the symbol list over and over again for each of theLC_FUNCTION_STARTSentries. For binaries with couple million function starts and hundreds of thousands of symbols, the double nested loop doesn't seem to finish and takes hours even in powerful machines.Instead of the nested loop, exchange time for memory and add all the addresses of the symbols into a set that can be checked then for each of the
LC_FUNCTION_STARTSvery quickly. What took hours and hours and did not seem to finish now takes less than 10 seconds.Fixes #93944