Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BOLT][NFC] Refactor function matching #97502

Merged

Conversation

shawbyoung
Copy link
Contributor

@shawbyoung shawbyoung commented Jul 3, 2024

Moved function matching techniques into separate helper functions for
ease of understanding and to make space for additional function
matching techniques to be added (e.g. call graph function matching).

shawbyoung and others added 2 commits July 2, 2024 17:56
Created using spr 1.3.4
@shawbyoung shawbyoung marked this pull request as ready for review July 3, 2024 16:23
@llvmbot llvmbot added the BOLT label Jul 3, 2024
@llvmbot
Copy link

llvmbot commented Jul 3, 2024

@llvm/pr-subscribers-bolt

Author: Shaw Young (shawbyoung)

Changes

Moved function matching techniques into separate helper functions.


Full diff: https://github.com/llvm/llvm-project/pull/97502.diff

2 Files Affected:

  • (modified) bolt/include/bolt/Profile/YAMLProfileReader.h (+13)
  • (modified) bolt/lib/Profile/YAMLProfileReader.cpp (+49-38)
diff --git a/bolt/include/bolt/Profile/YAMLProfileReader.h b/bolt/include/bolt/Profile/YAMLProfileReader.h
index 7a8aa176c30f1..a5bd3544bd999 100644
--- a/bolt/include/bolt/Profile/YAMLProfileReader.h
+++ b/bolt/include/bolt/Profile/YAMLProfileReader.h
@@ -73,6 +73,10 @@ class YAMLProfileReader : public ProfileReaderBase {
   bool parseFunctionProfile(BinaryFunction &Function,
                             const yaml::bolt::BinaryFunctionProfile &YamlBF);
 
+  /// Returns block cnt equality if IgnoreHash is true, otherwise, hash equality
+  bool profileMatches(const yaml::bolt::BinaryFunctionProfile &Profile,
+                      BinaryFunction &BF);
+
   /// Infer function profile from stale data (collected on older binaries).
   bool inferStaleProfile(BinaryFunction &Function,
                          const yaml::bolt::BinaryFunctionProfile &YamlBF);
@@ -80,6 +84,15 @@ class YAMLProfileReader : public ProfileReaderBase {
   /// Initialize maps for profile matching.
   void buildNameMaps(BinaryContext &BC);
 
+  /// Matches functions using exact name.
+  size_t matchWithExactName();
+
+  /// Matches function using LTO comomon name.
+  size_t matchWithLTOCommonName();
+
+  /// Matches functions using exact hash.
+  size_t matchWithHash(BinaryContext &BC);
+
   /// Update matched YAML -> BinaryFunction pair.
   void matchProfileToFunction(yaml::bolt::BinaryFunctionProfile &YamlBF,
                               BinaryFunction &BF) {
diff --git a/bolt/lib/Profile/YAMLProfileReader.cpp b/bolt/lib/Profile/YAMLProfileReader.cpp
index 554def697fa21..e8ce187367899 100644
--- a/bolt/lib/Profile/YAMLProfileReader.cpp
+++ b/bolt/lib/Profile/YAMLProfileReader.cpp
@@ -333,6 +333,12 @@ Error YAMLProfileReader::preprocessProfile(BinaryContext &BC) {
 
   return Error::success();
 }
+bool YAMLProfileReader::profileMatches(
+    const yaml::bolt::BinaryFunctionProfile &Profile, BinaryFunction &BF) {
+  if (opts::IgnoreHash)
+    return Profile.NumBasicBlocks == BF.size();
+  return Profile.Hash == static_cast<uint64_t>(BF.getHash());
+}
 
 bool YAMLProfileReader::mayHaveProfileData(const BinaryFunction &BF) {
   if (opts::MatchProfileWithFunctionHash)
@@ -350,44 +356,8 @@ bool YAMLProfileReader::mayHaveProfileData(const BinaryFunction &BF) {
   return false;
 }
 
-Error YAMLProfileReader::readProfile(BinaryContext &BC) {
-  if (opts::Verbosity >= 1) {
-    outs() << "BOLT-INFO: YAML profile with hash: ";
-    switch (YamlBP.Header.HashFunction) {
-    case HashFunction::StdHash:
-      outs() << "std::hash\n";
-      break;
-    case HashFunction::XXH3:
-      outs() << "xxh3\n";
-      break;
-    }
-  }
-  YamlProfileToFunction.resize(YamlBP.Functions.size() + 1);
-
-  auto profileMatches = [](const yaml::bolt::BinaryFunctionProfile &Profile,
-                           BinaryFunction &BF) {
-    if (opts::IgnoreHash)
-      return Profile.NumBasicBlocks == BF.size();
-    return Profile.Hash == static_cast<uint64_t>(BF.getHash());
-  };
-
-  uint64_t MatchedWithExactName = 0;
-  uint64_t MatchedWithHash = 0;
-  uint64_t MatchedWithLTOCommonName = 0;
-
-  // Computes hash for binary functions.
-  if (opts::MatchProfileWithFunctionHash) {
-    for (auto &[_, BF] : BC.getBinaryFunctions()) {
-      BF.computeHash(YamlBP.Header.IsDFSOrder, YamlBP.Header.HashFunction);
-    }
-  } else if (!opts::IgnoreHash) {
-    for (BinaryFunction *BF : ProfileBFs) {
-      if (!BF)
-        continue;
-      BF->computeHash(YamlBP.Header.IsDFSOrder, YamlBP.Header.HashFunction);
-    }
-  }
-
+size_t YAMLProfileReader::matchWithExactName() {
+  size_t MatchedWithExactName = 0;
   // This first pass assigns profiles that match 100% by name and by hash.
   for (auto [YamlBF, BF] : llvm::zip_equal(YamlBP.Functions, ProfileBFs)) {
     if (!BF)
@@ -402,10 +372,14 @@ Error YAMLProfileReader::readProfile(BinaryContext &BC) {
       ++MatchedWithExactName;
     }
   }
+  return MatchedWithExactName;
+}
 
+size_t YAMLProfileReader::matchWithHash(BinaryContext &BC) {
   // Iterates through profiled functions to match the first binary function with
   // the same exact hash. Serves to match identical, renamed functions.
   // Collisions are possible where multiple functions share the same exact hash.
+  size_t MatchedWithHash = 0;
   if (opts::MatchProfileWithFunctionHash) {
     DenseMap<size_t, BinaryFunction *> StrictHashToBF;
     StrictHashToBF.reserve(BC.getBinaryFunctions().size());
@@ -424,8 +398,12 @@ Error YAMLProfileReader::readProfile(BinaryContext &BC) {
       }
     }
   }
+  return MatchedWithHash;
+}
 
+size_t YAMLProfileReader::matchWithLTOCommonName() {
   // This second pass allows name ambiguity for LTO private functions.
+  size_t MatchedWithLTOCommonName = 0;
   for (const auto &[CommonName, LTOProfiles] : LTOCommonNameMap) {
     if (!LTOCommonNameFunctionMap.contains(CommonName))
       continue;
@@ -456,6 +434,39 @@ Error YAMLProfileReader::readProfile(BinaryContext &BC) {
       ++MatchedWithLTOCommonName;
     }
   }
+  return MatchedWithLTOCommonName;
+}
+
+Error YAMLProfileReader::readProfile(BinaryContext &BC) {
+  if (opts::Verbosity >= 1) {
+    outs() << "BOLT-INFO: YAML profile with hash: ";
+    switch (YamlBP.Header.HashFunction) {
+    case HashFunction::StdHash:
+      outs() << "std::hash\n";
+      break;
+    case HashFunction::XXH3:
+      outs() << "xxh3\n";
+      break;
+    }
+  }
+  YamlProfileToFunction.resize(YamlBP.Functions.size() + 1);
+
+  // Computes hash for binary functions.
+  if (opts::MatchProfileWithFunctionHash) {
+    for (auto &[_, BF] : BC.getBinaryFunctions()) {
+      BF.computeHash(YamlBP.Header.IsDFSOrder, YamlBP.Header.HashFunction);
+    }
+  } else if (!opts::IgnoreHash) {
+    for (BinaryFunction *BF : ProfileBFs) {
+      if (!BF)
+        continue;
+      BF->computeHash(YamlBP.Header.IsDFSOrder, YamlBP.Header.HashFunction);
+    }
+  }
+
+  size_t MatchedWithExactName = matchWithExactName();
+  size_t MatchedWithHash = matchWithHash(BC);
+  size_t MatchedWithLTOCommonName = matchWithLTOCommonName();
 
   for (auto [YamlBF, BF] : llvm::zip_equal(YamlBP.Functions, ProfileBFs))
     if (!YamlBF.Used && BF && !ProfiledFunctions.count(BF))

Copy link
Member

@dcci dcci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a couple of general comments about this.
Can you also please add a description explaining what this patch does?

@dcci
Copy link
Member

dcci commented Jul 3, 2024

I have a couple of general comments about this. Can you also please add a description explaining what this patch does?

i.e. why we're refactoring these functions.

Created using spr 1.3.4
Copy link
Contributor

@aaupov aaupov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG % nit

bolt/lib/Profile/YAMLProfileReader.cpp Outdated Show resolved Hide resolved
bolt/lib/Profile/YAMLProfileReader.cpp Outdated Show resolved Hide resolved
bolt/lib/Profile/YAMLProfileReader.cpp Outdated Show resolved Hide resolved
sayhaan and others added 3 commits July 5, 2024 11:11
Created using spr 1.3.4

[skip ci]
Created using spr 1.3.4
Created using spr 1.3.4
aaupov and others added 2 commits July 5, 2024 12:55
Created using spr 1.3.4

[skip ci]
Created using spr 1.3.4
MaskRay and others added 2 commits July 5, 2024 14:42
Created using spr 1.3.4

[skip ci]
Created using spr 1.3.4
@shawbyoung shawbyoung changed the base branch from users/shawbyoung/spr/main.boltnfc-refactor-function-matching to main July 5, 2024 21:43
@shawbyoung shawbyoung merged commit 37bee25 into main Jul 5, 2024
5 of 6 checks passed
@shawbyoung shawbyoung deleted the users/shawbyoung/spr/boltnfc-refactor-function-matching branch July 5, 2024 21:44
kbluck pushed a commit to kbluck/llvm-project that referenced this pull request Jul 6, 2024
Moved function matching techniques into separate helper functions for
ease of understanding and to make space for additional function 
matching techniques to be added (e.g. call graph function matching).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants