GSym aggregated output to JSON file

[kevinfrei]

In order to make tooling around dwarf health easier, I've added an --json-summary-file option to llvm-gsymutil that will spit out error summary data with counts to a JSON file.

I've added the same capability to llvm-dwarfdump in a different PR.

The format of the json is:

{ 
  "error-categories": { 
    "<first category description>": {"count": 1234},
    "<next category description>": {"count":4321}
  },
  "error-count": 5555
}

for a clean run:

{ 
  "error-categories": {},
  "error-count": 0
}

[llvmbot]

@llvm/pr-subscribers-debuginfo

Author: Kevin Frei (kevinfrei)

Changes

In order to make tooling around dwarf health easier, I've added an --aggregate-output-file file to gsymutil that will spit out error summary data with counts to a JSON file.

I've added the same capability to llvm-dwarfdump --verify in a different PR.

---

Full diff: https://github.com/llvm/llvm-project/pull/81763.diff

2 Files Affected:

• (modified) llvm/tools/llvm-gsymutil/Opts.td (+3)
• (modified) llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp (+23)

diff --git a/llvm/tools/llvm-gsymutil/Opts.td b/llvm/tools/llvm-gsymutil/Opts.td
index 740291479f9323..00e81a9f85aa70 100644
--- a/llvm/tools/llvm-gsymutil/Opts.td
+++ b/llvm/tools/llvm-gsymutil/Opts.td
@@ -35,3 +35,6 @@ defm address : Eq<"address", "Lookup an address in a GSYM file">;
 def addresses_from_stdin :
   FF<"addresses-from-stdin",
      "Lookup addresses in a GSYM file that are read from stdin\nEach input line is expected to be of the following format: <addr> <gsym-path>">;
+defm aggregate_error_file :
+  Eq<"aggregate-error-file",
+     "Output any aggregated errors into the file specified in JSON format.">;
diff --git a/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp b/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp
index 2de9c76fd68c0c..1068ac4e39b266 100644
--- a/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp
+++ b/llvm/tools/llvm-gsymutil/llvm-gsymutil.cpp
@@ -7,6 +7,7 @@
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/StringExtras.h"
 #include "llvm/DebugInfo/DIContext.h"
 #include "llvm/DebugInfo/DWARF/DWARFContext.h"
 #include "llvm/Object/Archive.h"
@@ -87,6 +88,7 @@ static std::vector<std::string> InputFilenames;
 static std::string ConvertFilename;
 static std::vector<std::string> ArchFilters;
 static std::string OutputFilename;
+static std::string AggregateJsonFile;
 static bool Verify;
 static unsigned NumThreads;
 static uint64_t SegmentSize;
@@ -138,6 +140,9 @@ static void parseArgs(int argc, char **argv) {
   if (const llvm::opt::Arg *A = Args.getLastArg(OPT_out_file_EQ))
     OutputFilename = A->getValue();
 
+  if (const llvm::opt::Arg *A = Args.getLastArg(OPT_aggregate_error_file_EQ))
+    AggregateJsonFile = A->getValue();
+
   Verify = Args.hasArg(OPT_verify);
 
   if (const llvm::opt::Arg *A = Args.getLastArg(OPT_num_threads_EQ)) {
@@ -515,10 +520,28 @@ int llvm_gsymutil_main(int argc, char **argv, const llvm::ToolContext &) {
     // Call error() if we have an error and it will exit with a status of 1
     if (auto Err = convertFileToGSYM(Aggregation))
       error("DWARF conversion failed: ", std::move(Err));
+
     // Report the errors from aggregator:
     Aggregation.EnumerateResults([&](StringRef category, unsigned count) {
       OS << category << " occurred " << count << " time(s)\n";
     });
+    if (!AggregateJsonFile.empty()) {
+      std::error_code EC;
+      raw_fd_ostream JsonStream(AggregateJsonFile, EC,
+                                sys::fs::OF_Text | sys::fs::OF_None);
+      if (EC) {
+        OS << "error opening aggregate error json file '" << AggregateJsonFile
+           << "' for writing: " << EC.message() << '\n';
+        return 1;
+      }
+      JsonStream << "{\"errors\":[\n";
+      Aggregation.EnumerateResults([&](StringRef category, unsigned count) {
+        JsonStream << "\"category\":\"";
+        llvm::printEscapedString(category, JsonStream);
+        JsonStream << "\",\"count\":" << count;
+      });
+      JsonStream << "]}\n";
+    }
     return 0;
   }
 

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[kevinfrei]

@clayborg Any feedback on this one?

[clayborg]

I will hold off on any comments here while we iterate on the llvm-dwarfdump version.

[kevinfrei]

I will hold off on any comments here while we iterate on the llvm-dwarfdump version.

I'll keep the two in sync...

[kevinfrei]

I've run a full check-llvm on my linux box, if that helps raise confidence. There appears to be weird test flakiness in the buildkite validation ☹️