[Clang] [Diagnostics] Simplify filenames that contain '..' #143520

Sirraide · 2025-06-10T12:43:55Z

This can significantly shorten file paths to standard library headers, e.g. on my system, <ranges> is currently printed as

/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/ranges

but with this change, we instead print

/usr/include/c++/15/ranges

This is of course just a heuristic, so there will definitely be paths that get longer as a result of this, but it helps with the standard library, and a lot of the diagnostics we print tend to originate in standard library headers (especially if you include notes listing overload candidates etc.). Update: We now always use whichever path ends up being shorter.

@AaronBallman pointed out that this might be problematic for network file systems since path resolution might take a while, so this is enabled only for paths that are part of a local filesystem.

The file names are cached in TextDiagnostic. While we could move it up into e.g. TextDiagnosticPrinter, DiagnosticsEngine, or maybe even the FileManager, to me this seems like something that we mainly care about when printing to the terminal (other diagnostics consumers probably don’t mind receiving the original file path). Moreover, this is already where we handle -fdiagnostics-absolute-paths or whatever that flag is called again.

llvmbot · 2025-06-10T12:44:32Z

@llvm/pr-subscribers-clang-tools-extra
@llvm/pr-subscribers-clang-tidy

@llvm/pr-subscribers-clang

Author: None (Sirraide)

Changes

This can significantly shorten file paths to standard library headers, e.g. on my system, <ranges> is currently printed as

/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/ranges

but with this change, we instead print

/usr/include/c++/15/ranges

This is of course just a heuristic, so there will definitely be paths that get longer as a result of this, but it helps with the standard library, and a lot of the diagnostics we print tend to originate in standard library headers (especially if you include notes listing overload candidates etc.).

@AaronBallman pointed out that this might be problematic for network file systems since path resolution might take a while, so this is enabled only for paths that are part of a local filesystem.

The file names are cached in TextDiagnostic. While we could move it up into e.g. TextDiagnosticPrinter, DiagnosticsEngine, or maybe even the FileManager, to me this seems like something that we mainly care about when printing to the terminal (other diagnostics consumers probably don’t mind receiving the original file path). Moreover, this is already where we handle -fdiagnostics-absolute-paths or whatever that flag is called again.

Full diff: https://github.com/llvm/llvm-project/pull/143520.diff

2 Files Affected:

(modified) clang/include/clang/Frontend/TextDiagnostic.h (+1)
(modified) clang/lib/Frontend/TextDiagnostic.cpp (+20-12)

diff --git a/clang/include/clang/Frontend/TextDiagnostic.h b/clang/include/clang/Frontend/TextDiagnostic.h
index e2e88d4d648a2..9c77bc3e00e19 100644
--- a/clang/include/clang/Frontend/TextDiagnostic.h
+++ b/clang/include/clang/Frontend/TextDiagnostic.h
@@ -35,6 +35,7 @@ namespace clang {
 class TextDiagnostic : public DiagnosticRenderer {
   raw_ostream &OS;
   const Preprocessor *PP;
+  llvm::StringMap<SmallString<128>> SimplifiedFileNameCache;
 
 public:
   TextDiagnostic(raw_ostream &OS, const LangOptions &LangOpts,
diff --git a/clang/lib/Frontend/TextDiagnostic.cpp b/clang/lib/Frontend/TextDiagnostic.cpp
index b9e681b52e509..edbad42b39950 100644
--- a/clang/lib/Frontend/TextDiagnostic.cpp
+++ b/clang/lib/Frontend/TextDiagnostic.cpp
@@ -738,12 +738,20 @@ void TextDiagnostic::printDiagnosticMessage(raw_ostream &OS,
 }
 
 void TextDiagnostic::emitFilename(StringRef Filename, const SourceManager &SM) {
-#ifdef _WIN32
-  SmallString<4096> TmpFilename;
-#endif
-  if (DiagOpts.AbsolutePath) {
-    auto File = SM.getFileManager().getOptionalFileRef(Filename);
-    if (File) {
+  auto File = SM.getFileManager().getOptionalFileRef(Filename);
+
+  // Try to simplify paths that contain '..' in any case since paths to
+  // standard library headers especially tend to get quite long otherwise.
+  // Only do that for local filesystems though to avoid slowing down
+  // compilation too much.
+  auto AlwaysSimplify = [&] {
+    return File->getName().contains("..") &&
+           llvm::sys::fs::is_local(File->getName());
+  };
+
+  if (File && (DiagOpts.AbsolutePath || AlwaysSimplify())) {
+    SmallString<128> &CacheEntry = SimplifiedFileNameCache[Filename];
+    if (CacheEntry.empty()) {
       // We want to print a simplified absolute path, i. e. without "dots".
       //
       // The hardest part here are the paths like "<part1>/<link>/../<part2>".
@@ -759,15 +767,15 @@ void TextDiagnostic::emitFilename(StringRef Filename, const SourceManager &SM) {
       // on Windows we can just use llvm::sys::path::remove_dots(), because,
       // on that system, both aforementioned paths point to the same place.
 #ifdef _WIN32
-      TmpFilename = File->getName();
-      llvm::sys::fs::make_absolute(TmpFilename);
-      llvm::sys::path::native(TmpFilename);
-      llvm::sys::path::remove_dots(TmpFilename, /* remove_dot_dot */ true);
-      Filename = StringRef(TmpFilename.data(), TmpFilename.size());
+      CacheEntry = File->getName();
+      llvm::sys::fs::make_absolute(CacheEntry);
+      llvm::sys::path::native(CacheEntry);
+      llvm::sys::path::remove_dots(CacheEntry, /* remove_dot_dot */ true);
 #else
-      Filename = SM.getFileManager().getCanonicalName(*File);
+      CacheEntry = SM.getFileManager().getCanonicalName(*File);
 #endif
     }
+    Filename = CacheEntry;
   }
 
   OS << Filename;

Sirraide · 2025-06-10T12:45:01Z

This is of course just a heuristic, so there will definitely be paths that get longer as a result of this

Actually, it just occurred to me that we could just cache whichever path ends up being shorter, the original one or the resolved one.

Sirraide · 2025-06-10T12:48:30Z

I’m not exactly sure how to test this change since this is not only platform-dependent but also path-dependent since we may end up producing absolute paths here.

AaronBallman · 2025-06-10T15:45:19Z

The file names are cached in TextDiagnostic. While we could move it up into e.g. TextDiagnosticPrinter, DiagnosticsEngine, or maybe even the FileManager, to me this seems like something that we mainly care about when printing to the terminal (other diagnostics consumers probably don’t mind receiving the original file path). Moreover, this is already where we handle -fdiagnostics-absolute-paths or whatever that flag is called again.

The downside to it being in TextDiagnostic is that consumers then all have to normalize the path themselves (some file system APIs on some systems are better about relative paths than others). If the paths are always equivalent, it might be kinder to pass the resolved path. WDYT?

AaronBallman · 2025-06-10T15:46:21Z

I’m not exactly sure how to test this change since this is not only platform-dependent but also path-dependent since we may end up producing absolute paths here.

I think this is a case where maybe we want to use unit tests. We have clang/unittests/Basic/DiagnosticTest.cpp or FileManagerTest.cpp already, so perhaps in one of those (depending on where the functionality ends up living)?

clang/lib/Frontend/TextDiagnostic.cpp

Sirraide · 2025-06-10T16:57:31Z

The downside to it being in TextDiagnostic is that consumers then all have to normalize the path themselves (some file system APIs on some systems are better about relative paths than others). If the paths are always equivalent, it might be kinder to pass the resolved path. WDYT?

I mean, that’s also true I suppose; the only thing is then that we’d be normalising them twice if -fdiagnostics-absolute-paths is passed—unless we move the handling for that elsewhere as well, but now that’s dependent on the diagnostic options, so it probably shouldn’t be in FileManager—which leaves DiagnosticsEngine? But consumers don’t generally have access to the DiagnosticsEngine, so it’d have to be in the FileManager after all.

I guess we could always compute both the absolute and the ‘short’ path for a file whenever FileManager opens one so that they’re always available. But that might have some impact on performance (though I guess this is a perf/ branch already so we can try and see how it goes)?

@AaronBallman Thoughts?

AaronBallman · 2025-06-10T17:31:27Z

The downside to it being in TextDiagnostic is that consumers then all have to normalize the path themselves (some file system APIs on some systems are better about relative paths than others). If the paths are always equivalent, it might be kinder to pass the resolved path. WDYT?

I mean, that’s also true I suppose; the only thing is then that we’d be normalising them twice if -fdiagnostics-absolute-paths is passed—unless we move the handling for that elsewhere as well, but now that’s dependent on the diagnostic options, so it probably shouldn’t be in FileManager—which leaves DiagnosticsEngine? But consumers don’t generally have access to the DiagnosticsEngine, so it’d have to be in the FileManager after all.

We definitely don't want to normalize twice. Could we parameterize FileManager so we don't have to have it directly depend on diagnostic options?

I guess we could always compute both the absolute and the ‘short’ path for a file whenever FileManager opens one so that they’re always available. But that might have some impact on performance (though I guess this is a perf/ branch already so we can try and see how it goes)?

I think we could try it to see how it goes in terms of performance. Again, I think I'd be most worried about network builds -- I would expect a measurable different in performance even if there are no diagnostics issued just because we need the file information for SourceManager.

Sirraide · 2025-06-10T17:34:59Z

We definitely don't want to normalize twice. Could we parameterize FileManager so we don't have to have it directly depend on diagnostic options?

One idea I just had is we could do something like:

enum class DiagnosticFileNameMode {
  Unmodified, // As specified by the user
  Canonical,  // Absolute path
  Short,      // Whichever is shorter
}

class FileManager {
  // ...
  StringRef getFileNameForDiagnostic(DiagnosticFileNameMode Mode);
};

And then have separate caches in FileManager for each kind of DiagnosticsFileNameMode and compute the corresponding file name lazily the first time it’s requested.

AaronBallman · 2025-06-10T17:40:29Z

We definitely don't want to normalize twice. Could we parameterize FileManager so we don't have to have it directly depend on diagnostic options?

One idea I just had is we could do something like:
enum class DiagnosticFileNameMode {
  Unmodified, // As specified by the user
  Canonical,  // Absolute path
  Short,      // Whichever is shorter
}

class FileManager {
  // ...
  StringRef getFileNameForDiagnostic(DiagnosticFileNameMode Mode);
};
And then have separate caches in FileManager for each kind of DiagnosticsFileNameMode and compute the corresponding file name lazily the first time it’s requested.

I think that approach makes sense! Thought "short path" means something different to those of us old enough to remember DOS 8.3 filenames. :-D

Sirraide · 2025-06-10T17:43:36Z

I think that approach makes sense! Thought "short path" means something different to those of us old enough to remember DOS 8.3 filenames. :-D

Ha, those I’m not planning to add support for thankfully...

Sirraide · 2025-06-10T18:19:24Z

Actually, we could also just put it in SourceManager because that already has a reference to the DiagnosticsEngine and then a single getNameForDiagnostic(StringRef Filename) function would do.

Sirraide · 2025-06-10T18:37:04Z

Actually, we could also just put it in SourceManager because that already has a reference to the DiagnosticsEngine and then a single getNameForDiagnostic(StringRef Filename) function would do.

I’ve done that. Also, SARIFDiagnostic::emitFilename() was just a copy-pasted version of TextDiagnostic::emitFilename(), so I’ve updated it to use the new function as well.

Sirraide · 2025-06-13T18:36:01Z

Ok, I’ve fixed a crash involving a dangling reference and also disabled the check for a local filesystem on windows. It also seems that we do have a single test for this already.

AaronBallman

In general, I'm in favor of this patch. Precommit CI found relevant failures that need to be fixed, but I think this is otherwise good to go once those are addressed.

clang/include/clang/Frontend/TextDiagnostic.h

cor3ntin · 2025-07-02T07:48:30Z

clang/lib/Basic/SourceManager.cpp

+#ifdef _WIN32
+  TempBuf = File->getName();
+  llvm::sys::fs::make_absolute(TempBuf);
+  llvm::sys::path::native(TempBuf);
+  llvm::sys::path::remove_dots(TempBuf, /* remove_dot_dot */ true);
+#else
+  TempBuf = getFileManager().getCanonicalName(*File);
+#endif


Sort of pre-existing, but I am not sure doing something different for windows actually make sense.
Symlinks on Windows exist, they are just very rare.

Hmm, I was interpreting the comment above this to mean that Windows itself doesn’t care about symlinks when resolving .. and actually just deletes preceding path segments, but I’m not much of a Windows person so I don’t know to be fair...

cor3ntin · 2025-07-02T07:53:03Z

clang/lib/Basic/SourceManager.cpp

+  if (!SimplifyPath)
+    return Filename;


I wonder if it would be more efficient to check the map first

Hmm, not sure. This is a perf/ branch anyway so when I’m done fixing the tests (I think it’s just a clang-tidy test that’s left at this point?) we can just check if there are any regressions and try doing this instead if so.

Sirraide · 2025-07-02T10:49:45Z

Ok, looks like the clang-tidy test failure is related to the -header-filter option:

// Check that `-header-filter` operates on the same file paths as paths in
// diagnostics printed by ClangTidy.
#include "dir1/dir2/../header_alias.h"
// CHECK_HEADER_ALIAS: dir1/dir2/../header_alias.h:1:11: warning: single-argument constructors

So, I guess my question is now, should the header filter apply to the original filename, the simplified filename, or both?

@AaronBallman Thoughts? Or alternatively who do best ping for clang-tidy related questions?

Sirraide · 2025-07-02T10:56:45Z

should the header filter apply to the original filename,

I mean, I guess this beacuse it’s what the user specified and it’s what we’re currently doing? It’s just that the end result might be weird, e.g. if a user writes -exclude-header-filter="a/foo.h" and then we print diagnostics in a/foo.h because it was actually included via "a/b/../foo.h" (assuming I’m not misinterpreting what this option does), but maybe that’s ok?

AaronBallman · 2025-07-02T18:17:01Z

Ok, looks like the clang-tidy test failure is related to the -header-filter option:
// Check that `-header-filter` operates on the same file paths as paths in
// diagnostics printed by ClangTidy.
#include "dir1/dir2/../header_alias.h"
// CHECK_HEADER_ALIAS: dir1/dir2/../header_alias.h:1:11: warning: single-argument constructors
So, I guess my question is now, should the header filter apply to the original filename, the simplified filename, or both?

@AaronBallman Thoughts? Or alternatively who do best ping for clang-tidy related questions?

CC @5chmidti @PiotrZSL @HerrCai0907 @LegalizeAdulthood for more opinions on this

I would naively expect that I'm giving the tool a path, the tool will resolve all symlinks and relative parts, etc for me same as it would do when specifying the file to compile/tidy.

LegalizeAdulthood · 2025-07-03T01:51:13Z

My expectation would be that if I specify a header filter I'm not going to use weird paths like a/b/../foo.h, but just a/foo.h because that is where foo.h lives.

AaronBallman · 2025-07-03T11:54:06Z

My expectation would be that if I specify a header filter I'm not going to use weird paths like a/b/../foo.h, but just a/foo.h because that is where foo.h lives.

What about symlinks though? Would you expect that passing path/to/file fails because path is a symlink and you should have specified /var/foo/bar/to/file? (It's basically the same problem -- do we make the user pass the resolved path or do we canonicalize the path for the user and use that to do the filtering?)

LegalizeAdulthood · 2025-07-03T15:07:37Z

What about symlinks though?

In that case WYGIWYD (what you get is what you deserve) applies.

Why would relative path simplification care about symlinks? My understanding is that this change simplies a/b/../ to a/ not chasing through arbitrary symlinks.

Sirraide · 2025-07-04T13:34:50Z

In that case WYGIWYD (what you get is what you deserve) applies.

Sgtm; I’ll leave the clang-tidy side of things as-is then and just update the test to reflect this.

Sirraide · 2025-07-04T14:02:42Z

Ok; that’s done now. Thanks to that clang-tidy test I’ve also managed to figure out how to writer a proper regression test for this patch (only for linux though to be fair; not sure we support Windows-only tests that create files etc., and symlinks on windows require administrator permissions anyway from what I know).

Sirraide · 2025-07-04T14:06:57Z

Also, looking at the compile-time tracker page for this branch, it doesn’t seem to have any performance impact.

AaronBallman

LGTM!

Sirraide · 2025-07-07T15:27:41Z

Windows CI failures seem unrelated.

I forgot to include a release note in #143520, and it also ocurred to me that while #143514 is technically a bugfix in LLVM/Support, I think we should have one for it as well.

[Clang] [Diagnostics] Simplify filenames that contain '..'

15c0a79

Sirraide requested a review from AaronBallman June 10, 2025 12:43

Sirraide added the clang:diagnostics New/improved warning or error message in Clang, but not in clang-tidy or static analyzer label Jun 10, 2025

llvmbot added the clang Clang issues not falling into any other category label Jun 10, 2025

Use whichever path ends up being shorter

97c0acc

AaronBallman reviewed Jun 10, 2025

View reviewed changes

clang/lib/Frontend/TextDiagnostic.cpp Outdated Show resolved Hide resolved

Move path computation into SourceManager

529aac1

llvmbot added the clang:frontend Language frontend issues, e.g. anything involving "Sema" label Jun 10, 2025

Fix crash and disable nfs check on windows

5d30887

AaronBallman reviewed Jun 26, 2025

View reviewed changes

cor3ntin reviewed Jul 2, 2025

View reviewed changes

remove unused member

c0f56c3

fix clang-tidy test

a91e711

llvmbot added clang-tools-extra clang-tidy labels Jul 4, 2025

add test

bd40f82

AaronBallman approved these changes Jul 7, 2025

View reviewed changes

Merge branch 'main' into perf/simplify-paths

7cf1829

Sirraide merged commit e3e7393 into llvm:main Jul 7, 2025
9 checks passed

Sirraide added a commit to Sirraide/llvm-project that referenced this pull request Jul 8, 2025

[Clang] [Docs] Add release notes for llvm#143514 and llvm#143520

0862ba0

Sirraide mentioned this pull request Jul 8, 2025

[Clang] [Docs] Add release notes for #143514 and #143520 #147562

Merged

[Clang] [Diagnostics] Simplify filenames that contain '..' #143520

[Clang] [Diagnostics] Simplify filenames that contain '..' #143520

Conversation

Sirraide commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

AaronBallman commented Jun 10, 2025

Uh oh!

AaronBallman commented Jun 10, 2025

Uh oh!

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

AaronBallman commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

AaronBallman commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 10, 2025

Uh oh!

Sirraide commented Jun 13, 2025

Uh oh!

AaronBallman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cor3ntin Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Sirraide Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

cor3ntin Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Sirraide Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Sirraide commented Jul 2, 2025

Uh oh!

Sirraide commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AaronBallman commented Jul 2, 2025

Uh oh!

LegalizeAdulthood commented Jul 3, 2025

Uh oh!

AaronBallman commented Jul 3, 2025

Uh oh!

LegalizeAdulthood commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Sirraide commented Jul 4, 2025

Uh oh!

Sirraide commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Sirraide commented Jul 4, 2025

Uh oh!

AaronBallman left a comment

Choose a reason for hiding this comment

Uh oh!

Sirraide commented Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

Sirraide commented Jun 10, 2025 •

edited

Loading

llvmbot commented Jun 10, 2025 •

edited

Loading

Sirraide commented Jul 2, 2025 •

edited

Loading

LegalizeAdulthood commented Jul 3, 2025 •

edited

Loading

Sirraide commented Jul 4, 2025 •

edited

Loading