[support] Use VFS in `SourceMgr` for loading includes #162903

jansvoboda11 · 2025-10-10T18:45:03Z

Most SourceMgr clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a vfs::FileSystem instance into SourceMgr.

llvmbot · 2025-10-10T18:45:41Z

@llvm/pr-subscribers-llvm-mc
@llvm/pr-subscribers-clang
@llvm/pr-subscribers-mlir
@llvm/pr-subscribers-mlir-core

@llvm/pr-subscribers-llvm-support

Author: Jan Svoboda (jansvoboda11)

Changes

Most SourceMgr clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a vfs::FileSystem instance into SourceMgr.

Full diff: https://github.com/llvm/llvm-project/pull/162903.diff

4 Files Affected:

(modified) llvm/include/llvm/Support/SourceMgr.h (+19-4)
(modified) llvm/lib/Support/SourceMgr.cpp (+22-2)
(modified) llvm/lib/TableGen/Main.cpp (+3)
(modified) llvm/lib/TableGen/Parser.cpp (+2)

diff --git a/llvm/include/llvm/Support/SourceMgr.h b/llvm/include/llvm/Support/SourceMgr.h
index 5637b64c4cbfd..8320006ff5f6e 100644
--- a/llvm/include/llvm/Support/SourceMgr.h
+++ b/llvm/include/llvm/Support/SourceMgr.h
@@ -15,6 +15,7 @@
 #ifndef LLVM_SUPPORT_SOURCEMGR_H
 #define LLVM_SUPPORT_SOURCEMGR_H
 
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/MemoryBuffer.h"
@@ -23,6 +24,10 @@
 
 namespace llvm {
 
+namespace vfs {
+class FileSystem;
+} // end namespace vfs
+
 class raw_ostream;
 class SMDiagnostic;
 class SMFixIt;
@@ -91,15 +96,25 @@ class SourceMgr {
   DiagHandlerTy DiagHandler = nullptr;
   void *DiagContext = nullptr;
 
+  // Optional file system for finding include files.
+  IntrusiveRefCntPtr<vfs::FileSystem> FS;
+
   bool isValidBufferID(unsigned i) const { return i && i <= Buffers.size(); }
 
 public:
-  SourceMgr() = default;
+  /// Create new source manager without support for include files.
+  SourceMgr();
+  /// Create new source manager with the capability of finding include files
+  /// via the provided file system.
+  explicit SourceMgr(IntrusiveRefCntPtr<vfs::FileSystem> FS);
   SourceMgr(const SourceMgr &) = delete;
   SourceMgr &operator=(const SourceMgr &) = delete;
-  SourceMgr(SourceMgr &&) = default;
-  SourceMgr &operator=(SourceMgr &&) = default;
-  ~SourceMgr() = default;
+  SourceMgr(SourceMgr &&);
+  SourceMgr &operator=(SourceMgr &&);
+  ~SourceMgr();
+
+  IntrusiveRefCntPtr<vfs::FileSystem> getVirtualFileSystem() const;
+  void setVirtualFileSystem(IntrusiveRefCntPtr<vfs::FileSystem> FS);
 
   /// Return the include directories of this source manager.
   ArrayRef<std::string> getIncludeDirs() const { return IncludeDirectories; }
diff --git a/llvm/lib/Support/SourceMgr.cpp b/llvm/lib/Support/SourceMgr.cpp
index a43cf37a79824..f2bbaab23ed7b 100644
--- a/llvm/lib/Support/SourceMgr.cpp
+++ b/llvm/lib/Support/SourceMgr.cpp
@@ -24,6 +24,7 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SMLoc.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
@@ -38,6 +39,22 @@ using namespace llvm;
 
 static const size_t TabStop = 8;
 
+// Out of line to avoid needing definition of vfs::FileSystem in header.
+SourceMgr::SourceMgr() = default;
+SourceMgr::SourceMgr(IntrusiveRefCntPtr<vfs::FileSystem> FS)
+    : FS(std::move(FS)) {}
+SourceMgr::SourceMgr(SourceMgr &&) = default;
+SourceMgr &SourceMgr::operator=(SourceMgr &&) = default;
+SourceMgr::~SourceMgr() = default;
+
+IntrusiveRefCntPtr<vfs::FileSystem> SourceMgr::getVirtualFileSystem() const {
+  return FS;
+}
+
+void SourceMgr::setVirtualFileSystem(IntrusiveRefCntPtr<vfs::FileSystem> FS) {
+  this->FS = std::move(FS);
+}
+
 unsigned SourceMgr::AddIncludeFile(const std::string &Filename,
                                    SMLoc IncludeLoc,
                                    std::string &IncludedFile) {
@@ -52,8 +69,11 @@ unsigned SourceMgr::AddIncludeFile(const std::string &Filename,
 ErrorOr<std::unique_ptr<MemoryBuffer>>
 SourceMgr::OpenIncludeFile(const std::string &Filename,
                            std::string &IncludedFile) {
+  if (!FS)
+    reportFatalInternalError("Opening include file from SourceMgr without VFS");
+
   ErrorOr<std::unique_ptr<MemoryBuffer>> NewBufOrErr =
-      MemoryBuffer::getFile(Filename);
+      FS->getBufferForFile(Filename);
 
   SmallString<64> Buffer(Filename);
   // If the file didn't exist directly, see if it's in an include path.
@@ -61,7 +81,7 @@ SourceMgr::OpenIncludeFile(const std::string &Filename,
        ++i) {
     Buffer = IncludeDirectories[i];
     sys::path::append(Buffer, Filename);
-    NewBufOrErr = MemoryBuffer::getFile(Buffer);
+    NewBufOrErr = FS->getBufferForFile(Buffer);
   }
 
   if (NewBufOrErr)
diff --git a/llvm/lib/TableGen/Main.cpp b/llvm/lib/TableGen/Main.cpp
index 42043f70768c5..f61f50aa6c0a3 100644
--- a/llvm/lib/TableGen/Main.cpp
+++ b/llvm/lib/TableGen/Main.cpp
@@ -26,6 +26,7 @@
 #include "llvm/Support/SMLoc.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/TableGen/Error.h"
 #include "llvm/TableGen/Record.h"
@@ -129,6 +130,8 @@ int llvm::TableGenMain(const char *argv0,
   // it later.
   SrcMgr.setIncludeDirs(IncludeDirs);
 
+  SrcMgr.setVirtualFileSystem(vfs::getRealFileSystem());
+
   TGParser Parser(SrcMgr, MacroNames, Records, NoWarnOnUnusedTemplateArgs);
 
   if (Parser.ParseFile())
diff --git a/llvm/lib/TableGen/Parser.cpp b/llvm/lib/TableGen/Parser.cpp
index 2c3726a339bb8..cdf4d013f4bc0 100644
--- a/llvm/lib/TableGen/Parser.cpp
+++ b/llvm/lib/TableGen/Parser.cpp
@@ -9,6 +9,7 @@
 #include "llvm/TableGen/Parser.h"
 #include "TGParser.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/TableGen/Record.h"
 
 using namespace llvm;
@@ -20,6 +21,7 @@ bool llvm::TableGenParseFile(SourceMgr &InputSrcMgr, RecordKeeper &Records) {
   // this reliance, we could drop all of this.
   SrcMgr = SourceMgr();
   SrcMgr.takeSourceBuffersFrom(InputSrcMgr);
+  SrcMgr.setVirtualFileSystem(InputSrcMgr.getVirtualFileSystem());
   SrcMgr.setIncludeDirs(InputSrcMgr.getIncludeDirs());
   SrcMgr.setDiagHandler(InputSrcMgr.getDiagHandler(),
                         InputSrcMgr.getDiagContext());

llvmbot · 2025-10-10T18:45:41Z

@llvm/pr-subscribers-tablegen

Author: Jan Svoboda (jansvoboda11)

Changes

Most SourceMgr clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a vfs::FileSystem instance into SourceMgr.

Full diff: https://github.com/llvm/llvm-project/pull/162903.diff

4 Files Affected:

(modified) llvm/include/llvm/Support/SourceMgr.h (+19-4)
(modified) llvm/lib/Support/SourceMgr.cpp (+22-2)
(modified) llvm/lib/TableGen/Main.cpp (+3)
(modified) llvm/lib/TableGen/Parser.cpp (+2)

diff --git a/llvm/include/llvm/Support/SourceMgr.h b/llvm/include/llvm/Support/SourceMgr.h
index 5637b64c4cbfd..8320006ff5f6e 100644
--- a/llvm/include/llvm/Support/SourceMgr.h
+++ b/llvm/include/llvm/Support/SourceMgr.h
@@ -15,6 +15,7 @@
 #ifndef LLVM_SUPPORT_SOURCEMGR_H
 #define LLVM_SUPPORT_SOURCEMGR_H
 
+#include "llvm/ADT/IntrusiveRefCntPtr.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/MemoryBuffer.h"
@@ -23,6 +24,10 @@
 
 namespace llvm {
 
+namespace vfs {
+class FileSystem;
+} // end namespace vfs
+
 class raw_ostream;
 class SMDiagnostic;
 class SMFixIt;
@@ -91,15 +96,25 @@ class SourceMgr {
   DiagHandlerTy DiagHandler = nullptr;
   void *DiagContext = nullptr;
 
+  // Optional file system for finding include files.
+  IntrusiveRefCntPtr<vfs::FileSystem> FS;
+
   bool isValidBufferID(unsigned i) const { return i && i <= Buffers.size(); }
 
 public:
-  SourceMgr() = default;
+  /// Create new source manager without support for include files.
+  SourceMgr();
+  /// Create new source manager with the capability of finding include files
+  /// via the provided file system.
+  explicit SourceMgr(IntrusiveRefCntPtr<vfs::FileSystem> FS);
   SourceMgr(const SourceMgr &) = delete;
   SourceMgr &operator=(const SourceMgr &) = delete;
-  SourceMgr(SourceMgr &&) = default;
-  SourceMgr &operator=(SourceMgr &&) = default;
-  ~SourceMgr() = default;
+  SourceMgr(SourceMgr &&);
+  SourceMgr &operator=(SourceMgr &&);
+  ~SourceMgr();
+
+  IntrusiveRefCntPtr<vfs::FileSystem> getVirtualFileSystem() const;
+  void setVirtualFileSystem(IntrusiveRefCntPtr<vfs::FileSystem> FS);
 
   /// Return the include directories of this source manager.
   ArrayRef<std::string> getIncludeDirs() const { return IncludeDirectories; }
diff --git a/llvm/lib/Support/SourceMgr.cpp b/llvm/lib/Support/SourceMgr.cpp
index a43cf37a79824..f2bbaab23ed7b 100644
--- a/llvm/lib/Support/SourceMgr.cpp
+++ b/llvm/lib/Support/SourceMgr.cpp
@@ -24,6 +24,7 @@
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SMLoc.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
@@ -38,6 +39,22 @@ using namespace llvm;
 
 static const size_t TabStop = 8;
 
+// Out of line to avoid needing definition of vfs::FileSystem in header.
+SourceMgr::SourceMgr() = default;
+SourceMgr::SourceMgr(IntrusiveRefCntPtr<vfs::FileSystem> FS)
+    : FS(std::move(FS)) {}
+SourceMgr::SourceMgr(SourceMgr &&) = default;
+SourceMgr &SourceMgr::operator=(SourceMgr &&) = default;
+SourceMgr::~SourceMgr() = default;
+
+IntrusiveRefCntPtr<vfs::FileSystem> SourceMgr::getVirtualFileSystem() const {
+  return FS;
+}
+
+void SourceMgr::setVirtualFileSystem(IntrusiveRefCntPtr<vfs::FileSystem> FS) {
+  this->FS = std::move(FS);
+}
+
 unsigned SourceMgr::AddIncludeFile(const std::string &Filename,
                                    SMLoc IncludeLoc,
                                    std::string &IncludedFile) {
@@ -52,8 +69,11 @@ unsigned SourceMgr::AddIncludeFile(const std::string &Filename,
 ErrorOr<std::unique_ptr<MemoryBuffer>>
 SourceMgr::OpenIncludeFile(const std::string &Filename,
                            std::string &IncludedFile) {
+  if (!FS)
+    reportFatalInternalError("Opening include file from SourceMgr without VFS");
+
   ErrorOr<std::unique_ptr<MemoryBuffer>> NewBufOrErr =
-      MemoryBuffer::getFile(Filename);
+      FS->getBufferForFile(Filename);
 
   SmallString<64> Buffer(Filename);
   // If the file didn't exist directly, see if it's in an include path.
@@ -61,7 +81,7 @@ SourceMgr::OpenIncludeFile(const std::string &Filename,
        ++i) {
     Buffer = IncludeDirectories[i];
     sys::path::append(Buffer, Filename);
-    NewBufOrErr = MemoryBuffer::getFile(Buffer);
+    NewBufOrErr = FS->getBufferForFile(Buffer);
   }
 
   if (NewBufOrErr)
diff --git a/llvm/lib/TableGen/Main.cpp b/llvm/lib/TableGen/Main.cpp
index 42043f70768c5..f61f50aa6c0a3 100644
--- a/llvm/lib/TableGen/Main.cpp
+++ b/llvm/lib/TableGen/Main.cpp
@@ -26,6 +26,7 @@
 #include "llvm/Support/SMLoc.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/TableGen/Error.h"
 #include "llvm/TableGen/Record.h"
@@ -129,6 +130,8 @@ int llvm::TableGenMain(const char *argv0,
   // it later.
   SrcMgr.setIncludeDirs(IncludeDirs);
 
+  SrcMgr.setVirtualFileSystem(vfs::getRealFileSystem());
+
   TGParser Parser(SrcMgr, MacroNames, Records, NoWarnOnUnusedTemplateArgs);
 
   if (Parser.ParseFile())
diff --git a/llvm/lib/TableGen/Parser.cpp b/llvm/lib/TableGen/Parser.cpp
index 2c3726a339bb8..cdf4d013f4bc0 100644
--- a/llvm/lib/TableGen/Parser.cpp
+++ b/llvm/lib/TableGen/Parser.cpp
@@ -9,6 +9,7 @@
 #include "llvm/TableGen/Parser.h"
 #include "TGParser.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/TableGen/Record.h"
 
 using namespace llvm;
@@ -20,6 +21,7 @@ bool llvm::TableGenParseFile(SourceMgr &InputSrcMgr, RecordKeeper &Records) {
   // this reliance, we could drop all of this.
   SrcMgr = SourceMgr();
   SrcMgr.takeSourceBuffersFrom(InputSrcMgr);
+  SrcMgr.setVirtualFileSystem(InputSrcMgr.getVirtualFileSystem());
   SrcMgr.setIncludeDirs(InputSrcMgr.getIncludeDirs());
   SrcMgr.setDiagHandler(InputSrcMgr.getDiagHandler(),
                         InputSrcMgr.getDiagContext());

benlangmuir

Would be good to have a unit test, but otherwise LGTM

Most `SourceMgr` clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a `vfs::FileSystem` instance into `SourceMgr`.

nathanchance · 2025-10-16T19:48:47Z

This change breaks using .incbin with -flto=thin, which is used by the Linux kernel in a couple of places.

$ cat test.c
asm("   .incbin \"foo.i\"      \n");

$ touch foo.i

$ clang --target=aarch64-linux-gnu -c -o /dev/null test.c

$ clang --target=aarch64-linux-gnu -flto=thin -c -o /dev/null test.c
fatal error: error in backend: Opening include file from SourceMgr without VFS
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: clang --target=aarch64-linux-gnu -flto=thin -c -o /dev/null test.c
1.	<eof> parser at end of file
2.	Optimizer
3.	Running pass "ThinLTOBitcodeWriterPass" on module "test.c"
 #0 0x000055d8362f5728 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (clang-22+0x5d86728)
 #1 0x000055d8362f2ea5 llvm::sys::RunSignalHandlers() (clang-22+0x5d83ea5)
 #2 0x000055d836274b47 (anonymous namespace)::CrashRecoveryContextImpl::HandleCrash(int, unsigned long) CrashRecoveryContext.cpp:0:0
 #3 0x000055d836274adf llvm::CrashRecoveryContext::HandleExit(int) (clang-22+0x5d05adf)
 #4 0x000055d8362ef967 llvm::sys::Process::Exit(int, bool) (clang-22+0x5d80967)
 #5 0x000055d8341b6956 (clang-22+0x3c47956)
 #6 0x000055d83627afb6 llvm::report_fatal_error(llvm::Twine const&, bool) (clang-22+0x5d0bfb6)
 #7 0x000055d83627aea6 (clang-22+0x5d0bea6)
 #8 0x000055d83627b14b (clang-22+0x5d0c14b)
 #9 0x000055d83629b193 llvm::SourceMgr::OpenIncludeFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&) (clang-22+0x5d2c193)
#10 0x000055d83629acbd llvm::SourceMgr::AddIncludeFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, llvm::SMLoc, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>&) (clang-22+0x5d2bcbd)
#11 0x000055d8360c0642 (anonymous namespace)::AsmParser::parseDirectiveIncbin() AsmParser.cpp:0:0
#12 0x000055d8360bd1cc (anonymous namespace)::AsmParser::parseStatement((anonymous namespace)::ParseStatementInfo&, llvm::MCAsmParserSemaCallback*) AsmParser.cpp:0:0
#13 0x000055d8360b5af1 (anonymous namespace)::AsmParser::Run(bool, bool) AsmParser.cpp:0:0
#14 0x000055d83946bdcc initializeRecordStreamer(llvm::Module const&, llvm::function_ref<void (llvm::RecordStreamer&)>) ModuleSymbolTable.cpp:0:0
#15 0x000055d83946b935 llvm::ModuleSymbolTable::CollectAsmSymbols(llvm::Module const&, llvm::function_ref<void (llvm::StringRef, llvm::object::BasicSymbolRef::Flags)>) (clang-22+0x8efc935)
#16 0x000055d83560340c llvm::buildModuleSummaryIndex(llvm::Module const&, std::function<llvm::BlockFrequencyInfo* (llvm::Function const&)>, llvm::ProfileSummaryInfo*, std::function<llvm::StackSafetyInfo const* (llvm::Function const&)>) (clang-22+0x509440c)
#17 0x000055d83560b3a4 llvm::ModuleSummaryIndexAnalysis::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (clang-22+0x509c3a4)
#18 0x000055d83734c032 llvm::detail::AnalysisPassModel<llvm::Module, llvm::ModuleSummaryIndexAnalysis, llvm::AnalysisManager<llvm::Module>::Invalidator>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) PassBuilder.cpp:0:0
#19 0x000055d835e23f8f llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) (clang-22+0x58b4f8f)
#20 0x000055d836b64eee llvm::ThinLTOBitcodeWriterPass::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (clang-22+0x65f5eee)
#21 0x000055d835e224e7 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (clang-22+0x58b34e7)
#22 0x000055d836a62129 (anonymous namespace)::EmitAssemblyHelper::RunOptimizationPipeline(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>&, std::unique_ptr<llvm::ToolOutputFile, std::default_delete<llvm::ToolOutputFile>>&, clang::BackendConsumer*) BackendUtil.cpp:0:0
#23 0x000055d836a57f4c clang::emitBackendOutput(clang::CompilerInstance&, clang::CodeGenOptions&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream>>, clang::BackendConsumer*) (clang-22+0x64e8f4c)
#24 0x000055d836a6d8e8 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (clang-22+0x64fe8e8)
#25 0x000055d83808ea59 clang::ParseAST(clang::Sema&, bool, bool) (clang-22+0x7b1fa59)
#26 0x000055d836f66006 clang::FrontendAction::Execute() (clang-22+0x69f7006)
#27 0x000055d836ececdd clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (clang-22+0x695fcdd)
#28 0x000055d83703db65 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (clang-22+0x6aceb65)
#29 0x000055d8341b6297 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (clang-22+0x3c47297)
#30 0x000055d8341b2015 ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>) driver.cpp:0:0
#31 0x000055d8341b45ed int llvm::function_ref<int (llvm::SmallVectorImpl<char const*>&)>::callback_fn<clang_main(int, char**, llvm::ToolContext const&)::$_0>(long, llvm::SmallVectorImpl<char const*>&) driver.cpp:0:0
#32 0x000055d836d34009 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const::$_0>(long) Job.cpp:0:0
#33 0x000055d836274a7e llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (clang-22+0x5d05a7e)
#34 0x000055d836d33843 clang::driver::CC1Command::Execute(llvm::ArrayRef<std::optional<llvm::StringRef>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, bool*) const (clang-22+0x67c4843)
#35 0x000055d836cf537c clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&, bool) const (clang-22+0x678637c)
#36 0x000055d836cf5597 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&, bool) const (clang-22+0x6786597)
#37 0x000055d836d0eda8 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*>>&) (clang-22+0x679fda8)
#38 0x000055d8341b1860 clang_main(int, char**, llvm::ToolContext const&) (clang-22+0x3c42860)
#39 0x000055d8341c1b67 main (clang-22+0x3c52b67)
#40 0x00007fa50f227635 (/usr/lib/libc.so.6+0x27635)
#41 0x00007fa50f2276e9 __libc_start_main (/usr/lib/libc.so.6+0x276e9)
#42 0x000055d8341afa25 _start (clang-22+0x3c40a25)
clang: error: clang frontend command failed with exit code 70 (use -v to see invocation)
ClangBuiltLinux clang version 22.0.0git (https://github.com/llvm/llvm-project.git 9deba01c1d5db607da5b943bd5a6185384608605)
...

# bad: [7fe069121b57a187e2072f01ecc84523948d9f04] [Headers][X86] Allow AVX512 masked shuffles to be used in constexpr (#162301)
# good: [9a46060aedc6bde26e939999de36442a23c07783] [mlir][linalg] Fix crash caused by nullptr dereference (#163132)
git bisect start '7fe069121b57a187e2072f01ecc84523948d9f04' '9a46060aedc6bde26e939999de36442a23c07783'
# bad: [3e251e727f6a214d11272c58db50b56544ee023d] [lldb] Correct bridgeOS -> BridgeOS spelling (#163479)
git bisect bad 3e251e727f6a214d11272c58db50b56544ee023d
# good: [b54709e9f68a6687eece0585e46f12e5fa218520] [NFC][MLIR][TableGen] Adopt NamespaceEmitter more widely  (#163289)
git bisect good b54709e9f68a6687eece0585e46f12e5fa218520
# bad: [d60d0381b41ee814c270ccee0e764b84096f8171] [MLIR][Conversion] XeGPU to XeVM: Remove unused type converter source materializations. (#162947)
git bisect bad d60d0381b41ee814c270ccee0e764b84096f8171
# good: [cf1cdde24e76a4ff5bbda2128800ba3ca4132b6d] [llvm][DebugInfo] Add 'sourceLanguageVersion' field support to DICompileUnit (#162632)
git bisect good cf1cdde24e76a4ff5bbda2128800ba3ca4132b6d
# good: [c89d721165da0c50a810c3f72bf000925e756e4b] [lldb-dap] Remove timings from TestDAP_attach (#163452)
git bisect good c89d721165da0c50a810c3f72bf000925e756e4b
# bad: [f49e3d178b8b8ab3ad6a87f999b42a5a76353c2e] Repair test for WG14 N3364 (#163551)
git bisect bad f49e3d178b8b8ab3ad6a87f999b42a5a76353c2e
# bad: [9deba01c1d5db607da5b943bd5a6185384608605] [support] Use VFS in `SourceMgr` for loading includes (#162903)
git bisect bad 9deba01c1d5db607da5b943bd5a6185384608605
# good: [097f1e7625966673b881df63a241f755317b0bb9] [lldb] Do not stop the process on SIGWINCH by default. (#163182)
git bisect good 097f1e7625966673b881df63a241f755317b0bb9
# first bad commit: [9deba01c1d5db607da5b943bd5a6185384608605] [support] Use VFS in `SourceMgr` for loading includes (#162903)

jansvoboda11 · 2025-10-16T19:55:47Z

Thank you for the report. I'll take a look and will attempt a forward-fix and I'll revert in case I can't get that done by EOD.

This commit more gracefully handles situations where `SourceMgr` isn't initialized with a VFS and tries to resolve an include. That's what happens with the test case `clang -flto=thin -c test.c -o /dev/null` where test.c contains `asm(" .incbin \"foo.i\" \n");`. Propagating the actual VFS all the way is very difficult. This is a follow-up to llvm#162903.

) This commit more gracefully handles situations where `SourceMgr` isn't initialized with a VFS and tries to resolve an include. That's what happens with the test case `clang -flto=thin -c test.c -o /dev/null` where test.c contains `asm(" .incbin \"foo.i\" \n");`. Propagating the actual VFS all the way is very difficult. This is a follow-up to #162903.

Most `SourceMgr` clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a `vfs::FileSystem` instance into `SourceMgr`.

uenoku · 2025-10-21T08:10:46Z

mlir/lib/IR/Diagnostics.cpp

+    auto bufferOrErr = llvm::MemoryBuffer::getFile(filename);
+    if (!bufferOrErr)
+      return 0;
+    unsigned id = mgr.AddNewSourceBuffer(std::move(*bufferOrErr), SMLoc());
    filenameToBufId[filename] = id;


Hello! Is this change intentional? It broke a downstream tool ;) I think a new code doesn't search include directories (and also not using VFS as well?).

You mean the change in this file in particular? How does it break downstream? The change is intentional. The intention of this PR was to force clients to set up SourceMgr with VFS to use includes, or load files themselves however they please and just pass MemoryBuffer to SourceMgr. In the end, I needed to roll this back a bit: #163862.

mlir/lib/IR/Diagnostics.cpp specifically. AddIncludeFile uses OpenIncludeFile which searches files under include directories. I understand the intention of the PR but the change in mlir/lib/IR/Diagnostics.cpp looks rather unrelated to VFS. I think AddIncludeFile also uses VFS now (#163862) so I don't think this change is necessary. Maybe I'm missing something?

Note that setting setVirtualFileSystem(llvm::vfs::getRealFileSystem()) didn't fix our issue.

Ah, that makes sense, sorry for the breakage. With #163862 this is no longer necessary. Fix is here: #164464.

With llvm#163862, this is not really necessary and causes downstream issues.

With #163862, this is not really necessary and causes downstream issues.

jansvoboda11 requested review from benlangmuir and cachemeifyoucan October 10, 2025 18:45

llvmbot added tablegen llvm:support labels Oct 10, 2025

llvmbot added mlir:core MLIR Core Infrastructure mlir labels Oct 10, 2025

benlangmuir reviewed Oct 13, 2025

View reviewed changes

llvmbot added clang Clang issues not falling into any other category llvm:codegen llvm:mc Machine (object) code labels Oct 14, 2025

jansvoboda11 added 7 commits October 14, 2025 15:42

[support] Use VFS in SourceMgr for loading includes

0889cde

Most `SourceMgr` clients don't make use of include files, but those that do might want to specify the file system to use. This patch enables that by making it possible to pass a `vfs::FileSystem` instance into `SourceMgr`.

Fix MLIR build

6115ecc

Add a unit test, try to fix tests failing in CI

b085458

MLIR

64da965

MLIR

71330a9

Fir MLIR tests

50ac6b0

Don't test the path

a41a3d2

jansvoboda11 force-pushed the sourcemgr-vfs branch from 2d7a19c to a41a3d2 Compare October 14, 2025 22:43

Merge branch 'main' into sourcemgr-vfs

874ebf3

benlangmuir approved these changes Oct 15, 2025

View reviewed changes

jansvoboda11 merged commit 9deba01 into llvm:main Oct 15, 2025
8 of 9 checks passed

jansvoboda11 deleted the sourcemgr-vfs branch October 15, 2025 16:24

jansvoboda11 mentioned this pull request Oct 16, 2025

[support] Don't require VFS in SourceMgr for loading includes #163862

Merged

uenoku mentioned this pull request Oct 21, 2025

Bump LLVM llvm/circt#9121

Open

uenoku reviewed Oct 21, 2025

View reviewed changes

jansvoboda11 added a commit to jansvoboda11/llvm-project that referenced this pull request Oct 21, 2025

[mlir] Partially revert llvm#162903

99bbea1

With llvm#163862, this is not really necessary and causes downstream issues.

jansvoboda11 added a commit that referenced this pull request Oct 21, 2025

[mlir] Partially revert #162903 (#164464)

e1e4154

With #163862, this is not really necessary and causes downstream issues.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[support] Use VFS in `SourceMgr` for loading includes #162903

[support] Use VFS in `SourceMgr` for loading includes #162903

Uh oh!

jansvoboda11 commented Oct 10, 2025

Uh oh!

llvmbot commented Oct 10, 2025 •

edited

Loading

Uh oh!

llvmbot commented Oct 10, 2025

Uh oh!

benlangmuir left a comment

Uh oh!

Uh oh!

nathanchance commented Oct 16, 2025

Uh oh!

jansvoboda11 commented Oct 16, 2025

Uh oh!

uenoku Oct 21, 2025

Uh oh!

jansvoboda11 Oct 21, 2025

Uh oh!

uenoku Oct 21, 2025

Uh oh!

jansvoboda11 Oct 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[support] Use VFS in SourceMgr for loading includes #162903

[support] Use VFS in SourceMgr for loading includes #162903

Uh oh!

Conversation

jansvoboda11 commented Oct 10, 2025

Uh oh!

llvmbot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Oct 10, 2025

Uh oh!

benlangmuir left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nathanchance commented Oct 16, 2025

Uh oh!

jansvoboda11 commented Oct 16, 2025

Uh oh!

uenoku Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

jansvoboda11 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

uenoku Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

jansvoboda11 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[support] Use VFS in `SourceMgr` for loading includes #162903

[support] Use VFS in `SourceMgr` for loading includes #162903

llvmbot commented Oct 10, 2025 •

edited

Loading