Invalid order of command-line and built-in macro definitions in .debug_macros

[simark]

Using this source file:

#define ZERO 0

int main(void) {
    return ZERO + ONE;
}   

Compiled with:

$ clang --version
Ubuntu clang version 15.0.0-++20220320052914+3b2e605e33bd-1~exp1~20220320173006.195
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ clang -O0 -gdwarf-5 -fdebug-macro test.c -DONE=1

We get

$ llvm-dwarfdump-14 -debug-macro a.out
a.out:	file format elf64-x86-64

.debug_macro contents:
0x00000000:
macro header: version = 0x0005, flags = 0x02, format = DWARF32, debug_line_offset = 0x00000000
DW_MACRO_start_file - lineno: 0 filenum: 0
  DW_MACRO_define_strx - lineno: 1 macro: ZERO 0
DW_MACRO_end_file
DW_MACRO_define_strx - lineno: 0 macro: __llvm__ 1
DW_MACRO_define_strx - lineno: 0 macro: __clang__ 1
<snip>
DW_MACRO_define_strx - lineno: 0 macro: __STDC_UTF_16__ 1
DW_MACRO_define_strx - lineno: 0 macro: __STDC_UTF_32__ 1
DW_MACRO_define_strx - lineno: 0 macro: ONE 1
DW_MACRO_define_strx - lineno: 0 macro: __GCC_HAVE_DWARF2_CFI_ASM 1

We see that the definition for ONE (a macro defined on the command line) is placed after the DW_MACRO_end_file leaving the main source file. That makes the definition not visible from a point somewhere in the main source file. Same for all the built-in macros (like __llvm__).

According to DWARF5 section 6.3.2.4 Entries for Command Line Options:

A DWARF producer generates a define or undefine entry for each pre-processor
symbol which is defined or undefined by some means other than such a directive
within the compiled source text. In particular, pre-processor symbol definitions
and undefinitions which occur as a result of command line options (when
invoking the compiler) are represented by their own define and undefine entries.

All such define and undefine entries representing compilation options appear
before the first DW_MACRO_start_file entry for that compilation unit (see
Section 6.3.3 following) and encode the value 0 in their line number operands.

I think that the definition of ONE should be placed before the first DW_MACRO_start_file. The paragraph above talks about the macros defined using command line options, but in my opinion the built-in macros should also be placed before the first DW_MACRO_start_file. They fit the defined or undefined by some means other than such a directive within the compiled source text description.

The goal is that when stopped in the main source file (or any source file it includes) in a debugger, we are able to print ONE or print __llvm__.

[llvmbot]

@llvm/issue-subscribers-debuginfo

[dwblaikie]

Yeah, looks like this is because we enter the source file being compiled here:

Run till exit from #0  llvm::DIBuilder::createTempMacroFile (this=0x555567c9a900, Parent=0x0, LineNumber=0, File=0x555567c9c6c0) at /usr/local/google/home/blaikie/dev/llvm/src/llvm/lib/IR/DIBuilder.cpp:251
0x00005555611a8ba3 in clang::CodeGen::CGDebugInfo::CreateTempMacroFile (this=0x555567c9a8f0, Parent=0x0, LineLoc=..., FileLoc=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/CGDebugInfo.cpp
:3267
3267      return DBuilder.createTempMacroFile(Parent, Line, FName);
Value returned is $19 = (llvm::DIMacroFile *) 0x555567cddbe0
(gdb) up
#1  0x000055556201d8b2 in clang::MacroPPCallbacks::FileEntered (this=0x555567c59af0, Loc=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/MacroPPCallbacks.cpp:131
131       Scopes.push_back(Gen->getCGDebugInfo()->CreateTempMacroFile(getCurrentScope(),
(gdb) bt
#0  0x00005555611a8ba3 in clang::CodeGen::CGDebugInfo::CreateTempMacroFile (this=0x555567c9a8f0, Parent=0x0, LineLoc=..., FileLoc=...)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/CGDebugInfo.cpp:3267
#1  0x000055556201d8b2 in clang::MacroPPCallbacks::FileEntered (this=0x555567c59af0, Loc=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/MacroPPCallbacks.cpp:131
#2  0x000055556201d9fd in clang::MacroPPCallbacks::FileChanged (this=0x555567c59af0, Loc=..., Reason=clang::PPCallbacks::EnterFile, FileType=clang::SrcMgr::C_User, PrevFID=...)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/MacroPPCallbacks.cpp:163
#3  0x0000555566f0f4cb in clang::PPChainedCallbacks::FileChanged (this=0x555567cdbfd0, Loc=..., Reason=clang::PPCallbacks::EnterFile, FileType=clang::SrcMgr::C_User, PrevFID=...)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/include/clang/Lex/PPCallbacks.h:438
#4  0x0000555566f2b20c in clang::Preprocessor::EnterSourceFileWithLexer (this=0x555567c69bb0, TheLexer=0x555567cd3f30, CurDir=...)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/PPLexerChange.cpp:140
#5  0x0000555566f2b012 in clang::Preprocessor::EnterSourceFile (this=0x555567c69bb0, FID=..., CurDir=..., Loc=..., IsFirstIncludeOfFile=true)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/PPLexerChange.cpp:106
#6  0x0000555566f63506 in clang::Preprocessor::EnterMainSourceFile (this=0x555567c69bb0) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/Preprocessor.cpp:555
#7  0x00005555649e0d53 in clang::ParseAST (S=..., PrintStats=false, SkipFunctionBodies=false) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Parse/ParseAST.cpp:143

Before we process the builtin macros here:

#0  clang::CodeGen::CGDebugInfo::CreateMacro (this=0x555567c9a8f0, Parent=0x0, MType=1, LineLoc=..., Name="__llvm__", Value="1")
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/CGDebugInfo.cpp:3258
#1  0x000055556201dbe7 in clang::MacroPPCallbacks::MacroDefined (this=0x555567c59af0, MacroNameTok=..., MD=0x555567c6cdb0)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/CodeGen/MacroPPCallbacks.cpp:186
#2  0x0000555566f1060a in clang::PPChainedCallbacks::MacroDefined (this=0x555567cdbfd0, MacroNameTok=..., MD=0x555567c6cdb0)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/include/clang/Lex/PPCallbacks.h:601
#3  0x0000555566f17781 in clang::Preprocessor::HandleDefineDirective (this=0x555567c69bb0, DefineTok=..., ImmediatelyAfterHeaderGuard=false)
    at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/PPDirectives.cpp:3160
#4  0x0000555566f1827e in clang::Preprocessor::HandleDirective (this=0x555567c69bb0, Result=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/PPDirectives.cpp:1246
#5  0x0000555566ed1334 in clang::Lexer::LexTokenInternal (this=0x555567c5a4a0, Result=..., TokAtPhysicalStartOfLine=true) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/Lexer.cpp:4362
#6  0x0000555566ecccfc in clang::Lexer::Lex (this=0x555567c5a4a0, Result=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/Lexer.cpp:3576
#7  0x0000555566f658bf in clang::Preprocessor::Lex (this=0x555567c69bb0, Result=...) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Lex/Preprocessor.cpp:889
#8  0x00005555649ee99e in clang::Parser::ConsumeToken (this=0x555567cdbff0) at /usr/local/google/home/blaikie/dev/llvm/src/clang/include/clang/Parse/Parser.h:499
#9  0x00005555649e5938 in clang::Parser::Initialize (this=0x555567cdbff0) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Parse/Parser.cpp:565
#10 0x00005555649e0dd5 in clang::ParseAST (S=..., PrintStats=false, SkipFunctionBodies=false) at /usr/local/google/home/blaikie/dev/llvm/src/clang/lib/Parse/ParseAST.cpp:155

Not sure exactly what the right way to fix this is - maybe we just fix this up in LLVM & reorder all the file-less macros before the others...