Error running "ninja check" after building latest LLVM and Clang on Linux Ubuntu 22.04

[bobhairgrove]

Platform is Linux Ubuntu 22.04 with g++ version 11.4.0.

I tried to clone and build LLVM and Clang from source following the instructions found here:
https://clang.llvm.org/docs/LibASTMatchersTutorial.html

My clang-llvm directory is not directly under the $HOME directory, but in /home/bob/code/clang-llvm, so I cloned the source code from GitHub as follows:

cd ~/code
mkdir clang-llvm
cd clang-llvm
git clone https://github.com/llvm/llvm-project.git

Output:

Cloning into 'llvm-project'...
remote: Enumerating objects: 6204242, done.
remote: Counting objects: 100% (7573/7573), done.
remote: Compressing objects: 100% (2494/2494), done.
remote: Total 6204242 (delta 6074), reused 5948 (delta 5065), pack-reused 6196669 (from 1)
Receiving objects: 100% (6204242/6204242), 1.52 GiB | 29.79 MiB/s, done.
Resolving deltas: 100% (5134259/5134259), done.
Updating files: 100% (150857/150857), done.

CMake and ninja were already installed in my system; their versions are:

cmake --version
cmake version 3.26.3

ninja --version
1.10.1

Then I did:

cd clang-llvm/
mkdir build && cd build
cmake -G Ninja ../llvm-project/llvm -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_TESTS=ON

Then I ran:

ninja

There were quite a few warnings, but it seemed to run successfully. Finally, I ran:

ninja check

The last command failed. Here is the complete output with the error message at the end:

[2/3] cd /home/bob/code/clang-llvm/llvm-project/clang/bindings/python && /home/bob/.local/lib/python3...FIG=1 CLANG_LIBRARY_PATH=/home/bob/code/clang-llvm/build/lib /usr/bin/python3.10 -m unittest discover
......................................................................................................................................................
----------------------------------------------------------------------
Ran 150 tests in 0.600s

OK
[2/3] Running all regression tests
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using split-file: /home/bob/code/clang-llvm/build/bin/split-file
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using yaml2obj: /home/bob/code/clang-llvm/build/bin/yaml2obj
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using llvm-objcopy: /home/bob/code/clang-llvm/build/bin/llvm-objcopy
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using clang: /home/bob/code/clang-llvm/build/bin/clang
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using clang: /home/bob/code/clang-llvm/build/bin/clang
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using clang: /home/bob/code/clang-llvm/build/bin/clang
FAIL: LLVM :: Transforms/PhaseOrdering/AArch64/predicated-reduction.ll (68485 of 76318)
******************** TEST 'LLVM :: Transforms/PhaseOrdering/AArch64/predicated-reduction.ll' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 2: /home/bob/code/clang-llvm/build/bin/opt -passes="default<O3>" -S < /home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll  | /home/bob/code/clang-llvm/build/bin/FileCheck /home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll
+ /home/bob/code/clang-llvm/build/bin/opt '-passes=default<O3>' -S
+ /home/bob/code/clang-llvm/build/bin/FileCheck /home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll
/home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll:26:15: error: CHECK-NEXT: expected string not found in input
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <2 x double> [ zeroinitializer, %[[VECTOR_PH]] ], [ [[TMP18:%.*]], %[[VECTOR_BODY]] ]
              ^
<stdin>:26:67: note: scanning from here
 %index = phi i64 [ 0, %vector.ph ], [ %index.next, %vector.body ]
                                                                  ^
<stdin>:26:67: note: with "VECTOR_PH" equal to "vector\\.ph"
 %index = phi i64 [ 0, %vector.ph ], [ %index.next, %vector.body ]
                                                                  ^
<stdin>:26:67: note: with "VECTOR_BODY" equal to "vector\\.body"
 %index = phi i64 [ 0, %vector.ph ], [ %index.next, %vector.body ]
                                                                  ^
<stdin>:39:9: note: possible intended match here
 %8 = fcmp fast ogt <2 x double> %6, zeroinitializer
        ^
/home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll:349:12: error: undefined variable: LOOP0
; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]]}
           ^
/home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll:349:35: error: undefined variable: LOOP0
; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]]}
                                  ^
<stdin>:172:2: note: possible intended match here
!0 = distinct !{!0, !1, !2}
 ^

Input file: <stdin>
Check file: /home/bob/code/clang-llvm/llvm-project/llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
             .
             .
             .
            21:  %broadcast.splatinsert17 = insertelement <2 x double> poison, double %Z, i64 0 
            22:  %broadcast.splat18 = shufflevector <2 x double> %broadcast.splatinsert17, <2 x double> poison, <2 x i32> zeroinitializer 
            23:  br label %vector.body 
            24:  
            25: vector.body: ; preds = %vector.body, %vector.ph 
            26:  %index = phi i64 [ 0, %vector.ph ], [ %index.next, %vector.body ] 
next:26'0                                                                         X error: no match found
next:26'1                                                                           with "VECTOR_PH" equal to "vector\\.ph"
next:26'2                                                                           with "VECTOR_BODY" equal to "vector\\.body"
            27:  %vec.phi = phi double [ 0.000000e+00, %vector.ph ], [ %17, %vector.body ] 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            28:  %vec.phi15 = phi double [ 0.000000e+00, %vector.ph ], [ %19, %vector.body ] 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            29:  %0 = getelementptr inbounds float, ptr %samples, i64 %index 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            30:  %1 = getelementptr inbounds i8, ptr %0, i64 8 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            31:  %wide.load = load <2 x float>, ptr %0, align 4 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             .
             .
             .
            34:  %3 = fpext <2 x float> %wide.load16 to <2 x double> 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            35:  %4 = fmul fast <2 x double> %broadcast.splat, %2 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            36:  %5 = fmul fast <2 x double> %broadcast.splat, %3 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            37:  %6 = fsub fast <2 x double> %4, %broadcast.splat18 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            38:  %7 = fsub fast <2 x double> %5, %broadcast.splat18 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            39:  %8 = fcmp fast ogt <2 x double> %6, zeroinitializer 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
next:26'3               ?                                             possible intended match
            40:  %9 = fcmp fast ogt <2 x double> %7, zeroinitializer 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            41:  %10 = fmul fast <2 x double> %6, %6 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            42:  %11 = fmul fast <2 x double> %7, %7 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            43:  %12 = tail call fast <2 x double> @llvm.maxnum.v2f64(<2 x double> %6, <2 x double> <double -0.000000e+00, double -0.000000e+00>) 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            44:  %13 = tail call fast <2 x double> @llvm.maxnum.v2f64(<2 x double> %7, <2 x double> <double -0.000000e+00, double -0.000000e+00>) 
next:26'0       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             .
             .
             .
           146:  
           147: for.end10: ; preds = %for.body, %for.cond1.for.inc8_crit_edge.us, %entry 
           148:  %v0.0.lcssa = phi double [ 0.000000e+00, %entry ], [ %v0.2.us, %for.cond1.for.inc8_crit_edge.us ], [ 0.000000e+00, %for.body ] 
           149:  %v1.0.lcssa = phi double [ 0.000000e+00, %entry ], [ %v1.2.us, %for.cond1.for.inc8_crit_edge.us ], [ 0.000000e+00, %for.body ] 
           150:  %add11 = fadd fast double %v1.0.lcssa, %v0.0.lcssa 
           151:  ret double %add11 
check:349'0                       X error: match failed for invalid pattern
check:349'1                         undefined variable: LOOP0
check:349'2                         undefined variable: LOOP0
           152: } 
check:349'0     ~~
           153:  
check:349'0     ~
           154: declare void @resample(i32 noundef, ptr noundef) local_unnamed_addr 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           155:  
check:349'0     ~
           156: ; Function Attrs: mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none) 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             .
             .
             .
           167:  
check:349'0     ~
           168: attributes #0 = { nofree norecurse nosync nounwind memory(argmem: read) } 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           169: attributes #1 = { mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none) } 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           170: attributes #2 = { nocallback nofree nosync nounwind speculatable willreturn memory(none) } 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           171:  
check:349'0     ~
           172: !0 = distinct !{!0, !1, !2} 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:349'3      ?                           possible intended match
           173: !1 = !{!"llvm.loop.isvectorized", i32 1} 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           174: !2 = !{!"llvm.loop.unroll.runtime.disable"} 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           175: !3 = distinct !{!3, !2, !1} 
check:349'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>

--

********************
********************
Failed Tests (1):
  LLVM :: Transforms/PhaseOrdering/AArch64/predicated-reduction.ll


Testing Time: 2361.66s

Total Discovered Tests: 111022
  Skipped          :     23 (0.02%)
  Unsupported      :   2636 (2.37%)
  Passed           : 108178 (97.44%)
  Expectedly Failed:    184 (0.17%)
  Failed           :      1 (0.00%)
FAILED: CMakeFiles/check-all /home/bob/code/clang-llvm/build/CMakeFiles/check-all 
cd /home/bob/code/clang-llvm/build && /usr/bin/python3.10 /home/bob/code/clang-llvm/build/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 /home/bob/code/clang-llvm/build/utils/mlgo-utils /home/bob/code/clang-llvm/build/tools/clang/tools/extra/include-cleaner/test /home/bob/code/clang-llvm/build/tools/clang/tools/extra/clangd/test/../unittests /home/bob/code/clang-llvm/build/tools/clang/tools/extra/clangd/test /home/bob/code/clang-llvm/build/tools/clang/tools/extra/test /home/bob/code/clang-llvm/build/tools/clang/test /home/bob/code/clang-llvm/build/utils/lit /home/bob/code/clang-llvm/build/test
ninja: build stopped: subcommand failed.

I'm not a stranger to configuring and compiling things from source, but I do feel somewhat overwhelmed by the messages here. Can anyone please tell me exactly what failed, and how I might rectify this?

[bobhairgrove]

FWIW, the Clang tests seem to pass OK:

ninja clang-test
[0/1] Running the Clang regression tests
llvm-lit: /home/bob/code/clang-llvm/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using clang: /home/bob/code/clang-llvm/build/bin/clang

Testing Time: 1090.41s

Total Discovered Tests: 45538
  Skipped          :     8 (0.02%)
  Unsupported      :   145 (0.32%)
  Passed           : 45357 (99.60%)
  Expectedly Failed:    28 (0.06%)

[bobhairgrove]

In the meantime, I deleted the project and cloned the repository again and rebuilt from scratch. All tests pass, except for those which are expected to fail or are unsupported.

Two main differences:
(1) I ran cmake from the project directory instead of doing mkdir build && cd build;
(2) Added lld to the -DLLVM_ENABLE_PROJECTS option.

Here are the commands I used:

cd llvm-project
cmake -S llvm -B build -G Ninja -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra;lld' -DCMAKE_BUILD_TYPE=Release -DLLVM_PARALLEL_COMPILE_JOBS=8 -DLLVM_BUILD_TESTS=ON
cmake --build build
cd build
ninja check
ninja clang-test

The clang-test is now 81% into its run, but I think these tests will pass. If not, I'll edit this post to indicate that something else failed.

But it seems like there might be something missing or not quite right in the "ASTMatchers" tutorial linked in my first message?

EDIT: Looks like all clang tests pass (or were skipped / unsupported / expectedly fail):

Total Discovered Tests: 45549
  Skipped          :     8 (0.02%)
  Unsupported      :   142 (0.31%)
  Passed           : 45371 (99.61%)
  Expectedly Failed:    28 (0.06%)

[efriedma-quic]

I suspect you just got unlucky with when you checked out LLVM. We have pre-commit buildbots, but merging isn't directly gated on a build succeeding, so on rare occasions "main" is broken, even in supported configurations. Let me know if you can still reproduce.

[bobhairgrove]

Thanks. Are the commands ninja check and ninja clang-test both necessary, or are the clang tests not actually a subset of the LLVM tests run by ninja check?

[efriedma-quic]

The canonical names are ninja check-clang and ninja check-all. And yes, check-clang is a subset of check-all. (I think at one point "check" meant something else, but that's the current state.)

[siya100]

hey @efriedma-quic can you ellaborate a little why this happens as i myself got unlucky while running those cmake and ninja commands

[siya100]

and why some files while executing are meant to fail

[efriedma-quic]

why this happens as i myself got unlucky while running those cmake and ninja commands

We have some pre-commit automated to try to avoid breaking things. But the precommit bots don't actually block merging. And the vast majority of our buildbots run post-commit, to reduce the required resources. So occasionally things end up slightly broken.

Let me know if you have more specific questions about this.

and why some files while executing are meant to fail

We XFAIL tests when they're only supposed to pass in certain configurations, or sometimes when there's work-in-progress code. So the expected output in a given configuration is actually a "failure".