Consistent ninja crash when my build folder has a specific path

[jcelerier]

Hi,

• ninja 1.11.1
• generated by cmake 3.27.1
• on an up-to-date arch linux x86_64

I'm having a bewildering issue.. my build suddenly started failing, and only when i'm doing it in a specific path on my machine (I don't have any specific mounts or something funky like that... basic linux on an XFS partition).

it's an issue I sometimes see sporadically but just rm'ing the build folder is enough.. this time it's a persistent repro though.

$ pwd                          
/home/jcelerier/score-master/3rdparty/libossia/3rdparty/build-libremidi-System_Qt6-Debug

$ cmake ~/score-master/3rdparty/libossia/3rdparty/libremidi -GNinja -DCMAKE_BUILD_TYPE=Debug   
-- The CXX compiler identification is GNU 13.1.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- libremidi: Using std::vector for libremidi::message
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found ALSA: /usr/lib/libasound.so (found version "1.2.9") 
-- libremidi: using ALSA
-- libremidi: using linked JACK
-- Looking for jack_port_rename
-- Looking for jack_port_rename - not found
-- Configuring done (0.4s)
-- Generating done (0.0s)
CMake Error:
  Running

   '/usr/bin/ninja' '-C' '/home/jcelerier/score-master/3rdparty/libossia/3rdparty/build-libremidi-System_Qt6-Debug' '-t' 'recompact'

  failed with:

   /usr/include/c++/12.2.1/bits/stl_vector.h:1123: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = Node*; _Alloc = std::allocator<Node*>; reference = Node*&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

  Subprocess aborted

If I build in any other path (even with the same final folder name) I don't see the issue. I thought there'd be some off-by-one name buffer issue or something like this somewhere which happens to trigger in just this exact case, but if I change even one character anywhere in the path, it seems to work so it does not look like a buffer overflow.

$ pwd
/home/jcelerier/score-master/3rdparty/libossia/3rdparty/build-libremidi-System_Qt6-DebuX
$ cmake ~/score-master/3rdparty/libossia/3rdparty/libremidi -GNinja -DCMAKE_BUILD_TYPE=Debug   
-- The CXX compiler identification is GNU 13.1.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- libremidi: Using std::vector for libremidi::message
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found ALSA: /usr/lib/libasound.so (found version "1.2.9") 
-- libremidi: using ALSA
-- libremidi: using linked JACK
-- Looking for jack_port_rename
-- Looking for jack_port_rename - not found
-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /home/jcelerier/score-master/3rdparty/libossia/3rdparty/build-libremidi-System_Qt6-DebuX

Trace:

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff7a9f2d3 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007ffff7a4fa08 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff7a38538 in __GI_abort () at abort.c:79
#4  0x00007ffff7cdd3b2 in std::__glibcxx_assert_fail (file=file@entry=0x55555558ad00 "/usr/include/c++/12.2.1/bits/stl_vector.h", line=line@entry=1123, 
    function=function@entry=0x55555558b330 "std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = Node*; _Alloc = std::allocator<Node*>; reference = Node*&; size_type = long unsigned int]", condition=condition@entry=0x55555558a1b8 "__n < this->size()") at /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/debug.cc:61
#5  0x000055555558519f in std::vector<Node*, std::allocator<Node*> >::operator[] (this=<optimized out>, __n=<optimized out>) at /usr/include/c++/12.2.1/bits/stl_vector.h:1123
#6  std::vector<Node*, std::allocator<Node*> >::operator[] (this=0x7fffffffdd30, __n=<optimized out>) at /usr/include/c++/12.2.1/bits/stl_vector.h:1121
#7  DepsLog::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, State*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) [clone .constprop.0] (this=0x7fffffffdd00, path=".ninja_deps", state=0x7fffffffdb68, err=0x7fffffffd850) at src/deps_log.cc:219
#8  0x000055555555ea1b in (anonymous namespace)::NinjaMain::OpenDepsLog (this=this@entry=0x7fffffffdb50, recompact_only=recompact_only@entry=false) at src/ninja.cc:1278
#9  0x0000555555561f11 in (anonymous namespace)::real_main (argc=<optimized out>, argv=<optimized out>) at src/ninja.cc:1572
#10 0x000055555555a59f in main (argc=<optimized out>, argv=<optimized out>) at src/ninja.cc:1620

I attach a core dump, my ninja binary and the source & build folder with their absolute paths:

repro.tar.gz

[thesamesam]

Note that this needs -D_GLIBCXX_ASSERTIONS to trigger. It might happen with the libcxx equivalent too.

[digit-google]

Thank you for your archive. I wrote a small Python script to dump the content of your .ninja_deps file which shows that it is malformed! In particular, it contains multiple input path id values like 1441 that are out of bounds with respect of the max ids allocated in the file.

This means two things:

1. Somehow Ninja managed to generate a malformed .ninja_deps file entries, which is unexplained and very concerning :-/

2. Ninja doesn't protect against malicious / malformed input correctly. In particular, the following code in src/deps_log.cc should catch the bad value in the first assert here, but I assume you didn't build your Ninja with assertions enabled. Moreover, an assert() is not a valid protection against malformed input like this one, and should be turned into real compile-time checks + error handling:

      for (int i = 0; i < deps_count; ++i) {
        assert(deps_data[i] < (int)nodes_.size());
        assert(nodes_[deps_data[i]]);
        deps->nodes[i] = nodes_[deps_data[i]];
      }

It should be easy to fix the deps log loader against such input. On the other hand I do not understand how something like that could be produced. Maybe it's a bad recompaction, may
be it's something else. And I'd love to better understand the issue because it's just a little scary.

read_ninja_deps.zip