run-clang-tidy.py and absolute paths in compile_commands.json

[pozzugno]

In my cmake build system, compile_commands.json contains absolute paths both for "directory" and "source". So the script run-clang-tidy.py doesn't work here, when tries to calculates the absolute path of a source file joining "directory" and "file" paths.

I think the script assumes "file" path is always relative to "directory" path, but it isn't true in general.

One fix could be:

files = {e["file"] if os.path.isabs(e["file"]) else os.path.abspath(os.path.join(e["directory"], e["file"])) for e in database}

[llvmbot]

@llvm/issue-subscribers-clang-tidy

Author: None (pozzugno)

In my cmake build system, compile_commands.json contains absolute paths both for "directory" and "source". So the script run-clang-tidy.py doesn't work [here](https://github.com/llvm/llvm-project/blob/07fa6d1d90c714fa269529c3e5004a063d814c4a/clang-tools-extra/clang-tidy/tool/run-clang-tidy.py#L526), when tries to calculates the absolute path of a source file joining "directory" and "file" paths.

I think the script assumes "file" path is always relative to "directory" path, but it isn't true in general.

One fix could be:

files = {e["file"] if os.path.isabs(e["file"]) else os.path.abspath(os.path.join(e["directory"], e["file"])) for e in database}

[vbvictor]

Could you provide a small reproducer with compile_commands.json file and clang-tidy and python3 version you use?

On my system I have such entry in compile_commands.json with absolute paths:

{
  "directory": "/home/victor/llvm/llvm-project/build",
  "command": "/usr/bin/clang++-19 -o tools/clang/tools/extra/clang-tidy/CMakeFiles/obj.clangTidy.dir/ClangTidyDiagnosticConsumer.cpp.o -c /home/victor/llvm/llvm-project/clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp",
  "file": "/home/victor/llvm/llvm-project/clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp",
  "output": "tools/clang/tools/extra/clang-tidy/CMakeFiles/obj.clangTidy.dir/ClangTidyDiagnosticConsumer.cpp.o"
},

When I run such python3 script:

import os

dir = "/home/victor/llvm/llvm-project/build"
file = "/home/victor/llvm/llvm-project/clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp"
print(f'os.path.join result: {os.path.abspath(os.path.join(dir, file))}')

The output is os.path.join result: /home/victor/llvm/llvm-project/clang-tools-extra/clang-tidy/ClangTidyDiagnosticConsumer.cpp

So it handles abs paths correctly, and run-clang-tidy.py works fine.
My python version is Python 3.12.3.

[pozzugno]

I found the problem. I was in a Windows 10 machine and my compile_commands.json was something similar to this:

{
  "directory": "C:/cmake-vscode-embedded-sample/build/full-sim",
  "file": "C:/cmake-vscode-embedded-sample/src/main.c",
}

Python launcher py.exe read the shebang in the run-clang-tidy.py script (#!/usr/bin/env python3) and tried to launch python3 on the PATH. In my case, python3 was found in c:\msys\usr\bin\python3. Standard python3 for Windows installs python.exe and not python3.exe, so msys python3 is launched in my case.

It seems to me os.path.join() of msys python3 doesn't work as expected.

$ c:\msys64\usr\bin\python3
Python 3.12.10 (main, Apr 10 2025, 10:39:24) [GCC 13.3.0] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os.path
>>> print(os.path.abspath(os.path.join("c:/cmake-vscode-embedded-sample/build/full-sim", "c:/cmake-vscode-embedded-sample/src/main.c")))
c:/cmake-vscode-embedded-sample/build/full-sim/c:/cmake-vscode-embedded-sample/src/main.c

The os.path.isabs("c:/cmake-vscode-embedded-sample/src/main.c") returns True, so this kind of path is recognized as absolute, but the join operation doesn't work as expected.

The fixed I proposed in my original post works, but it fixes an issue on msys python3 and not in the run-clang-tidy.py script. I don't know if you want to add it.

[nicovank]

This is only because of mixing Windows/POSIX paths. compile_commands.json contains Windows/NT paths, Cygwin Python expects POSIX paths. CPython has a complete separate implementation for both, and the POSIX one wouldn't handle C:/ drives. If you run that command (os.path.join("c:/foo", "c:/bar")) in Windows Python, it should work as expected.

Note that the fix above won't work for the same reason: os.path.isabs("C:/") is False on POSIX.

So it all depends how you are running run-clang-tidy -- Is it possible to just launch it with python.exe, which as I understand it is the Windows Python? Alternatively, could you generate build commands within Cygwin?

[pozzugno]

Yes, I think I can solve this problem on my side, but...

On Windows platform it's common to use Python Launcher for Windows (py.exe) to launch python scripts. The launcher tries to reproduce the Unix behaviour and process the shebang. Even the shebang used in run-clang-tidy.py is supported:

#!/usr/bin/env python3

As the docs say:

The /usr/bin/env form of shebang line has one further special property. Before looking for installed Python interpreters, this form will search the executable PATH for a Python executable matching the name provided as the first argument. This corresponds to the behaviour of the Unix env program, which performs a PATH search. If an executable matching the first argument after the env command cannot be found, but the argument starts with python, it will be handled as described for the other virtual commands. The environment variable PYLAUNCHER_NO_SEARCH_PATH may be set (to any value) to skip this search of PATH.

However standard python for windows installer doesn't create a python3.exe file, only python.exe. So the launcher doesn't finds the main python Window installation, but finds the msys version (that is on my PATH and that has python3.exe).

Summarizing, it seems to me that the shebang used in run-clang-tidy.py isn't portable between Unix and Windows.

Why not changing to #!/usr/bin/env python?

[carlosgalvezp]

Why not changing to #!/usr/bin/env python

Because that's Python2. Modern Ubuntu versions don't have python installed, they have python3. You need to install a separate package or create a symlink to obtain python.