Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subdoc fails with missing modmap files #437

Closed
danakj opened this issue Dec 19, 2023 · 8 comments · Fixed by #440
Closed

Subdoc fails with missing modmap files #437

danakj opened this issue Dec 19, 2023 · 8 comments · Fixed by #440
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@danakj
Copy link
Collaborator

danakj commented Dec 19, 2023

A recent change in Clang 18 has made subdoc fail to execute the clang tool through libclang, generating an error for every .cc file it attempts to compile:

Error while processing /home/runner/work/subspace/subspace/sus/num/i8_unittest.cc.
error: no such file or directory: '@sus/CMakeFiles/subspace_unittests.dir/mem/addressof_unittest.cc.o.modmap'
[2/99] /home/runner/work/subspace/subspace/sus/mem/addressof_unittest.cc

Subspace is not using modules so there's no modmaps expected.

llvm/llvm-project#75142 looks like it could be the reason.

First appeared on the try run here: #436

@danakj danakj added the documentation Improvements or additions to documentation label Dec 19, 2023
@danakj danakj added this to the Subdoc milestone Dec 19, 2023
@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

Same error occurs in clang 17, seems to be something else. Maybe a cmake change?

@danakj danakj changed the title Subdoc fails with missing modmap files in clang 18 Subdoc fails with missing modmap files Dec 19, 2023
@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

Yeah it's CMake 3.28 which added C++20 module support. CMake's ninja generator puts something like this into the command line for each file:

@third_party\\googletest\\googlemock\\CMakeFiles\\gmock_main.dir\\src\\gmock-all.cc.obj.modmap

And libclang doesn't seem to know what to do with it.

@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

No clang docs seem to mention what the @ means either :(

@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

it's from here: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8611/diffs

Which adds the @foo.modmap to the command line, which CL docs say is a command-file which has other commands in it? https://learn.microsoft.com/en-us/cpp/build/reference/cl-command-files?view=msvc-170

But the files don't exist for me after building actual binaries either.

The test expects @foo.modmap or -fmodule-mapper=foo.modmap and GCC docs mention the latter: https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Module-Mapper.html

@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

Oh this blog post discusses it: https://nibblestew.blogspot.com/2023/10/the-road-to-hell-is-paved-with-good.html

Ninja does not support dynamic compiler argument generation. Which is good, because it makes things faster, simpler and more reliable. This does not stop CMake, which hacked it on top anyway. The compiler command they invoke looks like the following.

clang++ <some compiler flags> @path/to/somefile.modmap <more flags>
The @file syntax means "read the contents of the file and pretend that it contains command line arguments". The file itself is created at build time with a scanner program. Here's what its contents look like.

-x c++-module
-fmodule-output=CMakeFiles/modtest.dir/M0.pcm
-fmodule-file=M1=CMakeFiles/modtest.dir/M1.pcm
-fmodule-file=M2=CMakeFiles/modtest.dir/M2.pcm
-fmodule-file=M3=CMakeFiles/modtest.dir/M3.pcm
-fmodule-file=M4=CMakeFiles/modtest.dir/M4.pcm
-fmodule-file=M5=CMakeFiles/modtest.dir/M5.pcm
-fmodule-file=M6=CMakeFiles/modtest.dir/M6.pcm
-fmodule-file=M7=CMakeFiles/modtest.dir/M7.pcm
-fmodule-file=M8=CMakeFiles/modtest.dir/M8.pcm
-fmodule-file=M9=CMakeFiles/modtest.dir/M9.pcm

This is a file that cmake generates before running the compile command. Since clang tools are not cmake... the file does not exist and the resulting compile_commands.json is broken for any other use. Oh no.

@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

Since only CMake will generate these files during compilation, we can't use them obviously. I think our only option is to strip them out of the command line. This will work until code actually uses modules at which point CMake will not provide a command line in compile_commands.json that is suitable to reproduce building the C++ file. And we may need to get off of CMake or something.

@danakj
Copy link
Collaborator Author

danakj commented Dec 19, 2023

Posted to the cmake discussion here: https://discourse.cmake.org/t/how-to-control-the-location-of-the-c-20-binary-module-interface-bmi-output-directory/7968/13

For now, we will strip out the modmap command-file from the command-line that CMake generates specifically by looking for @.*\.modmap.

danakj added a commit to danakj/subspace that referenced this issue Dec 19, 2023
CMake 3.28 adds a modmap command file to the command line that it
writes into compile_commands.json but the file does not actually
exist outside of cmake compiling. This means the command line in
the compile_commands.json fails with an error about the missing
file:
```
error: no such file or directory: '@sus/CMakeFiles/subspace_unittests.dir/mem/addressof_unittest.cc.o.modmap'
```

To work around this we can strip out the use of the modmap command-file
for now. This will break compilation later when C++ 20 modules are being
used though. CMake will need to find a way to provide a command line in
the compile_commands.json file which can actually be used by other tools
to reproduce the compilation step.

CMake discussion: https://discourse.cmake.org/t/how-to-control-the-location-of-the-c-20-binary-module-interface-bmi-output-directory/7968/13
Fixes chromium#437
danakj added a commit that referenced this issue Dec 19, 2023
CMake 3.28 adds a modmap command file to the command line that it
writes into compile_commands.json but the file does not actually
exist outside of cmake compiling. This means the command line in
the compile_commands.json fails with an error about the missing
file:
```
error: no such file or directory: '@sus/CMakeFiles/subspace_unittests.dir/mem/addressof_unittest.cc.o.modmap'
```

To work around this we can strip out the use of the modmap command-file
for now. This will break compilation later when C++ 20 modules are being
used though. CMake will need to find a way to provide a command line in
the compile_commands.json file which can actually be used by other tools
to reproduce the compilation step.

CMake discussion: https://discourse.cmake.org/t/how-to-control-the-location-of-the-c-20-binary-module-interface-bmi-output-directory/7968/13
Fixes #437
@danakj
Copy link
Collaborator Author

danakj commented Dec 22, 2023

A followup blog post was made about this issue specifically: https://nibblestew.blogspot.com/2023/12/even-more-breakage-in-c-module-world.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant