Fix clang submodules #1982

aacirino · 2022-01-17T19:02:53Z

I made this pull request due to build failures with submodules and clang 13. I've tested it with the project xmake/tests/projects/c++/modules/submodules and it runs as expected.

…mefix

waruqi · 2022-01-17T23:46:31Z

xmake/rules/c++/modules/build_modules/module_parser.lua

-                local moduleinfo = data.moduleinfo
-                moduledeps = moduledeps or {}
-                moduledeps[moduleinfo.name] = moduleinfo
+                if data.moduleinfo then


and

local moduleinfo = data.moduleinfo if moduleinfo then moduledeps = moduledeps or {} moduledeps[moduleinfo.name] = moduleinfo end

waruqi · 2022-01-17T23:47:16Z

xmake/rules/c++/modules/build_modules/clang.lua

@@ -100,6 +100,14 @@ function build_with_batchjobs(target, batchjobs, sourcebatch, opt)
            opt2.objectfile = modulefiles[i]
            opt2.dependfile = target:dependfile(opt2.objectfile)
            opt2.sourcekind = assert(sourcebatch.sourcekind, "%s: sourcekind not found!", sourcefile)
+            if moduledep["deps"] then


use moduledep.deps moduledep.name

waruqi · 2022-01-17T23:48:22Z

xmake/rules/c++/modules/build_modules/clang.lua

+                    if found == nil then
+                        target:add("cxxflags", "-fmodule-file=" .. modulefile, {force = true})
+                    end
+                end


why? The code below will add it

xmake/xmake/rules/c++/modules/build_modules/clang.lua

Lines 122 to 129 in 1b2a5da

count = count + 1

if count == sourcefiles_total then

target:add("cxxflags", modulesflag, "-fmodules-cache-path=" .. cachedir, {force = true})

-- FIXME It is invalid for the module implementation unit

--target:add("cxxflags", "-fimplicit-modules", "-fimplicit-module-maps", "-fprebuilt-module-path=" .. cachedir, {force = true})

for _, modulefile in ipairs(modulefiles) do

target:add("cxxflags", "-fmodule-file=" .. modulefile, {force = true})

end

The code above doesn't add the pcm file to the compilation.

We've turned on -fimplicit-modules and module cache, clang should find submodules without explicitly setting -fmodule-file.

The root cause of it is that the module files for cache/module1.pcm are not named correctly, in case of submodules, it should be cache/main_module.module1.pcm so that clang can find them correctly.

I have tested your demo project and confirmed that it can be fixed successfully.

waruqi · 2022-01-18T01:16:53Z

I can't reproduce it, It work fine here

xmake f --toolchain=clang-13 -cvD;xmake -r  
checking for platform ... linux
checking for architecture ... x86_64
checking for clang-13 ... /usr/bin/clang-13
configure
{
    ndk_stdcxx = true
    ccache = true
    arch = x86_64
    mode = release
    buildir = build
    toolchain = clang-13
    clean = true
    plat = linux
    host = linux
    kind = static
}
[ 14%]: ccache compiling.release src/math.math1.mpp
[ 14%]: ccache compiling.release src/math.math2.mpp
[ 42%]: ccache compiling.release src/math.mpp
[ 57%]: ccache compiling.release src/main.cpp
[ 71%]: linking.release math
[100%]: build ok!

Can you give a reproducible example? or provide your error output?

aacirino · 2022-01-18T13:09:04Z

This is the complete flow that led to the error and the build with my patches

[~/git/modfix]$ tree
.
├── build
│   └── linux
│       └── x86_64
│           └── release
├── p.lua
├── src
│   ├── main.cpp
│   ├── main_module.mpp
│   ├── module1.cpp
│   ├── module1.mpp
│   ├── module2.cpp
│   └── module2.mpp
└── xmake.lua

5 directories, 8 files
[~/git/modfix]$ cat src/main.cpp
import main_module;

int main(int argc, char** argv) {
  hello1();
  hello2();
  return 0;
}
[~/git/modfix]$ cat src/main_module.mpp 
export module main_module;

export import main_module.module1;
export import main_module.module2;

export void hello() {
  hello1();
  hello2();
}
[~/git/modfix]$ cat src/module1.mpp    
export module main_module.module1;

export void hello1(void);
[~/git/modfix]$ cat src/module1.cpp
module main_module.module1;

import <iostream>;

void hello1(void) {
  std::cout << "Hello One\n";
}
[~/git/modfix]$ cat src/module2.mpp
export module main_module.module2;

export void hello2(void);
[~/git/modfix]$ cat src/module2.cpp
module main_module.module2;

import <iostream>;

void hello2(void) {
  std::cout << "Hello Two\n";
}
[~/git/modfix]$ cat xmake.lua      
add_rules("mode.debug", "mode.release")

target("modfix")
    set_kind("binary")
    add_files("src/*.cpp", "src/*.mpp")
    add_includedirs(
      "/usr/local/include",
      "$(shell brew --prefix)/include",
      "$(shell brew --prefix)/opt/llvm/include"
    )
    add_cxxflags(
      "-march=native",
      "-std=c++20",
      "-isystem $LLVM_DIR/include/c++/v1/include",
      "-stdlib=libc++"
    )
    add_ldflags(
      "-fuse-ld=mold"
    )
    add_links("c++")
target_end()
[~/git/modfix]$ which xmake
/home/linuxbrew/.linuxbrew/bin/xmake
[~/git/modfix]$ xmake clean && clear && xmake -b -v -j 6
generating.moduledeps src/module1.mpp
generating.moduledeps src/module2.mpp
generating.moduledeps src/main_module.mpp
[ 11%]: ccache compiling.release src/module1.mpp
/home/linuxbrew/.linuxbrew/bin/ccache /home/linuxbrew/.linuxbrew/opt/llvm/bin/clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -I/usr/local/include -I/home/linuxbrew/.linuxbrew/include -I/home/linuxbrew/.linuxbrew/opt/llvm/include -march=native -std=c++20 -isystem $LLVM_DIR/include/c++/v1/include -stdlib=libc++ -DNDEBUG -fmodules -fimplicit-modules -fimplicit-module-maps -fprebuilt-module-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache --precompile -x c++-module -fmodules-cache-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -o build/.gens/modfix/linux/x86_64/release/rules/modules/cache/module1.pcm src/module1.mpp
[ 11%]: ccache compiling.release src/module2.mpp
/home/linuxbrew/.linuxbrew/bin/ccache /home/linuxbrew/.linuxbrew/opt/llvm/bin/clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -I/usr/local/include -I/home/linuxbrew/.linuxbrew/include -I/home/linuxbrew/.linuxbrew/opt/llvm/include -march=native -std=c++20 -isystem $LLVM_DIR/include/c++/v1/include -stdlib=libc++ -DNDEBUG -fmodules -fimplicit-modules -fimplicit-module-maps -fprebuilt-module-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache --precompile -x c++-module -fmodules-cache-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -o build/.gens/modfix/linux/x86_64/release/rules/modules/cache/module2.pcm src/module2.mpp
/home/linuxbrew/.linuxbrew/bin/ccache /home/linuxbrew/.linuxbrew/opt/llvm/bin/clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -I/usr/local/include -I/home/linuxbrew/.linuxbrew/include -I/home/linuxbrew/.linuxbrew/opt/llvm/include -march=native -std=c++20 -isystem $LLVM_DIR/include/c++/v1/include -stdlib=libc++ -DNDEBUG -fmodules -fmodules-cache-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -fimplicit-modules -fimplicit-module-maps -fprebuilt-module-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -o build/.objs/modfix/linux/x86_64/release/src/module1.mpp.o build/.gens/modfix/linux/x86_64/release/rules/modules/cache/module1.pcm
/home/linuxbrew/.linuxbrew/bin/ccache /home/linuxbrew/.linuxbrew/opt/llvm/bin/clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -I/usr/local/include -I/home/linuxbrew/.linuxbrew/include -I/home/linuxbrew/.linuxbrew/opt/llvm/include -march=native -std=c++20 -isystem $LLVM_DIR/include/c++/v1/include -stdlib=libc++ -DNDEBUG -fmodules -fmodules-cache-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -fimplicit-modules -fimplicit-module-maps -fprebuilt-module-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -o build/.objs/modfix/linux/x86_64/release/src/module2.mpp.o build/.gens/modfix/linux/x86_64/release/rules/modules/cache/module2.pcm
[ 33%]: ccache compiling.release src/main_module.mpp
/home/linuxbrew/.linuxbrew/bin/ccache /home/linuxbrew/.linuxbrew/opt/llvm/bin/clang -c -Qunused-arguments -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -I/usr/local/include -I/home/linuxbrew/.linuxbrew/include -I/home/linuxbrew/.linuxbrew/opt/llvm/include -march=native -std=c++20 -isystem $LLVM_DIR/include/c++/v1/include -stdlib=libc++ -DNDEBUG -fmodules -fimplicit-modules -fimplicit-module-maps -fprebuilt-module-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache --precompile -x c++-module -fmodules-cache-path=build/.gens/modfix/linux/x86_64/release/rules/modules/cache -o build/.gens/modfix/linux/x86_64/release/rules/modules/cache/main_module.pcm src/main_module.mpp
error: src/main_module.mpp:3:15: fatal error: module 'main_module.module1' not found
export import main_module.module1;
       ~~~~~~~^~~~~~~~~~~
1 error generated.
[~/git/modfix]$ /usr/local/bin/xmake.new clean && /usr/local/bin/xmake.new -b -j 6
[ 11%]: ccache compiling.release src/module1.mpp
[ 11%]: ccache compiling.release src/module2.mpp
[ 33%]: ccache compiling.release src/main_module.mpp
[ 44%]: ccache compiling.release src/module2.cpp
[ 44%]: ccache compiling.release src/module1.cpp
[ 44%]: ccache compiling.release src/main.cpp
[ 77%]: linking.release modfix
[100%]: build ok!
[~/git/modfix]$ /usr/local/bin/xmake.new r                                        
Hello One
Hello Two

waruqi · 2022-01-18T13:14:10Z

can you upload your demo project?

aacirino · 2022-01-18T14:31:52Z

The zip file is attached modfix.zip <https://drive.google.com/file/d/1nSYd-79nsJKSkJQaiAg7TzvQ5fuQdx1_/view?usp=drivesdk> Em ter., 18 de jan. de 2022 10:14, ruki ***@***.***> escreveu:

…

can you upload your demo project? — Reply to this email directly, view it on GitHub <#1982 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARARENZQEEA3HBCJHGOO5LUWVRS3ANCNFSM5MFMM3MA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

waruqi · 2022-01-18T14:34:56Z

thanks, I will look at it in these days

waruqi · 2022-01-19T00:58:57Z

The zip file is attached modfix.zip https://drive.google.com/file/d/1nSYd-79nsJKSkJQaiAg7TzvQ5fuQdx1_/view?usp=drivesdk Em ter., 18 de jan. de 2022 10:14, ruki @.> escreveu:
…
can you upload your demo project? — Reply to this email directly, view it on GitHub <#1982 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARARENZQEEA3HBCJHGOO5LUWVRS3ANCNFSM5MFMM3MA . You are receiving this because you authored the thread.Message ID: @.>

I cannot downlload it, can you upload it to github issues attachment?

waruqi · 2022-01-19T02:21:40Z

I added your demo project as test. https://github.com/xmake-io/xmake/tree/dev/tests/projects/c%2B%2B/modules/submodules2

aacirino · 2022-01-19T12:08:13Z

Here follows the test project
modfix.zip

waruqi · 2022-01-19T12:14:34Z

Does it work for this patch now?

aacirino · 2022-01-19T14:42:32Z

Yes, I merged the latest master commits and it is working fine. Thanx

aacirino added 7 commits July 15, 2021 23:23

Merge remote-tracking branch 'upstream/master'

81ad292

Merge branch 'master' of https://github.com/xmake-io/xmake

286ba62

Little modifications to modules build

885d00d

Merged from upstream/master

ed0c0c5

Merge branch 'master' of https://github.com/xmake-io/xmake into modna…

274244b

…mefix

Merge branch 'master' of https://github.com/xmake-io/xmake into modna…

92978c6

…mefix

Solved building submodules with clang

1b2a5da

waruqi reviewed Jan 17, 2022

View reviewed changes

waruqi added 2 commits January 19, 2022 09:55

Update clang.lua

cc30e77

Update module_parser.lua

f4056be

waruqi merged commit 093fcdb into xmake-io:dev Jan 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix clang submodules #1982

Fix clang submodules #1982

aacirino commented Jan 17, 2022

waruqi Jan 17, 2022

waruqi Jan 17, 2022

waruqi Jan 17, 2022

waruqi Jan 17, 2022

aacirino Jan 18, 2022

waruqi Jan 19, 2022

waruqi commented Jan 18, 2022 •

edited

Loading

aacirino commented Jan 18, 2022

waruqi commented Jan 18, 2022

aacirino commented Jan 18, 2022 via email

waruqi commented Jan 18, 2022

waruqi commented Jan 19, 2022

waruqi commented Jan 19, 2022

aacirino commented Jan 19, 2022

waruqi commented Jan 19, 2022

aacirino commented Jan 19, 2022

	count = count + 1
	if count == sourcefiles_total then
	target:add("cxxflags", modulesflag, "-fmodules-cache-path=" .. cachedir, {force = true})
	-- FIXME It is invalid for the module implementation unit
	--target:add("cxxflags", "-fimplicit-modules", "-fimplicit-module-maps", "-fprebuilt-module-path=" .. cachedir, {force = true})
	for _, modulefile in ipairs(modulefiles) do
	target:add("cxxflags", "-fmodule-file=" .. modulefile, {force = true})
	end

Fix clang submodules #1982

Fix clang submodules #1982

Conversation

aacirino commented Jan 17, 2022

waruqi Jan 17, 2022

Choose a reason for hiding this comment

waruqi Jan 17, 2022

Choose a reason for hiding this comment

waruqi Jan 17, 2022

Choose a reason for hiding this comment

waruqi Jan 17, 2022

Choose a reason for hiding this comment

aacirino Jan 18, 2022

Choose a reason for hiding this comment

waruqi Jan 19, 2022

Choose a reason for hiding this comment

waruqi commented Jan 18, 2022 • edited Loading

aacirino commented Jan 18, 2022

waruqi commented Jan 18, 2022

aacirino commented Jan 18, 2022 via email

waruqi commented Jan 18, 2022

waruqi commented Jan 19, 2022

waruqi commented Jan 19, 2022

aacirino commented Jan 19, 2022

waruqi commented Jan 19, 2022

aacirino commented Jan 19, 2022

waruqi commented Jan 18, 2022 •

edited

Loading