Incremental compilation for module not supported on gcc 14

[c8ef]

Xmake Version

latest

Operating System Version and Architecture

Fedora with gcc 14

Describe Bug

Incremental compilation for module not supported

Expected Behavior

Incremental compilation for module work

Project Configuration

xmake f -c --yes

Additional Information and Error Logs

On gcc 14:

checking for flags (gcc_deps_format) ... ok
> gcc "-fdeps-format=p1689r5" "-m64"
checkinfo: @programdir/core/sandbox/modules/os.lua:273: cc1plus: error: to generate dependencies you must specify either ‘-M’ or ‘-MM’

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:973]:
    [@programdir/core/sandbox/modules/os.lua:273]: in function 'runv'
    [@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
checking for flags (gcc_deps_file) ... ok
> gcc "-fdeps-file=/tmp/.xmake0/240427/_AE562500993648108D2040DE53327830" "-m64"
checkinfo: @programdir/core/sandbox/modules/os.lua:273: cc1plus: error: to generate dependencies you must specify either ‘-M’ or ‘-MM’

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:973]:
    [@programdir/core/sandbox/modules/os.lua:273]: in function 'runv'
    [@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
checking for flags (gcc_deps_output) ... ok
> gcc "-fdeps-target=/tmp/.xmake0/240427/_551F05A8539645208A5B33788744C430.o" "-m64"
checkinfo: @programdir/core/sandbox/modules/os.lua:273: cc1plus: error: to generate dependencies you must specify either ‘-M’ or ‘-MM’

It means p1689r5 is shipped in gcc 14. In gcc 13, we use the fallback deps scanner, so the incremental build work as expected.

gcc 14:

{
  "rules": [
    {
      "primary-output": "build/.objs/submodules2/linux/x86_64/release/src/main_module.mpp.o",
      "provides": [
        {
          "logical-name": "main_module",
          "is-interface": true
        }
      ],
      "requires": [
        {
          "logical-name": "main_module.module1"
        },
        {
          "logical-name": "main_module.module2"
        }
      ]
    }
  ],
  "version": 0,
  "revision": 0
}

fallback:

{
  "version": 1,
  "revision": 0,
  "rules": [
    {
      "provides": [
        {
          "source-path": "src/main_module.mpp",
          "logical-name": "main_module",
          "compiled-module-path": "build/.gens/submodules2/linux/x86_64/release/rules/bmi/cache/modules/bb5e7e0a/main_module.gcm",
          "is-interface": true
        }
      ],
      "requires": [
        {
          "logical-name": "main_module.module1"
        },
        {
          "logical-name": "main_module.module2"
        }
      ],
      "outputs": [
        "build/.gens/submodules2/linux/x86_64/release/rules/bmi/cache/modules/bb5e7e0a/main_module.mpp.json"
      ],
      "primary-output": "build/.objs/submodules2/linux/x86_64/release/src/main_module.mpp.o"
    }
  ]
}

I debugged the code, but I'm not sure how xmake decides when to trigger a rebuild. Can anyone provide some insights?

[waruqi]

@Arthapz Any idea? https://github.com/xmake-io/xmake/actions/runs/8858190130/job/24326535957

[Arthapz]

@Arthapz Any idea? https://github.com/xmake-io/xmake/actions/runs/8858190130/job/24326535957

on it, but i need to build a GCC 14 because archlinux haven't rolled out GCC 14

[c8ef]

@Arthapz Any idea? https://github.com/xmake-io/xmake/actions/runs/8858190130/job/24326535957

on it, but i need to build a GCC 14 because archlinux haven't rolled out GCC 14

Using fedora rawhide docker image will suffice😋

[waruqi]

@Arthapz Any idea? https://github.com/xmake-io/xmake/actions/runs/8858190130/job/24326535957

on it, but i need to build a GCC 14 because archlinux haven't rolled out GCC 14

try this docker image, https://github.com/waruqi/dockerfiles/blob/master/fedora/Dockerfile

[Arthapz]

@Arthapz Any idea? https://github.com/xmake-io/xmake/actions/runs/8858190130/job/24326535957

on it, but i need to build a GCC 14 because archlinux haven't rolled out GCC 14

Using fedora rawhide docker image will suffice😋

I need to try other things with GCC master branch (gcc std module) so I need compile it (well it's done, AUR packages make it easy) D

[Arthapz]

mmmmh, incremental build for gcc is also broken on fallback dependency scanner

[Arthapz]

mmmh seems like something is invalidating dependinfo of cppfiles

xmake/xmake/rules/c++/modules/modules_support/builder.lua

Line 187 in 61498e2

if dryrun or depend.is_changed(dependinfo, {lastmtime = lastmtime, values = depvalues}) then

and seems like it's the mtime

xmake  b -vD
[check_build_deps]: file /tmp/.xmake1000/240428/hello/src/main.cpp is changed, mtime: 1714329688, lastmtime: 1714329682
[check_build_deps]: file src/main.cpp is changed, mtime: 1712014483, lastmtime: 0
[ 71%]: compiling.release src/main.cpp
/usr/bin/gcc -c -m64 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -std=c++20 -fmodules-ts -D_GLIBCXX_USE_CXX11_ABI=0 -DNDEBUG -fmodule-mapper=/tmp/.xmake1000/240428/hello/src/main.cpp -o build/.ob
js/hello/linux/x86_64/release/src/main.cpp.o src/main.cpp
[check_build_deps]: file build/.objs/hello/linux/x86_64/release/src/main.cpp.o is changed, mtime: 1714329688, lastmtime: 1714329682
[ 85%]: linking.release hello
/usr/bin/g++ -o build/linux/x86_64/release/hello build/.objs/hello/linux/x86_64/release/src/main.cpp.o build/.objs/hello/linux/x86_64/release/src/hello.mpp.o -m64 -s

build cache stats:
cache directory: build/.build_cache
cache hit rate: 0%
cache hit: 0
cache hit total time: 0,000s
cache miss: 0
cache miss total time: 0,000s
new cached files: 0
remote cache hit: 0
remote new cached files: 0
preprocess failed: 0
compile fallback count: 0
compile total time: 0,000s

[100%]: build ok, spent 0,068s

[Arthapz]

function is_changed(dependinfo, opt)
    print(dependinfo.files)

> xmake b
{ 
  "src/main.cpp",
  "/tmp/.xmake1000/240428/hello/src/main.cpp",
  "src/main.cpp" 
}
[100%]: build ok, spent 0,011s

> xmake b
{ 
  "src/main.cpp",
  "src/main.cpp",
  "/tmp/.xmake1000/240428/hello/src/main.cpp" 
}
[ 71%]: compiling.release src/main.cpp
[ 85%]: linking.release hello
[100%]: build ok, spent 0,057s

> xmake b
{ 
  "src/main.cpp",
  "/tmp/.xmake1000/240428/hello/src/main.cpp",
  "src/main.cpp" 
}

[ 71%]: compiling.release src/main.cpp
[ 85%]: linking.release hello
[100%]: build ok, spent 0,061s

> xmake b
{ 
  "src/main.cpp",
  "/tmp/.xmake1000/240428/hello/src/main.cpp",
  "src/main.cpp" 
}
[100%]: build ok, spent 0,013s

@waruqi any idea how we can have always the same order for dependfiles ? it's seems this is the root cause here
rebuild is trigger because dependinfo.file is not always in the same order

/tmp/.xmake1000/240428/hello/src/main.cpp is the mapper file added with fileconfig_add

[waruqi]

This is not the root cause, it doesn't depend on the order of the files, just that the mtime of the files has been changed.

The root cause is that gcc14 generates some different dependency information.

build/.deps/hello/linux/x86_64/release/src/main.cpp.o.d in gcc-13

 build/.objs/hello/linux/x86_64/release/src/main.cpp.o: src/main.cpp
build/.objs/hello/linux/x86_64/release/src/main.cpp.o: hello.c++m
CXX_IMPORTS += hello.c++m

build/.deps/hello/linux/x86_64/release/src/main.cpp.o.d in gcc-14

 build/.objs/hello/linux/x86_64/release/src/main.cpp.o: src/main.cpp  /tmp/.xmake995/240429/hello/src/main.cpp  build/.gens/hello/linux/x86_64/release/rules/bmi/cache/modules/c83caa2b/hello.gcm
build/.objs/hello/linux/x86_64/release/src/main.cpp.o: hello.c++m
CXX_IMPORTS += hello.c++m

so /tmp/.xmake995/240429/hello/src/main.cpp will be inserted dependinfo.files

xmake/xmake/modules/core/project/depend.lua

Line 35 in 61498e2

table.join2(dependinfo.files, depfiles)

but this file is always changed when doing incremental build.

xmake/xmake/rules/c++/modules/modules_support/gcc/builder.lua

Line 199 in 61498e2

module_mapper = _generate_modulemapper_file(target, opt.module, opt.cppfile)

About files order issues.

Because result is a hashset. it returned a unstable-order dep files array. But we need not fix it.

xmake/xmake/modules/private/tools/gcc/parse_deps.lua

Line 142 in 61498e2

return results:to_array()

[waruqi]

Therefore, we should avoid re-generating the mapping file each time.

[waruqi]

see #5030