support of icx/icpx and dpcpp toolchain #2474

jonathanpoelen · 2022-06-19T02:15:59Z

Is your feature request related to a problem? Please describe.

xmake should be able to use icx, icpx, dpcpp and dpcpp-cl who are intel llvm compiler.

Describe the solution you'd like

xmake f --toolchain=icx and xmake f --toolchain=dpcpp should work.

icx is a C compiler, others are C++ compiler.

dpcpp-cl is like clang-cl, others are like clang.

I don't know to what extent -E is used, but you should know that dpcpp (and dpcpp-cl?) has a problem with the -E flag: it expects a syntactically valid file, as if it were compiling.

$ cat a.cpp
__clang_major__
$ dpcpp a.cpp
a.cpp:1:1: error: expected unqualified-id
__clang_major__
^
<built-in>:404:25: note: expanded from here
#define __clang_major__ 14
                        ^
1 error generated.
$ cat b.cpp
int x = __clang_major__;
$ dpcpp b.cpp
[...]
int x = 14;
[...]
no error

Reading the file from stdin doesn't work either (it seems to want to read the stream twice)

$ dpcpp -x c++ - <<<'__clang_major__'
same errors as above
$ dpcpp -x c++ - <<<'int x = __clang_major__;'
append-file: input file not found

Describe alternatives you've considered

xmake f --toolchain=clang --cxx=icpx --cc=icx, but it doesn't work

error: ...mdir/core/sandbox/modules/import/core/base/scheduler.lua:56: cannot find known tool script for icpx
stack traceback:
	@programdir/core/base/utils.lua:290: in function <@programdir/core/base/utils.lua:280>
	[C]: in function 'error'
	@programdir/core/base/os.lua:872: in function 'os.raiselevel'
	(...tail calls...)
	...mdir/core/sandbox/modules/import/core/base/scheduler.lua:56: in field 'co_start_withopt'
	@programdir/modules/private/async/runjobs.lua:217: in function <@programdir/modules/private/async/runjobs.lua:162>
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'sandbox/modules/utils.trycall'
	@programdir/core/base/scheduler.lua:504: in function 'base/scheduler.co_group_begin'
	...mdir/core/sandbox/modules/import/core/base/scheduler.lua:94: in field 'co_group_begin'
	@programdir/modules/private/async/runjobs.lua:162: in function <@programdir/modules/private/async/runjobs.lua:56>
	(...tail calls...)
	@programdir/actions/build/build.lua:252: in function <@programdir/actions/build/build.lua:240>
	(...tail calls...)
	@programdir/actions/build/main.lua:103: in global '_do_build'
	@programdir/actions/build/main.lua:150: in function <@programdir/actions/build/main.lua:144>
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'sandbox/modules/utils.trycall'
	@programdir/core/sandbox/modules/try.lua:121: in global 'try'
	@programdir/actions/build/main.lua:142: in function <@programdir/actions/build/main.lua:108>
	(...tail calls...)
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'sandbox/modules/utils.trycall'
	(...tail calls...)
	@programdir/core/base/task.lua:519: in function 'base/task.run'
	@programdir/core/main.lua:278: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>

Additional context

No response

The text was updated successfully, but these errors were encountered:

waruqi · 2022-06-19T04:05:10Z

try xmake f --toolchain=icc

jonathanpoelen · 2022-06-19T14:44:27Z

icc/icpc are another compilers not based on llvm. Also, forcing paths doesn't work

$ xmake f --toolchain=icc --cxx=icpc --cc=icc && xmake -vD
checking for platform ... linux
checking for architecture ... x86_64
checking for Intel C/C++ Compiler (x86_64) ... ok
checking for icpc ... /home/jonathan/bin/icpc
checking for flags (-fPIC) ... ok
> icpc "-fPIC"
checking for icpc ... /opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc
checking for the linker (ld) ... icpc
checking for /opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc ... ok
checking for flags (-fPIC) ... no
> icpc "-fPIC"
checkinfo: @programdir/core/sandbox/modules/os.lua:257: ld: cannot find -limf: No such file or directory
ld: cannot find -lsvml: No such file or directory
ld: cannot find -lirng: No such file or directory

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:872]:
    [@programdir/core/sandbox/modules/os.lua:257]: in function 'runv'
    [@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
[ 50%]: linking.release test
/opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc -o build/linux/x86_64/release/test build/.objs/test/linux/x86_64/release/test.cpp.o -m64

icc/icpc or /home/jonathan/bin/icc//home/jonathan/bin/icpc should be used here. Who in are wrappers that add -shared-intel -L /opt/intel/oneapi/compiler/2022.1.0/linux/compiler/lib/intel64_lin/. Doing it at --ldflags= does not remove all errors, but compiles.

waruqi · 2022-06-19T15:20:50Z

try xmake f --toolchain=icc --cc=/home/jonathan/bin/icc --cxx=/home/jonathan/bin/icpc -c

jonathanpoelen · 2022-06-19T16:39:01Z

Still the same error. The wrong icpc is being used.

waruqi · 2022-06-20T07:45:04Z

please run

xmake f --toolchain=icc --cc=/home/jonathan/bin/icc --cxx=/home/jonathan/bin/icpc -cvD
xmake -rvD

and give me full logs.

jonathanpoelen · 2022-06-20T13:50:11Z

checking for platform ... linux
checking for architecture ... x86_64
checking for icc ... /opt/intel/oneapi/compiler/latest/linux/bin/intel64/icc
checking for Intel C/C++ Compiler (x86_64) ... ok
configure
{
    buildir = build
    toolchain = icc
    cc = /home/jonathan/bin/icc
    plat = linux
    mode = release
    cxx = /home/jonathan/bin/icpc
    clean = true
    ccache = true
    ndk_stdcxx = true
    kind = static
    arch = x86_64
    host = linux
}
checking for /home/jonathan/bin/icpc ... ok
checking for flags (-fPIC) ... ok
> icpc "-fPIC"
[ 25%]: ccache compiling.release test.cpp
/home/jonathan/bin/icpc -c -m64 -o build/.objs/test/linux/x86_64/release/test.cpp.o test.cpp
checking for flags (-MMD -MF) ... ok
> icpc "-MMD" "-MF" "/dev/null"
checking for icpc ... /opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc
checking for the linker (ld) ... icpc
checking for /opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc ... ok
checking for flags (-fPIC) ... no
> icpc "-fPIC"
checkinfo: @programdir/core/sandbox/modules/os.lua:257: ld: cannot find -limf: No such file or directory
ld: cannot find -lsvml: No such file or directory
ld: cannot find -lirng: No such file or directory

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:872]:
    [@programdir/core/sandbox/modules/os.lua:257]: in function 'runv'
    [@programdir/modules/detect/tools/gcc/has_flags.lua:42]:
[ 50%]: linking.release test
/opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc -o build/linux/x86_64/release/test build/.objs/test/linux/x86_64/release/test.cpp.o -m64
error: @programdir/modules/private/async/runjobs.lua:256: @programdir/actions/build/kinds/binary.lua:74: @programdir/core/sandbox/modules/os.lua:257: ld: cannot find -limf: No such file or directory
ld: cannot find -lsvml: No such file or directory
ld: cannot find -lirng: No such file or directory

stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:872]:
    [@programdir/core/sandbox/modules/os.lua:257]: in function 'runv'
    [@programdir/modules/core/tools/gcc.lua:363]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:280]:
    [@programdir/actions/build/kinds/binary.lua:74]: in function 'callback'
    [@programdir/modules/core/project/depend.lua:189]: in function 'on_changed'
    [@programdir/actions/build/kinds/binary.lua:55]: in function '_do_link_target'
    [@programdir/actions/build/kinds/binary.lua:102]:
    [@programdir/actions/build/kinds/binary.lua:129]: in function '_link_target'
    [@programdir/actions/build/kinds/binary.lua:157]: in function 'jobfunc'
    [@programdir/modules/private/async/runjobs.lua:232]:
    [C]: in function 'xpcall'
    [@programdir/core/base/utils.lua:280]: in function 'trycall'
    [@programdir/core/sandbox/modules/try.lua:121]: in function 'try'
    [@programdir/modules/private/async/runjobs.lua:218]: in function 'cotask'
    [@programdir/core/base/scheduler.lua:388]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:872: in function 'base/os.raiselevel'
	(...tail calls...)
	@programdir/modules/private/async/runjobs.lua:256: in field 'catch'
	@programdir/core/sandbox/modules/try.lua:127: in global 'try'
	@programdir/modules/private/async/runjobs.lua:218: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>

waruqi · 2022-06-20T15:06:54Z

you can update to dev and try it again. xmake update -s dev

jonathanpoelen · 2022-06-20T16:07:33Z

Same, only difference is the line number of the stack trace

-    [@programdir/modules/core/tools/gcc.lua:363]:
+    [@programdir/modules/core/tools/gcc.lua:368]:

waruqi · 2022-06-20T16:09:24Z

this patch 47d4445

jonathanpoelen · 2022-06-20T18:02:50Z

This is what I have.

waruqi · 2022-06-21T03:50:42Z

It should work, can you try

xmake f --toolchain=icc --cc=/home/jonathan/bin/icc --cxx=/home/jonathan/bin/icpc -cvD
xmake -rvD

and give me all output again?

waruqi · 2022-06-21T12:13:46Z

/home/jonathan/bin/icpc -c -m64 -o build/.objs/test/linux/x86_64/release/test.cpp.o test.cpp

I saw your previous logs, it works! /home/jonathan/bin/icpc

/opt/intel/oneapi/compiler/latest/linux/bin/intel64/icpc -o build/linux/x86_64/release/test build/.objs/test/linux/x86_64/release/test.cpp.o -m64

but you just switch compiler to /home/jonathan/bin/icpc, do not switch linker to /home/jonathan/bin/icpc.

if you want to switch linker, you need set --ld=/home/jonathan/bin/icpc

xmake f --toolchain=icc --cc=/home/jonathan/bin/icc --cxx=/home/jonathan/bin/icpc --ld=/home/jonathan/bin/icpc -cvD
xmake -rvD

jonathanpoelen · 2022-06-21T22:50:37Z

Oh... Quite simply.

Otherwise, for icx? It doesn't work specifying it as the compiler, either with toolset=icc (which is wrong) or toolset=clang. Same log in both cases:

$ xmake f --toolchain=clang --cc=/home/jonathan/bin/icx --cxx=/home/jonathan/bin/icpx --ld=/home/jonathan/bin/icpx -cvD
checking for platform ... linux
checking for architecture ... x86_64
checking for clang ... /usr/bin/clang
configure
{
    toolchain = clang
    cxx = /home/jonathan/bin/icpx
    mode = release
    ndk_stdcxx = true
    host = linux
    kind = static
    ld = /home/jonathan/bin/icpx
    plat = linux
    arch = x86_64
    clean = true
    ccache = true
    buildir = build
    cc = /home/jonathan/bin/icx
}
$ xmake -rvD    
error: @programdir/core/main.lua:280: @programdir/actions/build/main.lua:166: ...mdir/core/sandbox/modules/import/core/base/scheduler.lua:96: ...mdir/core/sandbox/modules/import/core/base/scheduler.lua:56: @programdir/modules/private/async/runjobs.lua:256: ...amdir/core/sandbox/modules/import/core/tool/compiler.lua:37: cannot find known tool script for /home/jonathan/bin/icpx
stack traceback:
    [C]: in function 'error'
    [@programdir/core/base/os.lua:872]:
    [...amdir/core/sandbox/modules/import/core/tool/compiler.lua:37]: in function 'load'
    [@programdir/modules/private/action/build/object.lua:40]: in function 'script'
    [@programdir/modules/private/action/build/object.lua:98]: in function 'build_object'
    [@programdir/modules/private/action/build/object.lua:123]: in function 'jobfunc'
    [@programdir/modules/private/async/runjobs.lua:232]:

stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:872: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/modules/private/async/runjobs.lua:256: in field 'catch'
	@programdir/core/sandbox/modules/try.lua:127: in global 'try'
	@programdir/modules/private/async/runjobs.lua:218: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>
stack traceback:
	@programdir/core/base/utils.lua:290: in function <@programdir/core/base/utils.lua:280>
	[C]: in function 'error'
	@programdir/core/base/os.lua:872: in function 'os.raiselevel'
	(...tail calls...)
	...mdir/core/sandbox/modules/import/core/base/scheduler.lua:56: in field 'co_start_withopt'
	@programdir/modules/private/async/runjobs.lua:217: in function <@programdir/modules/private/async/runjobs.lua:162>
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'base/utils.trycall'
	@programdir/core/base/scheduler.lua:504: in function 'base/scheduler.co_group_begin'
	...mdir/core/sandbox/modules/import/core/base/scheduler.lua:94: in field 'co_group_begin'
	@programdir/modules/private/async/runjobs.lua:162: in function <@programdir/modules/private/async/runjobs.lua:56>
	(...tail calls...)
	@programdir/actions/build/build.lua:252: in function <@programdir/actions/build/build.lua:240>
	(...tail calls...)
	@programdir/actions/build/main.lua:103: in global '_do_build'
	@programdir/actions/build/main.lua:150: in function <@programdir/actions/build/main.lua:144>
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'base/utils.trycall'
	@programdir/core/sandbox/modules/try.lua:121: in global 'try'
	@programdir/actions/build/main.lua:142: in function <@programdir/actions/build/main.lua:108>
	(...tail calls...)
	[C]: in function 'xpcall'
	@programdir/core/base/utils.lua:280: in function 'base/utils.trycall'
	(...tail calls...)
	@programdir/core/base/task.lua:501: in function 'base/task.run'
	@programdir/core/main.lua:278: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>
stack traceback:
	[C]: in function 'error'
	@programdir/core/base/os.lua:872: in function 'os.raiselevel'
	(...tail calls...)
	@programdir/core/main.lua:280: in upvalue 'cotask'
	@programdir/core/base/scheduler.lua:388: in function <@programdir/core/base/scheduler.lua:385>

icc and icx are really 2 different compilers, for example:

test.cpp

template<auto T> struct A {};
struct B { using type = int; };

int main() {
  A<B{}>();
}

$ icpx -std=c++20 -fsanitize=address test.cpp
ok, compiles and asan is recognized
$ icpc -std=c++20 -fsanitize=address test.cpp
icpc: command line warning #10148: option '-fsanitize=address' not supported
test.cpp(5): error: a nontype template parameter may not have class type
    A<B{}>();
      ^

compilation aborted for test.cpp (code 2)

icx is clang/llvm + Intel proprietary optimizations and code generation.

waruqi · 2022-06-22T06:57:23Z

Yes, only the icc and icpc compilers are currently supported, not the icx icpx compiler, which is unknown to xmake.

I don't have the icx icpx compiler environment to support them or test them at the moment. You can open pr to support them.

you can add tools/icpx.lua and tools/icx.lua, like this https://github.com/xmake-io/xmake/blob/master/xmake/modules/core/tools/icpc.lua

and add find_icx.lua and find_icpx.lua, like this https://github.com/xmake-io/xmake/blob/master/xmake/modules/detect/tools/find_icpc.lua

explocion · 2022-10-27T20:46:10Z

It seems that for dpcpp toolchain, xmake will check for icxenv. However, dpcpp should be standalone.

waruqi · 2022-10-28T01:00:09Z

It seems that for dpcpp toolchain, xmake will check for icxenv. However, dpcpp should be standalone.

It just use find_icxenv to find dpcpp,

xmake/xmake/toolchains/dpcpp/check.lua

Line 33 in cea2f82

icxenv = find_icxenv()

you can also improve it and open a pr.

explocion · 2022-10-28T16:00:11Z

I mean the dpcpp supported in xmake now refers to a part of Intel's oneapi framework. However, I refer to the dpcpp that is built as a standalone dpcpp-llvm compiler. I'm thinking of renaming the current dpcpp toolchain as oneapi and adding a dpcpp-llvm toolchain as a variant of current llvm toolchain.

waruqi · 2022-10-28T16:04:46Z

oneapi contains many compilers , icc, icx, ifortran, dpcpp, and we should not rename dpcpp to oneapi. but you can add dpcpp-llvm as a separate toolchain.

jonathanpoelen added the feature request label Jun 19, 2022

This was referenced Jun 27, 2022

add dpcpp toolchain #2499

Merged

add icx toolchain #2498

Merged

waruqi closed this as completed Jun 27, 2022

waruqi added this to the v2.6.9 milestone Jun 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support of icx/icpx and dpcpp toolchain #2474

support of icx/icpx and dpcpp toolchain #2474

jonathanpoelen commented Jun 19, 2022

waruqi commented Jun 19, 2022

jonathanpoelen commented Jun 19, 2022

waruqi commented Jun 19, 2022

jonathanpoelen commented Jun 19, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 21, 2022

waruqi commented Jun 21, 2022

jonathanpoelen commented Jun 21, 2022

waruqi commented Jun 22, 2022 •

edited

Loading

explocion commented Oct 27, 2022

waruqi commented Oct 28, 2022

explocion commented Oct 28, 2022

waruqi commented Oct 28, 2022

support of icx/icpx and dpcpp toolchain #2474

support of icx/icpx and dpcpp toolchain #2474

Comments

jonathanpoelen commented Jun 19, 2022

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

waruqi commented Jun 19, 2022

jonathanpoelen commented Jun 19, 2022

waruqi commented Jun 19, 2022

jonathanpoelen commented Jun 19, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 20, 2022

jonathanpoelen commented Jun 20, 2022

waruqi commented Jun 21, 2022

waruqi commented Jun 21, 2022

jonathanpoelen commented Jun 21, 2022

waruqi commented Jun 22, 2022 • edited Loading

explocion commented Oct 27, 2022

waruqi commented Oct 28, 2022

explocion commented Oct 28, 2022

waruqi commented Oct 28, 2022

waruqi commented Jun 22, 2022 •

edited

Loading