-
Notifications
You must be signed in to change notification settings - Fork 809
[SYCL][clang-linker-wrapper] Fix argument passing to ocloc. #20270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SYCL][clang-linker-wrapper] Fix argument passing to ocloc. #20270
Conversation
// "-g,-cl-opt-disable"`. Otherwise, only the first arg is processed by | ||
// ocloc as an arg for -options, and the rest are processed as standalone | ||
// flags, possibly leading to errors. | ||
std::pair<StringRef, StringRef> OptionsArgs = OptC.split("-options "); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i've never seen this before, i';ve always used quotes and that works -options "-g -cl-take-address -whatever", does it really not work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's another option, but it needs all the args to be within the quotes. Without this patch, the args for the example would be passed as -options "-g" -cl-opt-disable
. That breaks ocloc, because it processes cl-opt-disable
as a standalone flag, and errors out. I prefer to go with commas than playing with quotes, because they can cause weird escape issues, but I'm open to try that if you prefer it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no i agree commas is better if it works, i didnt even know it worked, cool
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you want me to update the description to include the alternative you mentioned and why I/we prefer to go with commas?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, just a one liner might be useful like 'Quotes would also work, but then we get into escaping issues"
StringRef OptC = | ||
Args.getLastArgValue(OPT_sycl_backend_compile_options_from_image_EQ); | ||
OptC.split(CmdArgs, " ", /*MaxSplit=*/-1, /*KeepEmpty=*/false); | ||
if (!IsCPU) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can we swap the branches so its not if(!something) else something and its if(something) else something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course. Will do all the changes in one commit.
// Only add if not empty, an empty arg can lead to ocloc errors. | ||
if (!OptionsArgs.first.empty()) | ||
CmdArgs.push_back(OptionsArgs.first); | ||
if (!OptionsArgs.second.empty()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is it possible OptionsArgs.first
is empty and OptionsArgs.second
is not empty?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, actually, my test case behaved like that. The way this split function works is that OptionsArgs.first
will be everything before the separator (-options
in our case), and OptionsArgs.second
will be everything after the separator. My test case was -options -g -cl-opt-disable
, so the first is empty because there is nothing before the separator.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok i see, but do we need to handle the case where there are some non-options related arguments after -options, or are we saying anything after -options must be part of -options?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did some investigation on that, I didn't find any case where we add anything else after -options
that doesn't belong to options. The only place I found where we use pass -options
is
llvm/clang/lib/Driver/ToolChains/SYCL.cpp
Line 1975 in dfc06dd
CmdArgs.push_back("-options"); |
-options
to try this scenario.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no i think that assumption is correct actually
// "-g,-cl-opt-disable"`. Otherwise, only the first arg is processed by | ||
// ocloc as an arg for -options, and the rest are processed as standalone | ||
// flags, possibly leading to errors. | ||
std::pair<StringRef, StringRef> OptionsArgs = OptC.split("-options "); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we could use structured bindings here to get some better variable names, eg
auto [OptionStr, OptionValues] = OptC.split("-options ");
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it, just not sure if some of our older CI systems (RHEL 7.6 I think) supports that. Do we use it anywhere else? Is it okay? I'm happy to go change it if it works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LLVM requires c++17 to build and this feature is in c++17 so we should be fine
// flags, possibly leading to errors. | ||
std::pair<StringRef, StringRef> OptionsArgs = OptC.split("-options "); | ||
// Only add if not empty, an empty arg can lead to ocloc errors. | ||
if (!OptionsArgs.first.empty()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i thought OptionArgs.first
was -options
but it seems we push that here and then `options" explicitly in the next branch, so probably im mistaken
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I explained in #20270 (comment). I thought so too when I first used the function, but you can see here the llvm doc: https://llvm.org/doxygen/classllvm_1_1StringRef.html#a0320b2a5a6d440bf4479a02e78cf5ca7
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given it's caused confusion to both you and me, I'll better add a comment in the code explaining, so nobody needs to go find the doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cool, for me the confusion was around the uncommon separate (usually ,
or something), if we explain that we're splitting around that i think it's clearer
StringRef OptC = | ||
Args.getLastArgValue(OPT_sycl_backend_compile_options_from_image_EQ); | ||
if (!IsCPU) { | ||
if (IsCPU) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
any way to add a test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be already. The case I'm fixing here is because a test failed downstream. Let me double-check if there are also for CPU case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So for the CPU case, I think this test covers it: https://github.com/intel/llvm/blob/sycl/sycl/test-e2e/NewOffloadDriver/cpu.cpp
For the GPU case, https://github.com/intel/llvm/blob/sycl/sycl/test-e2e/SYCLBIN/simple_kernel_aot. was the one failing downstream, so it covers the GPU case.
ocloc
-options
args need to be comma separated, e.g.-options -g,-cl-opt-disable
. Otherwise, only the first arg is processed by ocloc as an arg for-options
, and the rest are processed as standalone flags, possibly leading to errors. Another possibility would be-options "-g -cl-opt-disable"
, but quotes might cause escaping issues.