Variadic Macros Support for OpenCL

[davidrohr]

I am wondering why the OpenCL standard forbids variadic macros. They are actually allowed by a couple of OpenCL implementations I have been using, but they are disabled in clang (yielding "error: variadic macros not supported in OpenCL")
I do not understand the reasoning behind disabling variadic macros, in particular since OpenCL compilers usually are based on A C / C++ frontent, which usually has a preprocessor that would actually support variadic macros.

I agree variadic macros do not belong to a modern coding style in the direction of C++20, but I have some code where they are widely used, in particular when you try to have common code segments for different GPU APIs, where you do not have access to the full C++ feature set.

Prime examples are

• Wrappers around printf with some additional checks / functionality (I agree this could also be done with variadic templates nowadays)
• Conditional disabling of parts of the code via defining #define DEBUG(...) to nothing, which is much cleaner than #ifdes everywhere in the code.
• At several places we create some code fragments automatically using macro string concattenation (##) and variadic macros.

My suggestion is to simply allow variadic macros in the code, since they do not increase compiler complexity anyway. At least, there could be an OpenCL extension or in case of clang a compile-time switch to allow variadic macros. My current workaround is to run the .cl file through the -std=c99 frontend of clang for the preprocessor phase and then I am giving the preprocessed file to clang again with -std=c++

[bashbaug]

+1, I've never understood why this restriction exists, either.

At the very least it would be nice to support variadic macros via an offline compiler, say when compiling to SPIR-V.

I suspect supporting variadic macros via an online compiler would require an extension, just to formalize that some implementations support this functionality, while others don't.

[AnastasiaStulova]

I don't think compiler actually knows if it's offline or online compilation. Also I wouldn't like the code rejected differently depending on that. I would just allow it unconditionally.

[bashbaug]

compiler actually knows if it's offline or online compilation.

I think this depends what "the compiler" is. If it's a newer version of open source CLANG, I can see how it might not know if it is being used for offline or online compilation. However, we have been shipping snapshots of CLANG with our drivers for years, which only get used for online compilation (meaning: for programs created using clCreateProgramWithSource) and report errors for kernels that use variadic macros:

1:1:15: error: variadic macros not supported in OpenCL
#define MAGIC(...)  5
              ^

The extension would be a way to differentiate between online compilers that may report errors for variadic macros and online compilers where variadic macros are supported.

[davidrohr]

I don't really see why it should be more difficult for an online compiler, therefore I'd propose to just allow it unconditionally. If they are concerns that some compilers cannot do it, then I'd go for the extension.

[AnastasiaStulova]

That's really great that Intel compiler has such a feature! I suspect some other vendors might have it too. However, we probably won't want to force every vendor to implement the same? From the compiler design side I would also minimise divergence between online and offline mode to reduce the complexity. But what I am concerned most about that for the OpenCL application developers we will provide a language with inconsistent behaviour. It is not uncommon to develop kernel code using offline compiler (where you can see error reported easly) and then once it is ready for execution it's compiled with the online compilation flow. I think it would really be a pain if you need to change your code every time between phases. There are plenty of other use cases with mixing online and offline parsing flow.

Unless we see a real value in doing this conditionally I would recommend to change it coherently for both. As a matter of fact do we even have separate language spec for offline and online mode? I also don't see it as any extension either but rather general update to the spec. It brings no compatibility issue for old code because we are lifting the restriction and not adding it.

[bashbaug]

Sorry about that, I don't think I'm explaining very well. Let me try again.

• We've been shipping drivers that implement the letter of the spec, meaning that programs created from source that include variadic macros will fail to compile. We can't change this - they've already shipped.

• It sounds like we want to relax this restriction and allow variadic macros. Cool, I agree!

• The issue is that if we relax the restriction and allow variadic macros, a programmer doesn't have a way to know if they are running on an "older" driver where variadic macros were illegal, or on a "newer" driver where variadic macros are legal. We need to have some way to differentiate between the two or new code may fail unexpectedly on older drivers. I'm suggesting that an extension is one way to represent this, but it's not the only way.

• Note that there is no such issue with offline compilation, because the variadic macro gets resolved when the IL or program binary or whatever gets created offline, so the driver never sees it.

Does this help?

[AnastasiaStulova]

Thanks for clarification! I see what you mean now. However, I am just trying to understand why is this case different from the other similar bugs in the spec we are fixing. We don't create an extension for each of them typically and I don't think this is even helpful.

As for the extensions I would vote against adding new ones in general as much as possible but especially for this case. I generally would like to minimize use of them because they are just extra complications for the application and tools developers.

[AnastasiaStulova]

Since there is no objection to relax this in general the plan is to relax this restriction as a Clang extension and then potentially adopt in the language spec in some way.

[AnastasiaStulova]

Clang change in review: https://reviews.llvm.org/D59492

[AnastasiaStulova]

Committed in r356987. This will be supported from clang9 onwards.

I will keep this open until we decide what to do about the spec changes.

[AnastasiaStulova]

Need conclusion on #65 to proceed.