Generating bindings for C library/ABI

[oysteinkrog]

What is the story if I want to generate bindings for e.g. ffmpeg?
Having a "C" mode seems like a very useful thing to me, is this something that is a goal/desired? (I'm really tired of creating p/invoke stuff by hand ...)

I have not found a way to generate nice C bindings using the existing options, but I've been able to get calling convention etc. correct by modifying Parser.cpp to pass -std=gnu99.
I guess there might be some work needed other places as well to get things working, any ideas/suggestions?

[tritao]

Yes, the project actually started as a C binding tool before C++ support was added.

I don't think we have any tests for C support right now, and as you probably noticed, some generated code might not be too nice, what I think we should do is:

• Add a way to pass arbitrary Clang command line arguments from Options (so you can easily pass -std=gnu99 without modifying the native code)
• Add some tests for explicit C support
• Improve whatever patterns are used in C code to get some nice bindings (either by modifying the generators or adding new passes)

Can you give some examples what is currently generated vs. what you would expect to be generated?

[oysteinkrog]

As you say the code generator does not generate pinvoke that is very nice/correct for C bindings.
Here is an example of a C header and what is generated now, VS hand-made nice/correct bindings:

C Header : http://pastebin.com/zybEFyrr
Current C++ Generation: http://pastebin.com/96zQqewa
Handmade bindings : http://pastebin.com/TpKv5t1x

The handmade bindings are from an earlier version of the C header, but from what I can tell the big problems with the current autogenerated bindings are:

1. C #defines are not included at all (in my handmade bindings I made them public const int)
2. The class + internal struct pattern is not necessary for C structs. I think they can be mapped directly to c# structs, also should not generate ctor/dtor etc.
3. The entry points for the dllimport mappings are wrong because clang parses using c++ ABI instead of C ABI (this was fixed by passing std=gnu99 etc and adding support for Record parsing in the AST, instead of just CXXRecord).
4. Missing support for passing strings directly to pinvoke/dllimport function (by specifying CharSet in the DllImport attribute).

[oysteinkrog]

Also, a question: why is there two copies of the Parser/AST system?
What versions should I work on?
It's a bit confusing...

[tritao]

CppSharp.Parser was the first parser, and it relies on C++/CLI for the interop between Clang and the managed AST structures.

CppSharp.CppParser specifies a pure C++ API which is then binded by CppSharp itself (Parser.Gen) to get cross-platform C# or C++/CLI bindings.

Right now we still have both parsers, but the plan is to switch to CppParser as soon as everything is working properly with it.

[oysteinkrog]

Ok, that makes sense. Is there a list of TODO's on the new parser?

Regarding the "C" project, I'm considering just writing a new code generator for it, since reusing and modifying the existing generator is really getting a bit messy.
It seems to me that the existing c++ binding code generator is massively overkill for generating C bindings, I'm powering through with a "C" mode to get up to spec on the code and get something working, and as part of that I might write a new "simple"/C code generator.
It's possible to do C bindings with really just plain structs and dllimport's (and the correct attributes), and that is IMO the best way.
By avoiding explicit sizes/offsets in structs the generated bindings can even be portable, which seems to be like a big win, what do you feel about that? (e.g. no FieldOffsets on fields, instead relying on generating correct number/placement of fields in the struct).

[tritao]

Well as of yesterday the full test suite is now finally passing with the CppParser. There's a specific VS build configuration in GenerateProjects.bat (option 3 IIRC), you can give it a try.

What's missing is completing the support for documentation comments parsing and parsing of all template specializations (I added some stubs for this a day or two ago). While this is added it'd be good to have some more tests for these things.

What you suggest is in fact how the current generator started, but it's been adapted a lot to work for C++ constructs. It seems to me that a lot of the code for a new "C" generator would be duplicated, and we'd need to duplicate it once again if we were to keep supporting C# and C++/CLI as a binding technology for C code.

I'd rather adapt the current generators to support C, if it gets too messy in the same file we can make the generator class partial and add a new file with C-related generation functionality. I'd be OK in duplicating specific methods (GenerateClass, GenerateFunction, etc) for C-specific stuff if this makes the code any less messier.

Does this sound reasonable?

[oysteinkrog]

Ah, good to hear that the new code is so close to finished.
Regarding the plan for a C generator; your proposal does sound reasonable.
One of the big reasons I've been tempted to create a separate generator is the fear that I will break some of the c++ generation. However; having a test suite should certainly help prevent that, I'll see about getting it up and running locally.

For now I'm still working on just getting the desired output (and understanding everything), once I have that I'll work on making it integrate cleanly with the rest of the system and make sure it does not break the c++ output.

If I do find that a separate generator is better, I will reuse as much as I possibly can, and it would probably be reasonable to refactor much of existing generator so the code could be shared.

[tritao]

Hi,

Any news regarding your C generator?

Just curious if you've made any progress or still planning to work on it.

[oysteinkrog]

Hi @tritao
I stopped working on it a while ago, as it was good enough for my intended use.
It works quite well now and is able to generate production-quality bindings for several C libraries we use (among them FFmpeg).
I might take a look at trying to rebase my work.
One of the big problems I ran into was that I had to comment out or add a lot of if-logic in the code generator, which made it very convoluted and messy and also hard to rebase:P