Support for the native project (C++)

[Shad0w1nk]

Gradle now support native compilation (C++) and it would be great if this plugin could natively support this. The big problem is how the native project differ from the java project in Gradle. However, I'm confident that we can find a way to support both at the same time with minimal code duplication. I'm willing to help on this contribution but some guideline on the roadmap of this plugin would be required. The code is extremely Java project centric and some big change would be required in order to support both Java and C++. As for some background about me, I have been working with protobuf in a C++ environment compiled with Gradle. I had to hacked together a native plugin for protobuf with what was available at the time. I cannot contribute the code directly has it was really a one off solution but I do have idea on how we can make this possible for longer term.

Ideas? Thought? Complain?

[zhangkun83]

The current focus is supporting Android projects (#11). It is a major refactoring of the codebase. Please take a look and share any thoughts about we can move from that to C++ support.

Looking at the native component model, I imagine that we have one protoc run per component, and the generated files are added to the cpp sources of the same component. DSL may look like this:

model {
    components {
        hello(NativeLibrarySpec) {
            sources {
                cpp {
                    source {
                        srcDir "src/source"
                        include "**/*.cpp"
                    }
                    exportedHeaders {
                        srcDir "src/include"
                    }
                }
                proto {
                    source {
                        include "**/*.proto"
                    }
                }
            }
        }
}

[Shad0w1nk]

That sounds about right. There are some little things that C++ world may want.

For instance, the generated C++ files creates warning when the warning level is at the highest. One way we solve this was to disable the warning directly in the generated C++ files by appending/prepending #pragma warning(...). This could be solve in lots of different ways: fixing protoc, adding a feature to Gradle in order to specify per file compile flags, injecting a task before the compilation to edit the file or enabling the per-file post modification.

I saw some discussion about sharing proto files between project. In C++, Gradle is not at the point yet where it's supporting repositories like it does in Java. It will come but until then, there need to be a way to do this, at least locally.

I want to make sure it's easy to incorporate any future pull request so how do we go over these changes? The first step would be to add the proto source for each component and wire them in, then we will quickly need proto sharing between component and project. Is there an architecture or coding guideline that need to be followed (expect the Google coding style obviously)?

[zhangkun83]

Please take a look at CONTRIBUTING.md before contributing code. Most importantly, please make sure to sign the CLA before making any pull requests.

I want to make sure it's easy to incorporate any future pull request so how do we go over these changes? The first step would be to add the proto source for each component and wire them in, then we will quickly need proto sharing between component and project. Is there an architecture or coding guideline that need to be followed (expect the Google coding style obviously)?

You shouldn't have much trouble in the first step. With regarding to sharing proto between component and project, there are some thoughts for Java (#8) and I haven't started on it. We can have more discussions to figure out the best architecture that works for all languages.

As for code style, please follow the Groovy style guide, and follow the style of existing code. New features should come with tests. The C++ support will need a separate test project.

[Shad0w1nk]

Is it an hard requirement for the plugin to be coded in Groovy? Is Java also good? The way I understand the internal of Gradle, they usually use Java for the functionality and Groovy for configuration (if needed). The added advantage of Java is the syntax validation at compile time and some performance gain. The performance is simply theoretical as I never profile it but it do have a smaller execution stack as it shortcut the Groovy runtime.

Also, is there a hard requirement on the Gradle version to use? The native world is highly dependent on the model and it's getting lots of update at every release. What is your opinion on that?

[zhangkun83]

Using Groovy is not a hard requirement, but I feel if the plugin was coded in Java, it would have much more boilerplate code. It's true that Java has stronger typing and makes me feel more confident when it compiles, but the plugin doesn't have complex logic to require strong typing that bad. What the plugin does is still more like configuring and assembling stuff, and Groovy is really good at it.

I have not really thought about the Gradle version. Your opinion is welcome.

[Shad0w1nk]

I just submitted an initial support for Gradle's model used in the C++ support. More work is definitely needed to make it more sexy. However, the baseline is present and can be used in a C++ project. My next pull request will aim to more forward with merging more code between Java/Android and C++ as well as the possibility to easily integrate 3rd party plugin for C support. Any input is welcome as well as any roadmap of which things should be work on first is also welcome.

[jivk]

Hi, is this support in the plugin now?

[zhangkun83]

No, it's not yet done.

[jivk]

I like being able to build native using gradle. We need the facility to be more functionally rich though.

Right now we are having to do the following:

    def incs = 'pkg-config fix8 boost spdlog libpq --cflags'.execute()
        incs.waitFor()
        def incFlags = incs.in.text.trim()
        cppCompiler.args.addAll(incFlags.tokenize(' '))

        cppCompiler.args << "-I${projectDir}/src"

    def libs = 'pkg-config protobuf fix8 boost spdlog libpq --libs-only-L'.execute()
        libs.waitFor()
        def libsFlags = libs.in.text.trim()
        linker.args.addAll(libsFlags.tokenize(' '))

Is there a plugin or feature in the native plugin that can achieve this more nicely?

[jivk]

Would appreciate some thoughts/suggestions on this.

[enaess]

Gradle has come a long way to support cpp-application and cpp-library projects. The components {} model is AFAIK deprecated.

[lacasseio]

You are right about the software model plugins being phased out. To support the native project, I would suggest the Protobuf plugin build a smaller model that abstract the JVM SourceSet (most likely using ConfigurableFileCollection) as well as introduce a way to easily create variants of the GenerateProtoTask. In theory, the current integration with the other plugins would build upon this smaller model. All other integration (native as well as other types of projects) will be able to compose the Protobuf capability however they want. This would open the possibility for current users of the older software model plugins as well as the newer native plugins to easily build the integration. It also opens the path for integration with the Nokee plugins.

Looking at the source code, it seems that making the code compatible with the configuration cache would naturally lead to the smaller model internally. It would just be a matter of providing the proper conveniences and documentation of that model. Don't hesitate to ping me if you want to detail this work more.

[enaess]

I do have some experience with the cpp-application/library plugins. What you propose here sounds like a viable approach.