Integration of Go parts into build process

[Fabian-K]

Hi,

I´m currently looking into the the generator of knative with reference to #2117. When changing the go code, this is currently separate from the build process

What are your thoughts about the following:

• running mvn install on the project should also include the generators
• the compiled go program should not be committed to VCS (extensions/knative/generator/generate)
• the output of the generator (e.g. extensions/knative/model/src/main/resources/schema/knative-schema.json) should not be committed to VCS
• using a different build system (gradle) can improve the setup (e.g. incremental compilation) <- I know this is a big change also wrt the release process etc

Looking forward to your opinions :)

[rohanKanojia]

@oscerd @manusa @iocanel : thoughts??

[oscerd]

I personally don't like gradle and I don't think it will add something. I think it's possible to do incremental builds with maven without problem, there are different plugin for doing it (more or less mature). Incremental builds are a problem in really big project with a complex dependency graph like Apache Camel.

[manusa]

Gradle replacement

I do like Gradle and have been using it for a long time.

Having said this, for a multi-module project such as this one with so much autogenerated code, I think it will be very challenging to create a build script that performs what we are doing right now (generating code in several phases, support for multiple bundling systems, etc.).

IMHO, this means a lot of work with very little benefits. Gradle's incremental compilation would also be very difficult to configure reliably as it will depend on different resource changes for each of the modules (build script complexity, especially for the model project [go sources/dependencies change>go compiles> go generates schema>schema is parsed and annotated jsonschema2pojo>sundr.io processes schema>project is compiled).

mvn install should also include generators

The main drawback for this is that tfor any developer to work on any part of the project he/she would need to have go in the environment (and the project checked out in the appropriate file system location). So this will probably prevent some contributions from the community.

This would also mean that the currently very long compilation time would be increased.

Personal view

I don't enjoy very much working on the project because it requires a massive amount of computing resources and compilation time to reliably check each change.

I really like the way the project autogenerates every Java class from a go dependency in a dsl fluent way. This probably had no drawbacks when the Kubernetes schema was smaller. But I think this hasn't aged very well as now the schema is very large and we also include other models (Tekton, Knative, etc.).

Personally, I would rather go the other way around, persisting generated Java classes in the VCS. Following the incremental build feature suggestion, model generation should only happen when go dependencies/sources change, so why not make the whole process manual.

In contrast, for the suggested incremental compilation (Gradle/Maven), even if it works well, project should be compiled completely at least once. Which means not much added value.

So what I'm suggesting is that if any of the generator code changes (go dependencies/sources), a manual "code generation task" should be invoked too, and only the output of the task should be persisted (none of the intermediate generated files > go binaries, schemas, Java sundrio annotated pojos, etc.).

This is already similar to what we do now, as any go change requires generating the json schemas. But now, instead of persisting the generated Java classes, we are persisting the schema, which means that we are all running the same code generation which always produces the same output multiple times.

[stale]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!