Redesign the code generation API

[lukaseder]

The code generation "API" has grown organically over the past releases of jOOQ 2.x and 3.x. Users desiring to override some behaviour rely heavily on its internals.

This may seem acceptable as we're talking about "only" code generation, but it is still hard to maintain user-defined behaviour in a stable way across jOOQ minor releases. The risk of breaking things between minor releases is substantial.

We should definitely re-write the entire API in a much more object-oriented and extensible fashion in jOOQ 4.0.

---

Feedback from the user group (https://groups.google.com/forum/#!topic/jooq-user/aZ0CJwUfdkM):

1. Target languages and forms

• Java and Scala (current support)
• Ceylon, Kotlin, Clojure
• Interesting thoughts by customer: JavaScript (and TypeScript, ScalaJS, Ceylon) for client-side records, in case database active records should be made available also on the client (with custom sync protocol)

2. Templating language

Alternatives are

• Java (as it is today)
• Xtend
• Scala
• "Pure" templating languages like mustache, velocity

3. Style

• Little interest in customisation
• Apply Oracle's JDK styleguide

4. Generator strategies

5. Disambiguation / compiler "optimisations"

6. Custom code

• Repositories / Services / DAOs, possibly with custom (C)DI annotations
• Custom predicate builders in DAOs
• Data type management (see e.g. forcedType's includeTypes doesn't work for various types on H2 #10231 (comment))

---

See also:

• https://groups.google.com/forum/#!topic/jooq-user/aZ0CJwUfdkM
• Overload JavaGenerator.generateXXX() methods to contain a JavaWriter argument #3882
• final and private in JavaGenerator #4384
• various other issues

[napperley]

Will Kotlin support be part of the redesign? If that is the case then it would be very helpful to have Kotlin code generated that is Kotlinic (idiomatic Kotlin/the Kotlin way), and follows the official Kotlin Coding Conventions.

FYI there is a library for generating Kotlin code called Kotlin Poet which is maintained by Square Inc.

[lukaseder]

Will Kotlin support be part of the redesign?

@napperley It says right there: "1. Target languages and forms / Java and Scala (current support) / Ceylon, Kotlin, Clojure"

FYI there is a library for generating Kotlin code called Kotlin Poet which is maintained by Square Inc.

We'll definitely not use an API based approach to generating code. If we change anything at all to the way code is generated, we might replace the home-grown templating language by a more versatile one, see "2. Templating language"

[napperley]

Could the templating language be in the form of a DSL? If that is the case then using a language that has built-in support for developing DSL's (eg Kotlin) would be an option to take into consideration.

[lukaseder]

I don't see the value of Kotlin's (or Groovy's for that matter) DSL-esque notation, nor how this is related to templating. The only reason why Kotlin might be a candidate are its multi-line string and string interpolation capabilities.

Anyway, thanks for sharing your enthusiasm for Kotlin. If this task is implemented, certainly, we'll keep in mind potential improvements in that area.

[davinkevin]

Very interested in improvements in the code generator like:

• Optional support for nullable field in Java
• Nullable (and not nullable) type in Kotlin

If I want to write my own generator, I've looked at JavaGenerator class, but do you have some other examples or community improvements ?

[lukaseder]

@davinkevin Thank you for your suggestions. While an improved code generator should definitely make it easier for usersto introduce Optional and @Nullable and similar things in generated code, do note that it is not a good idea to do so. The rationale is here:
https://blog.jooq.org/5-things-you-may-not-have-known-about-jooq

In SQL, every type is per se nullable. Some columns do have a NOT NULL constraint, but that constraint is typically only present in the persistent form of the table. As soon as you load the data from the table, the type become nullable again through various SQL language features, the simplest one being OUTER JOIN.

If I want to write my own generator, I've looked at JavaGenerator class

There are no examples because if there were, we'd have to start supporting JavaGenerator's internals :)

but do you have some other examples or community improvements ?

While some tweaks are definitely possible right now, they are tweaks and not well designed improvements (due to the nature of JavaGenerator), so I wouldn't expect too many community contributions here...

[lukaseder]

Despite the number of upvotes on this issue, it's unlikely this will be implemented soon, even in a major release:

• There is a lot of complexity in the code generation and its tests. Rewriting it will risk tons of regressions and will take forever.
• The benefits are very unimportant. The main benefit is code style, which is really not a high priority for generated code. We try to generate idiomatic code in all 3 currently supported langauges, but there are weird things that are due to having to generate code that extends internal API, so I'm not sure if any other non-templating based approach would even produce anything better
• Custom code is an occasional requirement, but then again, why use jOOQ-codegen for generating arbitrary code? Generating code is really very simple, and if someone has very specific requirements, they can write their own code generator very easily, using their own templates, or JavaPoet, or whatever.
• If we ever support new languages, adding them will not be too hard in the status quo, as ScalaGenerator and KotlinGenerator have shown. In fact, we might soon implement a Scala3Generator, and it will behave like a new language that won't be too hard to support. There's no guarantee that using a new approach will simplify things for other languages, au contraire. If we're not using templates, things might be much much harder for new languages, who don't have a generation API yet, e.g. comparable to JavaPoet.

Given the very very poor cost / benefit ratio, I'm going to reject this issue now.