[FLINK-3226] Translation from and to POJOs for CodeGenerator

[twalthr]

This PR implements full POJO support as input and output type of the Table API. It is now possible to convert from an arbitrary type to an other arbitrary type. I fixed the failling AsITCase and implemented additional tests from/to tuples, from/to POJOs and from/to Case classes.

@vasia and @fhueske feel free to review.

[fhueske]

Hi, I thought about using POJOs as native types within Table/SQL operators. IMO, the gains are very little compared to the added code complexity. Given a POJO input, we can preserve the input type only for very few operations such as a Filter. For most other operations, we need to generate a new output type (Tuple or Row). I am a bit skeptical about adding a lot of codeGen code with special cases for POJOs (such as the field index mapping) which is very seldom used. Moreover, POJO field accesses (for operations and serialization) go through reflection and are not very efficient. So even the performance gain for those few cases where POJOs can be used is not clear.

I do not question the native type support in general. Tuples and primitives should definitely be supported, but I don't think we need to support POJOs within Table / SQL operators. Instead, I would convert POJO datasets into Row tables during table scan. Most of the code in this PR can be used to implement a codeGen'd converter Map function.

What do you think @twalthr?

[twalthr]

Hi @fhueske,

I completely share your opinion. This PR does exactly what you described. As you can read in DataSetSource: "It ensures that types without deterministic field order (e.g. POJOs) are not part of the plan translation." What I have implemented is proper reading of POJOs (into Table API) and writing of POJOs (out of Table API). Most of the code I have added is to provide codeGen'd converter Map functions for the source and the sink. POJOs are not used for any operatation within operations.

[fhueske]

Ah, OK :-)
I have to admit that I didn't look at the changes in detail but noticed that they go into several places of the code gen. Hence, I was wondering if it would be possible to extract the special POJO case from the CodeGenerator, but I guess that would lead to code duplication.

I'll start reviewing the changes.

[fhueske]

I would make name equivalence only required for POJOs and generic composite types types.
Rows and tuples can be matched by position. Otherwise, fields would need to be renamed to f0, f1, etc. for tuples.

[fhueske]

Ah, I see the comment is copied. But should be changed, no?

[fhueske]

Thanks @twalthr, the PR looks pretty good.

[twalthr]

Thanks for reviewing @fhueske!
I updated my PR and rebased it.
Feel free to merge ;-)

[vasia]

Thanks @twalthr. I'll merge this.

[vasia]

You can close the PR @twalthr. It's merged (but doesn't automatically close).