KAFKA-8713 KIP-581 Add accept.optional.null to solve optional null

[pan3793]

https://issues.apache.org/jira/browse/KAFKA-8713

https://cwiki.apache.org/confluence/display/KAFKA/KIP-581:+Value+of+optional+null+field+which+has+default+value

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[rgibaiev]

Any update? @rhauch

[rhauch]

This PR has a conflict that has to be resolved first.

Also, KIP-581 must be passed before this can be merged. See the discussion thread for the KIP.

[pan3793]

Fixed the mail thread, do you have any comments on the discussion and KIP doc? @rhauch

[coded9]

Any update or possible workaround ? @rhauch

[ahamedmulaffer]

Any update or possible workaround ? @rhauch

[camilesing]

Do you have any updates or possible workaround? @rhauch @ijuma @hachikuji @guozhangwang @mjsax
@rajinisivaram @cmccabe

[AChangFeng]

Any update or possible workaround ? @rhauch

[ZoeyLiu]

Do you have any updates or possible workaround? @rhauch @ijuma @hachikuji @guozhangwang @mjsax
@rajinisivaram @cmccabe

[ahamedmulaffer]

@ZoeyLiu possible workaround you have to do at your application level for now
For example for fields which have a default value to NOT NULL but it will accept NULL for those fields you have to query that field after you capture the cdc data

[Sergii-Bilokur]

Guys do you plan to merge this PR? We also faced the problem with default values for nullable fields.

[tombentley]

@pan3793 As far as I can see you never started a VOTE thread for your KIP. If this is something you're still interested in working on it would be good to get a vote started. If not someone else might be prepared to take it over.

[Stonepreheim]

We have run into the same issue and believe the underlying problem is with the struct get method. While searching for a potential workaround we fixed the issue by changing this line https://github.com/apache/kafka/blob/trunk/connect/api/src/main/java/org/apache/kafka/connect/data/Struct.java#L87
so that it does not replace a null value with the default. We are going to maintain our own version of the class until this is resolved. We believe a official solution should be drafted as soon as possible.

[ekeric13]

I think it would be great to get this in

[mimaison]

This has been done in #13419. Closing.