Set explicit default null values for nullable fields in generated schemas

[gafiatulin]

Some schema consumers require explicitly set "default": null for the "nullable union field".

Related: AVRO-1803

[grouzen]

Need this PR merged, please! :)

[bplommer]

Sorry for the slow response - I'll try to take a look this evening.

[bplommer]

I agree this behaviour should be available but I also think it should be optional - so keep the existing derive with its current behaviour but add something like deriveWithDefaultNulls. I'll have a go at adding that to the PR when I have a chance, unless you beat me to it 🙂

[gafiatulin]

@bplommer thanks for looking into this.
I can change this to be annotation-enabled, something like @AvroDefaultNulls. I'll push something a bit later today.

[gafiatulin]

@bplommer please see 7df614d.

[gafiatulin]

I guess it's also possible to use magnolia's param default value extractor and have more generic annotation:

case class ABC(
  @AvroDefaultValue a: Int = 5,
  b: Option[String],
  @AvroDefaultValue c: Option[Long] = None 
)

=>

{
  "type" : "record",
  "name" : "ABC",
  "fields" : [ {
    "name" : "a",
    "type" : "int",
    "default" : 5
  }, {
    "name" : "b",
    "type" : [ "null", "string" ]
  }, {
    "name" : "c",
    "type" : [ "null", "long" ],
    "default" : null
  } ]
}

With implementation like:

...
param.annotations.collectFirst {
  case adv: AvroDefaultValue =>
    param.default match {
      case Some(defaultValue) =>
        param.typeclass.encode(defaultValue) match {
          case Left(err) => throw err.throwable
          case Right(value) =>
            // TODO: Handle other unions except for [null, ???]  
            if (schema.isNullable && value == null)
              Schema.Field.NULL_DEFAULT_VALUE
            else if (schema.isNullable)
              throw new Exception(
                s"For optional fields only supported default value is None, field ${param.label} has default value $defaultValue."
              )
            else value
        }
      case None =>
        throw new Exception(
          s"Missing default value for a field ${param.label} annotated with $adv."
        )
    }
}.orNull
...

Which kinda works, but because in avro default values for union fields have to correspond to the first schema in the union would have to either:

1. Restrict possible default values (like in the code above for non-None optional defaults).
2. Change the order elements are listed in the union schema based on provided default value which is also not ideal.

[bplommer]

This looks great, thanks!

Which kinda works, but because in avro default values for union fields have to correspond to the first schema in the union would have to either:

Restrict possible default values (like in the code above for non-None optional defaults).
Change the order elements are listed in the union schema based on provided default value which is also not ideal.

I think it's best to leave this as is to avoid too much complexity.

@vlovgr any thoughts on this before I merge?

[vlovgr]

Agreed, let's keep the derivation module as simple as possible.