Original schema not guaranteed to match generated schema from models

[bengels97]

Is your feature request related to a problem? Please describe.
Kafka ValueDeserializationError thrown when deserialising message using the generated schema from AvroModel.avro_schema(), due to a mismatch in between the original schema, retrieved from our schema registry, and the generated schema.

In short, the flow: original_schema -> generated_object -> generated_schema, will not always work, as compatibility between original_schema and generated_schema is not guaranteed.

See pictures in additional context.

Describe the solution you'd like
The easiest work-around would be to include the original schema inside the generated object. Should be relatively easy to implement, the schema is already passed to the ModelGenerator.render() method. This would prevent any mismatch to happen, as you are able to refer to the original schema in the resulting models.

A more generic approach: Make it possible to pass extra meta information to the ModelGenerator.render() method.

Describe alternatives you've considered

1. Implement a temporary fix in our own codebase by modifying the resulting model_code string. But this would be prone to break, as the order of the generated models might not be guaranteed.

Additional context
The thrown error:

Cause of the error:
Original schema:
"fields": [{"name": "accumulation", "type": {"type": "enum", "name": "Accumulation_analog", "symbols": [ "Average", "Maximum", "Minimum", "Summation", "None"]}}]
Generated schema from model:
"fields": [{"name": "accumulation", "type": {"type": "enum", "name": "AccumulationAnalog", "symbols": ["Average", "Maximum", "Minimum", "Summation", "None"]}}]

[bengels97]

I have prepared a local branch with a solution in place. Leveraging the ModelGenerator.metadata_field_templates attribute. I don't have the permissions to create remote branch here though.

Would like to be able to create a PR and get your thoughts on the implementation.

[marcosschroh]

HI @bengels97

Interesting case. Please send the PR and we can take a look. You can fork the repository and then send a PR from your fork

[bengels97]

Appreciate the quick response @marcosschroh.

You can find the PR here: #545

[marcosschroh]

Mitigated by #545 but still we need a solution for fields with inner names

[marcosschroh]

@bengels97 After merging #544 this issue will be fixed. Hopefully you won't need anymore to have the original schema in your model.