-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deserialization Schema error when change from azure-schemaregistry-kafka-avro 1.0.0.beta.4 to 1.1.0.beta with auto.register.schemas = true #43
Comments
To add more information, without auto.register.schemas does not work the deserializer, ItemNotFound. So I can image that it is a problem regarding the internal azure APIs. If I disable the auto.register.schema but maintain the schema in the portal, I can see this GET request to retrieve it.
But if I recreate the schema in the portal (manually), the same request trying to get the same schema fails. |
The problem really occurs when upgrade from beta.4 to beta.6 in the 1.0.0. There is some kind of breaking changes in the Azure API, it would be nice if there would be some kind of configuration to allow backward compatibility. |
Due to a migration of azure-schemaregistry-kafka-avro of 1.0.0.beta.4 to 1.1.0-beta.1 we have detected that KafkaAvroDeserializer fails due to a problem with the Schema deserialization. To sum up, the first revision of the library registers the schema (auto.register.schema=true) and retrieve it as String (The JSON is inside the String) so when the new version of the library tries to retrieve it, it get as String and fail to parse the Schema.
Steps to reproduce with auto.register.schemas = true
I have created a mock project located in my GitHub https://github.com/mgvinuesa/avro-eventhub-bug-steps with a test using the following versions:
Adding valid values for Schema (client, tenant...) in the test
AvroSerDesOldVersionTest
you are going to register the schema in the portal and deserialized it properly. Then the new schema is created and it is valid:Now, using the second project we are going to do the same thing, serialize and deserialize, in this case, the schema is not going to be created due to it already exists.
But in this case the schema deserialization fails. The error is:
The problem is that the received schema is a String, not a JSON, as you can see in the next debug screenshot:
You can see the scape characters to define the JSON inside the String. So the following line fails
Is there any release notes or capability to make this change backward compatible ? Due to currently all our schemas are created with this old library in the past, and now they are not compatible with the new one.
The text was updated successfully, but these errors were encountered: