Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVRO-3772: [Rust] Deserialize Errors for an Unknown Enum Symbol instead of Returning Default #2280

Merged
merged 2 commits into from
Jun 13, 2023

Conversation

martin-g
Copy link
Member

AVRO-3772

Take into account the field's default value when resolving an enum in a reader schema

What is the purpose of the change

  • Fix the resolution of an enum with a symbol which is no more available in the reader schema. Use any default value if available before returning an error

See https://avro.apache.org/docs/1.11.1/specification/#schema-resolution

Verifying this change

  • An old test has been updated to the new logic

Documentation

  • Does this pull request introduce a new feature? no

…ad of Returning Default

Take into account the field's default value when resolving an enum in a
reader schema

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
@github-actions github-actions bot added the Rust label Jun 12, 2023
Do not take into account the number of the symbols in the reader schema.
There is no such requirement for this in the specification.
Just try to find the written symbol in the list of symbols in the reader
schema.

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
@martin-g
Copy link
Member Author

martin-g commented Jun 12, 2023

I've pushed a second commit about (I'm also curious if there would be another issue if the record contained a symbol that exists in the writer enum and had an index greater than the len of the reader's symbols since it appears to be doing a check for that on line 859).
I don't see a requirement in the specification about the number of the symbols, so I removed that.
The logic about the number of the symbols is valid only for the decode functionality.

@martin-g martin-g merged commit 88985fa into master Jun 13, 2023
14 checks passed
martin-g added a commit that referenced this pull request Jun 13, 2023
…ad of Returning Default (#2280)

* AVRO-3772: [Rust] Deserialize Errors for an Unknown Enum Symbol instead of Returning Default

Take into account the field's default value when resolving an enum in a
reader schema

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>

* AVRO-3772: [Rust] Ignore the length of the enum symbols

Do not take into account the number of the symbols in the reader schema.
There is no such requirement for this in the specification.
Just try to find the written symbol in the list of symbols in the reader
schema.

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>

---------

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
(cherry picked from commit 88985fa)
@martin-g martin-g deleted the avro-3772-enum-compatibility branch June 13, 2023 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
1 participant