Skip to content

Conversation

@martin-g
Copy link
Member

@martin-g martin-g commented Sep 17, 2021

Jira

Tests

  • The relevant unit tests were updated. Now parsing a schema with mismatching type and logicalType will just log a warning and use the type. Just as Avro Java does.

Commits

  • My commits all reference Jira issues in their subject lines.

Documentation

  • No API changes! Only implementation details have been changed to match the expected behavior, as in the spec and Avro Java

@github-actions github-actions bot added the Rust label Sep 17, 2021
@martin-g martin-g marked this pull request as ready for review September 21, 2021 13:42
@martin-g
Copy link
Member Author

The PR is ready for review!

use avro_rs::{schema::Name, Error, Schema};
use lazy_static::lazy_static;

fn init() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

init is quite general. It might be a bit clearer if this is named init_logging or something similar to that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that it might be used for other needs later.
Too bad Rust Testing does not provide something like before()/after(), beforeAll()/afterAll().
Using ctor crate looks OK-ish (https://stackoverflow.com/questions/58006033/how-to-run-setup-code-before-any-tests-run-in-rust) but it is one more dev-dependency and it helps only for the before() need.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ye I agree, or even something like pytests fixtures.
I see your point with possibly adding more to the init, however I often favour separating the inits unless they are necessary inits for all of the tests. In this case it is more of a nitpick comment, so I don't mind you disregarding it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was also wondering about the name when introducing it, so I am almost convinced to rename it! :-)
Thanks for the review!

@martin-g
Copy link
Member Author

@iemejia Could you please review this PR ? Thank you!

Read the complex type recursively. It seems Avro Java may produce {"type": {"type": "string", "avro.java.string": "String"}, "logicalType": "timestamp-millis"}}, i.e. logicalType is on the same level as the outer "type"
@martin-g martin-g force-pushed the avro-3197-fallback-to-string-when-logical-type-does-not-match branch from 6051dc8 to 1b80abd Compare January 5, 2022 08:57
@martin-g martin-g merged commit bce3866 into apache:master Jan 5, 2022
@martin-g martin-g deleted the avro-3197-fallback-to-string-when-logical-type-does-not-match branch January 5, 2022 11:15
martin-g added a commit that referenced this pull request Feb 1, 2022
…rt the type (#1340)

* AVRO-3197 Fallback to the 'type' when the logical type does not support the type

* AVRO-3197 Better formatting

* AVRO-3197 Allow only when the "type" is "string"

* AVRO-3197 Handle problematic complex type for date/time logical types

Read the complex type recursively. It seems Avro Java may produce {"type": {"type": "string", "avro.java.string": "String"}, "logicalType": "timestamp-millis"}}, i.e. logicalType is on the same level as the outer "type"

* AVRO-3197 Make Clippy happy

* AVRO-3197 Log a warning when a logical type is not supported by the Schema type

This is how Avro Java behaves.

(cherry picked from commit bce3866)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants