Skip to content

Comments

fix!: Stricter schema parsing#479

Open
Kriskras99 wants to merge 2 commits intomainfrom
fix/stricter_schema_parsing
Open

fix!: Stricter schema parsing#479
Kriskras99 wants to merge 2 commits intomainfrom
fix/stricter_schema_parsing

Conversation

@Kriskras99
Copy link
Contributor

This will break code, as it no longer accepts invalid schemas.

Closes #476

Copy link
Member

@martin-g martin-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are some comments on the proposed changes in this PR.

But as I commented at #476 (comment) the Java and Python SDKs do support parsing such flat schemas too.

assert_eq!(expected, err);
assert_eq!(
Schema::parse_str(schema_str).unwrap_err().to_string(),
"`default`'s value type of field `f1` in `ns.record1` must be a `{\"name\":\"ns.record2\",\"type\":\"record\",\"fields\":[{\"name\":\"f1_1\",\"type\":\"int\"}]}`. Got: String(\"invalid\")"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Let's use raw string to make a bit more readable. r#"..."#
Same for the other similar assertions.

let result = Schema::parse_str(schema_str).unwrap_err();
assert_eq!(
result.to_string(),
"Invalid schema: There is no type called 'enum', if you meant to define a non-primitive schema, it should be defined inside `type` attribute. Please review the specification"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Invalid schema: There is no type called 'enum', if you meant to define a non-primitive schema, it should be defined inside `type` attribute. Please review the specification"
"Invalid schema: There is no type called 'enum', if you meant to define a non-primitive schema, it should be defined inside `type` attribute. Please consult with the specification"

#[error("No `type` in complex type")]
GetComplexTypeField,

#[error("No `type` in in record field")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[error("No `type` in in record field")]
#[error("No `type` in record field")]


if let Some(logical_type) = field.get("logicalType") {
warn!(
"Ignored the {enclosing_record}.logicalType property (`{logical_type}`). It should probably be nested instide the `type` for the field"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Ignored the {enclosing_record}.logicalType property (`{logical_type}`). It should probably be nested instide the `type` for the field"
"Ignored the {enclosing_record}.logicalType property (`{logical_type}`). It should probably be nested inside the `type` for the field"

for (key, value) in field {
match key.as_str() {
"type" | "name" | "doc" | "default" | "order" | "position" | "aliases"
| "logicalType" => continue,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs reworking too.
logicalType, symbols, size, items and values cannot be at that level anymore.

field_schema.canonical_form(),
field_schema
.independent_canonical_form(&schemata)
.unwrap_or("Unknown".to_string()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about falling back to the canonical form instead of this "Unknown" ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

type.enum is valid while it should fail, good is type.type.enum according to spec

2 participants