Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Records do not support logical types #61

Closed
davydog187 opened this issue Feb 23, 2022 · 2 comments · Fixed by #62
Closed

Records do not support logical types #61

davydog187 opened this issue Feb 23, 2022 · 2 comments · Fixed by #62
Labels

Comments

@davydog187
Copy link
Member

davydog187 commented Feb 23, 2022

    test "works with logicalType field values" do
      schema = AvroEx.parse_schema!(~S({"type": "record", "name": "Record", "fields": [
          {"type": "long", "name": "timestamp", "logicalType": "timestamp-millis"}
        ]}))

      timestamp = ~U[2022-02-23 20:28:13.498428Z]

      assert {:ok, _} = @test_module.encode(schema, %{timestamp: timestamp})
    end

Adding a test like this results in

code:  assert {:ok, _} = @test_module.encode(schema, %{timestamp: timestamp})
left:  {:ok, _}
right: {:error, %AvroEx.EncodeError{message: "Schema Mismatch: Expected value of long, got ~U[2022-02-23 20:28:13.498428Z]"}}
@davydog187 davydog187 added the bug label Feb 23, 2022
@davydog187
Copy link
Member Author

The reason is due to Record.Field failing to take into account logical types when parsing the schema

@davydog187
Copy link
Member Author

davydog187 commented Feb 24, 2022

Turns out they do support them, I was just doing it wrong. If you want a logical type, you need to expand the type.

I should add a strict mode to the parser that raises for unrecognized fields

{
  "name": "my_record"
  "type": "record",
  "fields": [
    {"name": "timestamp", "type": {
      "type": "long", "logicalType": "timestamp-millis"}
    }
  ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant