Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

converts :null to nil #47

Closed
wants to merge 3 commits into from
Closed

converts :null to nil #47

wants to merge 3 commits into from

Conversation

matreyes
Copy link

@matreyes matreyes commented Oct 8, 2020

Hi!, Great work with Avrora!

Erlavro recommends to use hooks to convert the :null value (erlang) to nil for elixir.

I've added this to the default hook for Plain codec because it should be the default behaviour, but I couldn't find how to implement the hook on OCF.

It breaks the APIs, so if you approve it, it should go as major version.

Best

@Strech
Copy link
Owner

Strech commented Oct 8, 2020

Hey @matreyes nice catch, thanks for your support. I think you are right, I will have to put a major version on release.

I would like to ask you to write a few tests to cover the change. Meanwhile, I will try to help you with OCF, if I could find something.

UPD: What I have found so far:

  1. https://github.com/klarna/erlavro/blob/master/src/avro_binary_decoder.erl#L79-L80 this is a decoder function signature with decoder hook
  2. https://github.com/klarna/erlavro/blob/master/src/avro_ocf.erl#L210 this is OCF decode function which is using N#1, but with another form

So ... I guess that for OCF we would need to find a way to pass the decoder hook ... maybe I would like to open PR to erlavro repo ✌🏼

@matreyes
Copy link
Author

Hi, i've added tests for this issue. Do you think it's OK ?
Best.

Copy link
Owner

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think about an optional flag to enable/disable this behavior, but not sure about it, WDYT?

defp payment_schema do
{:ok, schema} = Schema.parse(payment_json_schema())
%{schema | id: nil, version: nil}
end

defp payment_json_schema do
~s({"namespace":"io.confluent","name":"Payment","type":"record","fields":[{"name":"id","type":"string"},{"name":"amount","type":"double"}]})
~s({"namespace":"io.confluent","name":"Payment","type":"record","fields":[{"name":"id","type":"string"},{"name":"amount","type":["null","double"]}]})
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you already introduce null_payment_payload, null_payment_message, I would suggest going with another schema (most of the things already different). It will also allow us to avoid confusion from the logical side (nullable amount).

@@ -154,17 +165,26 @@ defmodule Avrora.Codec.PlainTest do

defp payment_payload, do: %{"id" => "00000000-0000-0000-0000-000000000000", "amount" => 15.99}

defp null_payment_payload,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's introduce another schema with a more logical null in it. Also, may I suggest to make null as a postfix, like ..._payload_with_null

@@ -106,6 +112,11 @@ defmodule Avrora.Codec.PlainTest do
assert encoded == payment_message()
end

test "when payload with null value is matching the schema and schema is usable" do
{:ok, encoded} = Codec.Plain.encode(null_payment_payload(), schema: payment_schema())
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent, can you make 1 blank line right before the assertion (like you did in the case above)

@Strech
Copy link
Owner

Strech commented Nov 12, 2020

@matreyes Hey, how are you doing? Finally, my PR was merged and now in OCF you have access to the same set of decoder options as for binary decoder klarna/erlavro@a716c41

Also, I think for now I would like to keep the major version 0, so it's better to have this behavior optional as on by default, but with an ability to be turned off

@Strech
Copy link
Owner

Strech commented Nov 16, 2020

@matreyes If you don't mind I will continue this PR from now on. Thanks for your contribution 🤝

@Strech Strech added the enhancement Improvement of existing functionality or request of improvement label Nov 16, 2020
@matreyes
Copy link
Author

matreyes commented Nov 19, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improvement of existing functionality or request of improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants