Support for Union-Types (Serde, Schemas, Avro-Values) #76

RGafiyatullin · 2019-03-14T14:37:20Z

Difference between the Value::Union representation this PR and @kphelps 's one: Value::Union(variant_index: usize, variant_value: Box<Value>) instead of Value::Union(union_ref: UnionRef, variant_value: Box<Value>).

This option does not allow for creation of union-type instance from the inner variant-value "implicitly" (i.e. without knowing the variant-index able to carry that exact variant-data).
This should help a lot with Serde because this restriction allows "lossless" translations between Structs<->Avro-Values<->Avro-Binary in both directions.

Example of usage: https://gist.github.com/RGafiyatullin/4fec518e3cd8e3c6fe718ca976a4cdc5
Output: https://gist.github.com/RGafiyatullin/8aaf4c427bec5456edde2e0913dde472

…l to be done

flavray

Looks very good overall, just a comment :)

flavray · 2019-03-24T09:55:39Z

src/types.rs

-            .find_schema(&v)
-            .ok_or_else(|| SchemaResolutionError::new("Could not find matching type in union"))?;
-        v.resolve(inner)
+        match self {


It looks like this code is missing the resolution when the writer value is not a union but reader is a union.

e.g: writer writes a Long, reader is Union(Null, Long), then resolve_union should output a Union(1, Value::Long)

See the "if reader's is a union, but writer's is not" section in https://avro.apache.org/docs/1.8.2/spec.html#Schema+Resolution

The Standard relies on availability of both writer- and reader-schema for resolution.
So I believe the signature of the method should go as pub(crate) fn resolve(reader_schema, writer_schema) -> Result<Value, Error>.
The only usage of the method is pub fn from_avro_datum<R: Read>. I think it only would be right to remove the method value.resolve from the public API.

…o create a union-schema other than parsing a string

nlopes · 2019-09-02T11:15:48Z

Any movement on this? I have an incoming PR for Schema Compatibility that would gain a bit by having this PR in.

CtrlC-Root · 2019-10-04T04:23:38Z

I would also benefit from having this work merged. Is there anything I can do to help?

poros · 2020-01-14T22:39:19Z

There is also #60 implementing bits and pieces of the same feature in a slightly different way and both PRs are pretty old... (definitely because of the fault of us maintainers).

I'll need to budget some time to properly understand how serde handles enums (tonight I only managed to read the serialization half of the data format) and at the same time follow up with the authors to understand who is still interested in contributing.

@RGafiyatullin are you still interested in contributing? Shall we try to revitalize this PR? I asked the same thing to @jdeschenes on #60.

poros · 2020-06-07T12:13:51Z

I believe that #110 landed the same feature of this PR only in a slightly different form. Closing for now, but please feel free to open a new one based on current master in case there is still something missing that you'd like to see added.

Documenting avro-rs v0.14.0 (/home/martin/git/apache/avro/lang/rust) warning: this URL is not a hyperlink --> src/lib.rs:696:5 | 696 | //! flavray/avro-rs#76). | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/flavray/avro-rs/pull/76>` | = note: `#[warn(rustdoc::bare_urls)]` on by default = note: bare URLs are not automatically turned into clickable links

* AVRO-3205 Update Cargo.toml [package] information * AVRO-3205 Fix 'cargo doc` warning Documenting avro-rs v0.14.0 (/home/martin/git/apache/avro/lang/rust) warning: this URL is not a hyperlink --> src/lib.rs:696:5 | 696 | //! flavray/avro-rs#76). | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/flavray/avro-rs/pull/76>` | = note: `#[warn(rustdoc::bare_urls)]` on by default = note: bare URLs are not automatically turned into clickable links * AVRO-3205 Add keywords and documentation to [package] * AVRO-3205 Add categories = ["encoding"] * AVRO-3205 Update markdown files Replace flavray/avro-rs with apache/avro. Replace MIT license with Apache License 2

* AVRO-3205 Update Cargo.toml [package] information * AVRO-3205 Fix 'cargo doc` warning Documenting avro-rs v0.14.0 (/home/martin/git/apache/avro/lang/rust) warning: this URL is not a hyperlink --> src/lib.rs:696:5 | 696 | //! flavray/avro-rs#76). | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/flavray/avro-rs/pull/76>` | = note: `#[warn(rustdoc::bare_urls)]` on by default = note: bare URLs are not automatically turned into clickable links * AVRO-3205 Add keywords and documentation to [package] * AVRO-3205 Add categories = ["encoding"] * AVRO-3205 Update markdown files Replace flavray/avro-rs with apache/avro. Replace MIT license with Apache License 2 (cherry picked from commit ea07ac0)

Remove a link to an issue/PR at flavray/avro-rs/pull/76 because it is already resolved. Enable (uncomment) the tests related to this issue. Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>

Remove a link to an issue/PR at flavray/avro-rs/pull/76 because it is already resolved. Enable (uncomment) the tests related to this issue. Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org> (cherry picked from commit 69fda12)

* AVRO-3205 Update Cargo.toml [package] information * AVRO-3205 Fix 'cargo doc` warning Documenting avro-rs v0.14.0 (/home/martin/git/apache/avro/lang/rust) warning: this URL is not a hyperlink --> src/lib.rs:696:5 | 696 | //! flavray/avro-rs#76). | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: use an automatic link instead: `<https://github.com/flavray/avro-rs/pull/76>` | = note: `#[warn(rustdoc::bare_urls)]` on by default = note: bare URLs are not automatically turned into clickable links * AVRO-3205 Add keywords and documentation to [package] * AVRO-3205 Add categories = ["encoding"] * AVRO-3205 Update markdown files Replace flavray/avro-rs with apache/avro. Replace MIT license with Apache License 2

Remove a link to an issue/PR at flavray/avro-rs/pull/76 because it is already resolved. Enable (uncomment) the tests related to this issue. Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>

RGafiyatullin changed the title ~~Support union-types~~ Support union-types (multiple named record variants) Mar 14, 2019

Support for union-types (Schema, Value). The whole serde-stuff — stil…

12f76e5

…l to be done

RGafiyatullin force-pushed the rg-support-adt-nolog branch from 894f1bb to 12f76e5 Compare March 14, 2019 15:24

Support for union-types (Serde)

f4cb5a1

RGafiyatullin changed the title ~~Support union-types (multiple named record variants)~~ Support for Union-Types (Serde, Schemas, Avro-Values — the lot) Mar 15, 2019

RGafiyatullin changed the title ~~Support for Union-Types (Serde, Schemas, Avro-Values — the lot)~~ Support for Union-Types (Serde, Schemas, Avro-Values) Mar 15, 2019

add some tests about serde-ing structs

60801d6

flavray reviewed Mar 24, 2019

View reviewed changes

RGafiyatullin added 3 commits March 27, 2019 17:58

Slightly reworked Schema Resolution

e452e75

Getting up to speed with flavray/master@06a1b01

066902d

cargo fmt

3971f0d

RGafiyatullin mentioned this pull request Mar 28, 2019

Is it possible to Serialize/Deserialize rust enums? #80

Open

kphelps mentioned this pull request Apr 1, 2019

Support unions of named types #65

Closed

Made avro_rs::schema::UnionSchema::new(...) public: there is no way t…

7d37116

…o create a union-schema other than parsing a string

nlopes mentioned this pull request Sep 2, 2019

Add schema compatibility to library #88

Merged

CtrlC-Root mentioned this pull request Oct 4, 2019

Unions: Support Many Named Types #61

Open

CtrlC-Root mentioned this pull request Oct 4, 2019

Modify UnionSchema to support multiple named types. #90

Closed

poros added enhancement feature request labels Oct 13, 2019

poros mentioned this pull request Oct 13, 2019

Support same types but with different names in unions #95

Open

poros mentioned this pull request Jan 14, 2020

Partial Rust Enum Serialization/Deserialization Support #60

Closed

poros self-requested a review January 14, 2020 22:40

poros closed this Jun 7, 2020

martin-g mentioned this pull request Oct 15, 2021

AVRO-3205 Rust: Update Cargo.toml [package] information apache/avro#1344

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for Union-Types (Serde, Schemas, Avro-Values) #76

Support for Union-Types (Serde, Schemas, Avro-Values) #76

RGafiyatullin commented Mar 14, 2019 •

edited

Loading

flavray left a comment

flavray Mar 24, 2019

RGafiyatullin Mar 26, 2019 •

edited

Loading

nlopes commented Sep 2, 2019

CtrlC-Root commented Oct 4, 2019

poros commented Jan 14, 2020 •

edited

Loading

poros commented Jun 7, 2020

Support for Union-Types (Serde, Schemas, Avro-Values) #76

Support for Union-Types (Serde, Schemas, Avro-Values) #76

Conversation

RGafiyatullin commented Mar 14, 2019 • edited Loading

flavray left a comment

Choose a reason for hiding this comment

flavray Mar 24, 2019

Choose a reason for hiding this comment

RGafiyatullin Mar 26, 2019 • edited Loading

Choose a reason for hiding this comment

nlopes commented Sep 2, 2019

CtrlC-Root commented Oct 4, 2019

poros commented Jan 14, 2020 • edited Loading

poros commented Jun 7, 2020

RGafiyatullin commented Mar 14, 2019 •

edited

Loading

RGafiyatullin Mar 26, 2019 •

edited

Loading

poros commented Jan 14, 2020 •

edited

Loading