AVRO-2918: Polymorphism #1776

clesaec · 2022-07-21T14:46:44Z

AVRO-2918 : Proposition for polymorphism

lang/java/avro/src/main/java/org/apache/avro/Schema.java

lang/java/avro/src/test/java/org/apache/avro/io/TestResolvingIO.java

lang/java/avro/src/main/java/org/apache/avro/Resolver.java

opwvhk · 2022-07-22T10:10:08Z

lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java

+    final Schema realSchema;
+    if (expected.hasChild()) {
+      int index = in.readExtends();
+      realSchema = expected.findInHierachy(index);
+    } else {
+      realSchema = expected;
+    }


This seems to be at odds with GenericDatumWriter, lines 239-241: there, the extends is optional, based on whether the parent schema or one of its children is written.

indeed, i have to add unit test and fix that.

You were right, so now writer will always write the "extends" symbol if schema has child.

opwvhk · 2022-07-22T10:12:05Z

lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumWriter.java

+    if (!Objects.equals(realSchema, schema)) {
+      out.writeExtends(realSchema.getIndex());
+    }


This creates two possible binary representations of the same (parent) schema: one for writing a parent schema, and a different one for child schemata.

How can the reader distinguish between the two?

opwvhk · 2022-07-22T10:24:12Z

lang/java/avro/src/test/java/org/apache/avro/io/LegacyBinaryEncoder.java

@@ -178,6 +178,11 @@ public void writeIndex(int unionIndex) throws IOException {
    encodeLong(unionIndex, out);
  }

+  @Override
+  public void writeExtends(int extendsIndex) throws IOException {
+    encodeLong(extendsIndex, out);


Should the extends clause be supported by the legacy encoder at all?

Or should it throw?

I don't know ... i can throw an exception ....

opwvhk · 2022-07-22T10:27:23Z

lang/java/avro/src/main/java/org/apache/avro/Schema.java

@@ -345,6 +379,9 @@ public String getFullName() {
    return getName();
  }

+  public void terminateInit() {


What does this method do?

I understand methods like hasChild, visitHierarchy and findInHierarchy above (though they also need javadoc), but I cannot understand the what&why of this one.

It's a kind of PostConstruct, for SchemaRecord, once all schemas has been built and list of fields set.
But, indeed, i will see if we can do without it

Ah, sort of like how the fields are marked immutable once set for the first time? (and any field use before they are set yields an exception)

How about this idea:

in the constructor, mark if the record schema will have children (if the parameter is omitted, it will not)

have a method where you can set all child schemata, once

This behaves like setFields: it allows us both to finalise initialisation and makes the schema unusable for (de)serialisation until the child schemata have been set (so it cannot be forgotten).

So, i finally find an easy way to remove this weird method.

I saw; lazy initialisation. Also a good idea.

lang/java/avro/src/main/java/org/apache/avro/io/FastReaderBuilder.java

github-actions bot added the Java Pull Requests for Java binding label Jul 21, 2022

github-advanced-security bot found potential problems Jul 21, 2022

View reviewed changes

opwvhk reviewed Jul 22, 2022

View reviewed changes

clesaec force-pushed the avro-2918_Polymorphism branch from 9afb5d3 to c704d9a Compare July 28, 2022 11:15

RyanSkraba changed the title ~~Avro 2918 polymorphism~~ AVRO-2918: Polymorphism Aug 25, 2022

clesaec force-pushed the avro-2918_Polymorphism branch from c704d9a to 0796783 Compare August 30, 2022 09:37

clesaec force-pushed the avro-2918_Polymorphism branch from 0796783 to ed665ae Compare January 6, 2023 07:49

github-advanced-security bot found potential problems Jan 6, 2023

View reviewed changes

lang/java/avro/src/main/java/org/apache/avro/io/FastReaderBuilder.java Fixed Show fixed Hide fixed

clesaec force-pushed the avro-2918_Polymorphism branch from ec3bfc1 to bb84a43 Compare June 15, 2023 13:10

clesaec mentioned this pull request Jul 6, 2023

AVRO-2918: [java] Schema polymorphism #2322

Open

clesaec force-pushed the avro-2918_Polymorphism branch from bb84a43 to 9d3b24c Compare August 1, 2023 08:07

clesaec force-pushed the avro-2918_Polymorphism branch from 9d3b24c to 5d8ab89 Compare August 17, 2023 15:05

clesaec force-pushed the avro-2918_Polymorphism branch from 5d8ab89 to 050fb06 Compare September 12, 2023 08:55

opwvhk approved these changes Sep 12, 2023

View reviewed changes

clesaec added 2 commits December 13, 2023 10:57

AVRO-2918: Polymorphism

e7e865c

AVRO-2918 - rebase

6c8ddae

clesaec force-pushed the avro-2918_Polymorphism branch from 050fb06 to 6c8ddae Compare December 13, 2023 13:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AVRO-2918: Polymorphism #1776

AVRO-2918: Polymorphism #1776

clesaec commented Jul 21, 2022

opwvhk Jul 22, 2022

clesaec Jul 22, 2022

clesaec Jul 22, 2022

opwvhk Jul 22, 2022

clesaec Jul 22, 2022

opwvhk Jul 22, 2022

clesaec Jul 22, 2022

opwvhk Jul 22, 2022

clesaec Jul 22, 2022

opwvhk Jul 22, 2022

clesaec Jul 22, 2022

opwvhk Jul 22, 2022

AVRO-2918: Polymorphism #1776

Are you sure you want to change the base?

AVRO-2918: Polymorphism #1776

Conversation

clesaec commented Jul 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment