New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AVRO-3126: Add support for java records #1680
base: main
Are you sure you want to change the base?
Conversation
lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectDatumReader.java
Show resolved
Hide resolved
lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectRecordEncoding.java
Outdated
Show resolved
Hide resolved
lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
Outdated
Show resolved
Hide resolved
PR is ready for review. |
I would love the testing in java 17 module if nothing else but to see some examples of the workflow/use. Lots of really cool stuff here that Im still trying to get my head around! |
432bde0
to
9704244
Compare
@jklamer I have added a java17 module. Also built on top of this branch, I have added polymorphic support. |
I've approved the workflow run! The new CI related code could be simplified. I will do it soon! I'll let the Java developers to comment on the Java changes! |
@martin-g I noticed a bug in how I had conditionally included the java17 module with a profile. I changed it always to be included but have the compile and test plugins disabled so that it won't complain about the version of java. |
We should not merge this PR for |
Have extended the functionality of the CustomEncoding to implement this AvroEncode annotation can now also be added to a class adding a new module to write tests that require java 17 features Added module that doesn't compile or test by default
@ashley-taylor The PR should be reviewed and merged by a committer who knows better the Java SDK. |
Hello! Thanks for your contribution and for your patience! I've set this fix version for the next major release of Avro (1.12.0, later this year). I know record support would be a great feature to get in. Do you have a JIRA account so we can assign this JIRA to you and ensure that it's properly taken care of? |
Hi @RyanSkraba thanks for the update Have added a comment on the Jira ticket I also have another Branch to address AVRO-1568 Polymorphism that I have built on top of this branch. Looking to open that PR after this one. |
I tried to add Ashley to the Contributors role in JIRA but there are 5 JIRA users with that (display) name and I am not sure which one to add. |
ashley-taylor is my jira username |
I did nothing! :-) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a great addition.
I like the care given to the combination of records, wrapped classes, inheritance and custom encodings. That could otherwise cause bugs.
anything I can do to help this get merged? |
@ashley-taylor The JIRA ticket has |
I would like also like to express my interest in this getting merged. We use Avro with our Apache Beam classes, and being able to use Record classes would make things more seamless. |
I have made some additional Java 17 improvements that are based on this and I wondering if I should add this branch. |
Just wanted to share that record support for Avro would be really nice to have! |
To implement this feature, I used a
CustomEncoding
. Added two new methods to pass theDatumReader
/DatumWriter
, allowing otherCustomEncoding
to not have to define an entire custom schema but use a hybrid approach.With this, I also change
AvroEncode
to be able to specify at a class level. Feels generically helpful to be able to instrument the class instead of all fields referencing the class.Used reflection to read the
isRecord
method from theclass
class so that it will work on older versions of java.For records, fields can not be modified or accessed with unsafe offsets reads. Instead, the constructor must be invoked, and reads must be done using either method/field reflection.
To identify the appropriate constructor, I use the order of the fields. Since it is a record and can not be extended, reading the direct fields from the record class is safe.
Make sure you have checked all steps below.
Jira
Tests
Commits
Documentation