Skip to content

Apache Avro Rust SDK 0.22.0 release

Latest
Latest

Choose a tag to compare

View all tags
@martin-g martin-g released this 10 Aug 06:49
· 15 commits to main since this release
rel/release-0.22.0
This tag was signed with the committer’s verified signature.
martin-g Martin Grigorov
GPG key ID: 3194FD8C1AE300EF
Verified
Learn about vigilant mode.
ec5721c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The Apache Avro Rust SDK 0.22.0 release is a large release with lots of features, fixes and breaking changes. It is strongly recommended to read the Breaking changes and New features sections of the changelog.

What's changed

Breaking changes

  • Resolve schema(ta) in the builder of Writer by @Kriskras99 in #328
    • The builder now returns an AvroResult
  • Rework schema compatibility by @Kriskras99 in #342
    • SchemaCompatiblity::can_read and SchemaCompatiblity::mutual_read now return a Compatiblity type
  • Bump MSRV from 1.85 to 1.88 by @Kriskras99 in #342, #348
  • Schema::Duration now has a FixedSchema allowing access to attributes and Serialize and Deserialize is implemented for apache_avro::Duration by @jdarais in #382
  • AvroSchema and AvroSchemaComponent are no longer behind the derive feature and have moved from apache_avro::schema::derive to apache_avro::serde by @martin-g in #394 and @Kriskras99 in #433
  • Change the implementation of AvroSchemaComponent for Uuid from Schema::Uuid(String) to Schema::Uuid(Fixed(name: "org.apache.avro.rust.Uuid", size: 16)) and allow overwriting a field's schema with the with attribute when using #[derive(AvroSchema)] by @Kriskras99 in #397, #558
    • When #[avro(with)] is specified without an argument, it will call the get_schema_in_ctxt(&mut HashSet<Name>, NamespaceRef<'_>) -> Schema in the same module as specified by #[serde(with)]
    • When #[avro(with = path::to::some_fn)] is specified with a path, the function it's pointing to will be called with two parameters &mut HashSet<Name>, NamespaceRef<'_>
    • When #[avro(with = || {})] is specified with a closure, that closure is invoked
    • The name of the schema for uuid::Uuid is org.apache.avro.rust.uuid
    • The schema for Uuid can be changed back to a Schema::Uuid(String) using #[avro(with = || Schema::Uuid(UuidSchema::String))] on a field with a Uuid and setting human_readable to true
  • Replace the From<&str> implementations for Name and Alias with TryFrom<&str>, TryFrom<String>, and FromStr as the conversion is not fallible by @Kriskras99 in #423
  • Rework SpecificSingleObjectWriter removing ::with_capacity and changing write_avro_datum_ref to also take a &NamesRef parameter by @Kriskras99 in #445
  • Remove the default field from FixedSchema as it is not in the specification by @Kriskras99 in #460
  • AvroSchemaComponent::get_schema_in_ctxt now takes a &mut HashSet<Name> parameter for the named_schemas parameter by @Kriskras99 in #471
  • Replace the Schema::map{,_with_attributes} and Schema::array{,_with_attributes} with builders allowing the user to set more fields by @martin-g in #472
    • This also adds Schema::r#enum, Schema::fixed and Schema::record
  • Stricter schema parsing by @Kriskras99 in #479
    • This now rejects schemas that were previously accepted by this library, they were already rejected by parsers in different languages
  • Alias::name and Alias::namespace now return references by @Kriskras99 in #486
  • Remove unused order and position fields in RecordField by @Kriskras99 in #491
  • Rework Name to be more performant which requires changing the SchemaNameValidator trait by @Kriskras99 in #493
    • The regex returned by SchemaNameValidator::regex must have a capture group named name which captures the unqualified name when not overriding the SchemaNameValidator::validate function
    • The SchemaNameValidator::validate must now return the start byte of the unqualified name
  • Support enums and tuples in SchemaAwareSerializer, implement SchemaAwareDeserializer and document the mapping between the Serde and Avro data models by @Kriskras99 in #512, #558
    • The SchemaAware* now have specific requirements for what input they accept, so type definitions might need to be modified to make them work with the new code (especially for enums and tuples)
    • This also makes human_readable configurable per reader and writer instance
    • Array and Map serialisation will not write the block size in bytes unless configured to do so, allowing unbuffered serialisation
    • The AvroSchemaComponent implementation for std::time::Duration has changed to match the (de)serialisation implementation as defined by Serde
      • It is now a Schema::Record(name: "org.apache.avro.rust.Duration", fields: ["secs": "org.apache.avro.rust.u64", "nanos": "long"])
    • The AvroSchemaComponent implementation for BigDecimal is now a Schema::String matching the (de)serialisation implementation as defined by bigdecimal
      • It can be changed back to a Schema::String using #[serde(with = "apache_avro::serde::bigdecimal"), avro(with)] on a field with a BigDecimal
    • The SchemaAwareDeserializer does not support schema resolution, for that use the old from_value implementation (see #575)

New features

  • Make DeflateSettings::compression_level public by @EmilyMatt in #335
  • Support fixed as a type for uuid by @Kriskras99 in #339
  • Don't depend on Serde to provide fields in the right order by @Kriskras99 in #351
    • It is still recommended to match the field order between the type and the schema as out-of-order serialisation incurs a performance penalty
  • Support the #[serde(flatten)] attribute by @Kriskras99 in #359, #448
    • This attribute is not supported when using apache_avro::serde::to_value
  • Use the Serde attributes and check for conflict with the Avro attributes by @Kriskras99 in #377
    • This deprecates the Avro attributes for which the value must always match the Serde attribute:
      • Container attributes:
        • #[avro(name = "..")] -> #[serde(rename = "..")]
        • #[avro(rename_all = "..")] -> #[serde(rename_all = "..")]
      • Variant attributes:
        • #[avro(rename = "..")] -> #[serde(rename = "..")]
      • Field attributes:
        • #[avro(alias = "..")] -> #[serde(alias = "..")]
        • #[avro(rename = "..")] -> #[serde(rename = "..")]
        • #[avro(skip)] -> #[serde(skip)]
    • On nightly compilers, using deprecated attributes will throw a compiler warning
  • Support the #[serde(transparent)] attribute on structs by @Kriskras99 in #398
  • Implement AvroSchemaComponent for char, u64, u128, i128, () by @Kriskras99 in #414, #486, #558
    • char: Schema::String
    • u64: Schema::Fixed(name: "org.apache.avro.rust.u64", size: 8)
    • u128: Schema::Fixed(name: "org.apache.avro.rust.u128", size: 16)
    • i128: Schema::Fixed(name: "org.apache.avro.rust.i128", size: 16)
    • (): Schema::Null
  • Add RecordSchemaBuilder::try_name which accepts any TryInto<Name> and make RecordFieldBuilder::name take any Into<String> by @martin-g in #429
  • Move and rename the serde_avro_* modules to apache_avro::serde::*, the old modules are still available but deprecated and will be removed in a future release by @Kriskras99 in #430
    • apache_avro::serde_avro_bytes -> apache_avro::serde::bytes
    • apache_avro::serde_avro_bytes_opt -> apache_avro::serde::bytes_opt
    • apache_avro::serde_avro_fixed -> apache_avro::serde::fixed
    • apache_avro::serde_avro_fixed_opt -> apache_avro::serde::fixed_opt
    • apache_avro::serde_avro_slice -> apache_avro::serde::slice
    • apache_avro::serde_avro_slice_opt -> apache_avro::serde::slice_opt
  • Add posibility to append values to writer without validating by @Kriskras99 in #447
    • This renames Writer::append to Writer::append_value although the old function currently still exists and is deprecated
    • This adds Writer::unvalidated_append_value and Writer::unvalidated_append_value_ref which skip the validation step
  • Rework the documentation by @Kriskras99 in #451
  • The introduction to Avro itself was moved to the documentation::primer module
  • Using the crate the "Avro" way was moved to the documentation::generic module
  • Using the crate the "Serde" way was moved to the serde module
  • Calculating schema fingerprints was moved to the Schema::fingerprint function
  • Custom name validators was moved to the validator module
  • Custom schema equality was moved to the schema_equality module
  • Document #[derive(AvroSchema)] in the serde module
  • Suggest the boolean type when a schema contains a type named bool that does not resolve by @Kriskras99 in #459
  • Add support for resetting a Writer backed by a clearable buffer by @Kriskras99 in #469
  • Make Schema::denormalize public by @Kriskras99 in #470
  • Allow types implementing AvroSchemaComponent to provide a default value by @Kriskras99 in #477
  • Custom Debug implementations for *Schema types hiding empty fields for more readability by @Kriskras99 in #488
  • Add a builder for unions: UnionSchemaBuilder by @Kriskras99 in #486
  • Replace the from_avro_datum* functions with GenericDatumReader by @Kriskras99 in #496
    • The functions are still available, but are deprecated and will be removed in a future version
  • Replace the to_datum* functions with GenericDatumWriter by @Kriskras99 in #499
    • The functions are still available, but are deprecated and will be removed in a future version
  • Support #[serde(remote = "..")] by @Kriskras99 in #564
  • Support deriving AvroSchema on enums and tuples by @Kriskras99 in #569
    • If some enum or tuple is not supported, and you think it should, please open an issue!
  • Allow to supply own names when using Value::resolve and Value::Validate by @Kriskras99 in #579

Fixes

  • Choose correct schema when serializing an Option<Enum> by @PookieBuns in #337
  • Support deserialization of BigDecimal fields when using "the Serde way" by @eft-prima in #338
  • Don't write the avro.codec attribute if the codec is Null by @Kriskras99 in #356
  • Resolving a Value::Long into a Value::Int no longer truncates and returns an error instead by @Kriskras99 in #392
  • logicalType is no longer included in custom attributes by @jdarais in #395
  • Do not match types by name in the derive code by @Kriskras99 in #401
  • Calculate the RecordSchema lookup table on RecordSchemaBuilder::build() by @martin-g in #411
  • UnionSchema::new no longer accepts multiple schemas with the same name by @marting-g in #413
  • Support recursive types for Schema::independent_canonical_form by @Kriskras99 in #420
  • Select the first union variant with the correct byte length when serializing by @Kriskras99 in #421
  • Use RAII for setting thread locals used during (de)serialisation so failure doesn't impact the next (de)serialisation by @martin-g in #432
  • Ensure SpecificSingleObjectWriter writes header on every call by @flxo in #438
  • Don't allow serializing bytes as a Uuid::String by @Kriskras99 in #441
  • Don't allow resolving a schema that is already known by @martin-g in #444
  • Don't silently truncate numbers larger than i64::MAX when converting from JSON by @Kriskras99 in #450
  • Fix links in rustdoc by @martin-g in #505
  • Remove unused CHANGELOG.md by @martin-g in #541
  • Support BigDecimal in Deserializer when apache_avro::serde::bigdecimal is used by @jdarais in #543
  • Return an error instead of panicking on invalid name, alias, or type reference by @youichi-uda in #547
  • Accept JSON string defaults for decimal fields in unions by @valdo404 and @slavische in #580
  • Don't allow negative object counts and don't allow negative block lengths by @MitNarodia in #586
  • Bound decompression size to guard against decompression bombs by @iemejia in #582
  • Bound the size of decoded Value::Map and Value::Array to the max allocation size by @iemejia in #581
  • Support enum inside an untagged enum by @Kriskras99 in #591

Internal changes

Dependency bumps

  • Bump actions/cache from 5 to 6.1.0
  • Bump actions/checkout from 5 to 7.0.0
  • Bump actions/dependency-review-action from 4 to 5.0.0
  • Bump actions/setup-java from 5.0.0 to 5.6.0
  • Bump bigdecimal from 0.4.9 to 0.4.10
  • Bump bon from 3.8.1 to 3.9.3
  • Bump criterion from 0.7.0 to 0.8.2
  • Bump darling from 0.21.3 to 0.23.0
  • Bump digest from 0.11.2 to 0.11.3
  • Bump env_logger from 0.11.8 to 0.11.11
  • Bump github/codeql-action/* from 4.36.2 to 4.37.1
  • Bump hex-literal from 1.0.0 to 1.1.0
  • Bump liblzma from 0.4.5 to 0.4.7
  • Bump log from 0.4.28 to 0.4.33
  • Bump miniz_oxide from 0.8.9 to 0.9.1
  • Bump num-bigint from 0.4.6 to 0.4.8
  • Bump proc-macro2 from 1.0.103 to 1.0.107
  • Bump proptest from 1.9.0 to 1.11.0
  • Bump quote from 1.0.41 to 1.0.47
  • Bump rand from 0.9.2 to 0.10.2
  • Bump regex-lite from 0.1.8 to 0.1.9
  • Bump rustversion from 1.0.22 to 1.0.23
  • Bump serde from 1.0.228 to 1.0.229
  • Bump serde_json from from 1.0.145 to 1.0.150
  • Bump sha2 from 0.10.9 to 0.11.0
  • Bump shogo82148/actions-setup-perl from 1 to 1.42.0
  • Bump snap from 1.1.0 to 1.1.2
  • Bump strum from 0.27.2 to 0.28.0 and replace strum-macros with features = ["derive"] on strum by @Kriskras99 in #483
  • Bump syn from 2.0.108 to 2.0.119
  • Bump taiki-e/install-action from 2 to 2.84.0
  • Bump thiserror from 2.0.17 to 2.0.19
  • Bump trybuild from 1.0.114 to 1.0.118
  • Bump uuid from 1.18.1 to 1.23.5
  • Bump wasm-bindgen from 0.2.97 to 0.2.114
  • Bump wasm-bindgen-test from 0.3.55 to 0.3.64
  • Bump zizmorcore/zizmore-action from 0.5.7 to 0.6.0
  • Replace ctor (v0.6.0) with linktime (v0.18.0)
  • Pin cargo-rdme to 1.4.2 by @martin-g in #567
    • This is because cargo-rdme 2 depends on specific nightly versions and therefore easily breaks in CI

New Contributors

Full Changelog: rel/release-0.20.0...rel/release-0.22.0

Contributors

iemejia, flxo, and 14 other contributors
Assets 2
Loading