Skip to content

Releases

v0.7.0

Choose a tag to compare

@dannote dannote released this 26 Oct 15:35
9cb1744

Removed

  • The :decoder_sources option. Generated helpers are typed against the
    reference Decoder, whose method signatures a crate's own keeps.
  • The :sparse_messages option. Sparse message decoders always use one table
    of their fields, shared with skip decoders when both are generated, which is
    what :descriptor did; :match generated the same Rust another way.
  • Unknown options raise instead of being ignored.

Changed

  • The generator's helper Rust comes from ordinary RustQ modules, included whole
    per feature, instead of modules built at generation time and lists of the
    functions to take from them. Full and sparse decoders share their primitive
    readers, enum decoding, and field reading, and KiwiDecodeFn replaces
    KiwiSparseDecodeFn and KiwiFullDecodeFn. The skip helpers are one module,
    KiwiCodec.RustlerGenerator.Skip.Helpers, which RustQ code calling them names
    instead of Skip.ValueHelpers. This needs RustQ 1.0.0-rc.19.

  • The record feature writes each definition once, as a row of one
    kiwi_records! call: its fields and enum values one per line. The macro,
    in the record runtime, expands to the definition table and the
    kiwi_record_definitions and kiwi_records constants as before, which
    the generator used to write out separately, one table field per line.
    Figma's schema generates about 630 KB less Rust.

v0.6.0

Choose a tag to compare

@dannote dannote released this 25 Oct 16:04
c603ebb

Changed

  • The constants naming each definition's fields and enum values are associated
    constants of a type per definition, kiwi_records::NodeChange::FILL_PAINTS,
    instead of a module per definition, kiwi_records::node_change::FILL_PAINTS.
    Rust code naming them changes accordingly.
  • RustQ code reaches them through KiwiCodec.RustlerGenerator.Records:
    Records.NodeChange.FILL_PAINTS and Records.Definitions.NODE_CHANGE, in a
    module that aliases it, with no defrustmod of its own. This needs RustQ
    1.0.0-rc.17.

v0.5.1

Choose a tag to compare

@dannote dannote released this 25 Oct 10:39
2d15d81

Changed

  • KiwiCodec.RustlerGenerator.Records provides the record runtime to RustQ.
    RustQ code that works with records names the module through alias and
    writes Records.kiwi_record() in its specs, instead of reading the runtime's
    Rust file from the dependency's directory. The runtime is templates/records.rs. This needs RustQ 1.0.0-rc.16.

v0.5.0

Choose a tag to compare

@dannote dannote released this 24 Oct 17:06
1b69118

Added

  • KiwiCodec.RustlerGenerator.Decoder documents the Decoder generated
    decoders read with, and ships decoder.rs, a complete one. The generator
    reads its method signatures unless :decoder_sources names another
    Decoder, so every feature knows which reads return a NifResult.
  • Projects that depend on KiwiCodec can use KiwiCodec.RustlerGenerator.
    RustQ is an optional dependency, and the generator compiles wherever it is
    available, such as a project's development and test builds; it used to
    compile only in KiwiCodec's own, so projects loaded its sources themselves.
  • KiwiCodec.Schema.Token has a kind: :identifier, :integer,
    :symbol, or :eof.

Changed

  • The docs have a Native decoders guide covering each generator feature,
    entrypoints, and the Decoder. The generator architecture guide is now
    Internals, for people changing KiwiCodec, and the module groups separate the
    public API from internals.
  • The schema tokenizer reads with binary patterns rather than a regular
    expression, and is about 14 times faster on Figma's schema. Tokens,
    positions, and errors are unchanged.
  • Skip and sparse helpers are always built from Decoder metadata. Without
    :decoder_sources, they used to come from a separate set of helpers that
    did not know which reads could fail.
  • Item macro rows in generated Rust separate their values with commas, as
    RustQ 1.0.0-rc.14 writes them.
  • Records name a definition whose name is a Rust keyword with a raw
    identifier, such as kiwi_records::r#type, rather than appending _ from a
    list of keywords that missed gen, try, and others.
  • Generated Rust builds on rustler 0.37 to 0.39. It used to call
    rustler::wrapper, which rustler 0.39 removed.
  • Require RustQ 1.0.0-rc.14 for code generation.

Removed

  • RustlerGenerator.RustExpr.ident/1 and indent/2, and
    RustlerGenerator.Skip.field_expr/2. RustExpr.primitive/1 returns RustQ
    AST rather than Rust source.

Fixed

  • The :full feature, the default, generates Rust that compiles and decodes
    as KiwiCodec.decode/2 does. Its entrypoints moved the decoder they then
    finished, its message macro called an undefined function, its reads were
    not propagated with ?, its strings were read without env, and a float
    that is infinite or NaN raised instead of decoding to an atom.
  • Generating both sparse and skip decoders without :decoder_sources defined
    kiwi_skip_descriptor_kind twice, and skip decoders alone with
    :decoder_sources called it without defining it.
  • A schema name that is not a Rust identifier fails where it is rendered,
    rather than having its other characters replaced with _, which could give
    two names the same Rust name.

v0.4.0

Choose a tag to compare

@dannote dannote released this 23 Oct 11:31
167925c
  • Add the :unknown_fields Rustler generator option. :runtime_schema makes
    generated message decoders skip fields the compiled schema does not define,
    using a KiwiRuntimeSchema read from the schema that accompanies the data, so
    data written with a newer, compatible schema decodes instead of failing with
    BadArg. The default, :error, keeps rejecting such fields.
  • Route every generated message decoder's unknown fields through one
    kiwi_unknown_field helper, and pass full message decoders their definition
    name.
  • Accept a :schema option in KiwiCodec.decode/3 to skip message fields the
    generated modules do not define, using the schema the data was written with.
  • Add KiwiCodec.SchemaInterpreter.skip_message_field/4.
  • Add the :records Rustler generator feature. It decodes Kiwi data into
    native records Rust code can read, change, and share without building Elixir
    terms, then encodes them as the sparse maps generated decoders return,
    optionally only some fields. Generated constants name each definition, field,
    and enum value; records have typed getters and helpers that update nested
    records and scalar, string, enum, and float-array fields. See
    KiwiCodec.RustlerGenerator.Records.
  • Sparse decoders intern their module and field-name atoms once instead of for
    every map they build, which took two thirds of the time decoding large files.
  • Require RustQ 1.0.0-rc.13 for code generation.
  • Rename the Rustler generator's skip, sparse, and full decoder helpers by
    family: RustlerGenerator.SkipHelpers, SkipValueHelpers, and
    SkipDescriptor are RustlerGenerator.Skip.Helpers, Skip.ValueHelpers,
    and Skip.Descriptor; SparseHelpers is Sparse.Helpers; and
    FullDecoderHelpers is Full.Helpers.

v0.3.1

Choose a tag to compare

@dannote dannote released this 20 Jul 17:24
  • Compile the RustQ-backed Rustler generator only in development and test
    environments, keeping production codec consumers free of RustQ and its
    development-tool dependency graph.
  • Continue shipping the Rustler generator source for development-time native
    generation while keeping the production runtime package limited to the Kiwi
    codec and schema APIs.
  • Remove the unused Jason dependency from the production package.

v0.3.0

Choose a tag to compare

@dannote dannote released this 17 Jul 19:34
  • Restore the required slice borrow for match-backed sparse field tables when
    decoder source metadata is enabled.
  • Hide internal Rust-only helper types from ExDoc type-reference rendering.
  • Add reusable prepared runtime schemas with indexed definition, message-field,
    and enum lookup for repeated SchemaInterpreter calls.
  • Replace the timer-loop codec benchmark with Benchee coverage for small and
    256-field sparse/dense static and interpreted codecs, including allocation
    measurements.
  • Clarify Rustler generator architecture docs now that compact decoder macros
    are RustQ-authored rather than raw Rust heredoc escape boundaries.
  • Deduplicate Rustler generator skip descriptor selection and remove exact clone
    findings from the generator helper modules.
  • Add optional decoder_sources metadata for RustQ-authored skip helpers.
  • Emit Rustler macro prelude helpers only for requested generator features.
  • Add opt-in descriptor-backed sparse message generation with
    sparse_messages: :descriptor.
  • Move sparse primitive value helpers, descriptor scanning, descriptor dispatch,
    and sparse descriptor declarations into RustQ-authored defrust / @type
    support.
  • Author the sparse message descriptor macro with item-generating
    defrustmacro and compact repeat macro-template support.
  • Share descriptor field inventories between sparse and skip message decoders
    when both descriptor sparse and skip features are generated.
  • Author sparse enum and sparse struct descriptor decoding with RustQ-authored
    defrust helpers and item-generating defrustmacro.
  • Avoid emitting raw match-backed sparse/skip message macros when decoder-source
    descriptor sparse generation does not use them.
  • Author decoder-source skip struct decoders with RustQ-authored defrust
    helpers and item-generating defrustmacro.
  • Replace active shared sparse/skip descriptor kiwi_skip_kind! token macro use
    with explicit descriptor metadata and RustQ-authored enum construction.
  • Move descriptor-backed skip message scanning into RustQ-authored defrust
    helpers with sorted field lookup.
  • Generate skip descriptor function, enum, and field declarations from RustQ
    type metadata when decoder sources are available.
  • Collapse the separate sparse bytes descriptor mode into ordinary one
    descriptor mode.

v0.2.2

Choose a tag to compare

@dannote dannote released this 28 Jun 08:53
  • Centralize primitive Rustler decoder metadata.
  • Compact generated skip decoders.
  • Deduplicate sparse enum decoders when full enum decoders are generated.
  • Compact generated enum decoders.
  • Remove the unused Rustler decoder macro path.
  • Update RustQ dependency to ~> 0.8.2.