Skip to content

Release erli18n-v0.5.0

Choose a tag to compare

@github-actions github-actions released this 24 Jun 02:01

Packaging and public-API minor. Two coupled changes drive the minor bump under
the 0.x SemVer policy above: a new public export (erli18n_po:escape_string/1,
detailed under Added) and the repository's move to a rebar3 umbrella in which
erli18n is now a fully self-contained Hex package (detailed under
Packaging).

Added

  • erli18n_po:escape_string/1 is now exported as public API — a
    runtime/published-module change to erli18n_po (not a layout-only one).
    It applies the five GNU gettext PO escapes (backslash, double-quote,
    newline, tab, carriage return) and is the exact escaping dump/1 already
    used internally. It is promoted to public API so the separate
    rebar3_erli18n plugin can serialize the PO metadata it owns (the #|
    previous-msgid lines) byte-identically to dump/1 across the published
    {deps, [erli18n]} boundary, instead of vendoring a duplicate escaper that
    would have to stay in lock-step forever. Additive only; no existing behavior
    changes.

Security

  • erli18n_po:parse/1,2 plural-index validation no longer allocates a list
    sized by the untrusted nplurals= header (anti-DoS).
    The PSD-009 cross-check
    in validate_plural_indices/3 previously built lists:seq(0, Nplurals - 1),
    where Nplurals comes straight from the .po Plural-Forms header. The
    loader only caps that value's DIGIT COUNT (7 digits, up to 9,999,999), so a
    ~158-byte adversarial .po declaring nplurals=9999999 plus a single
    msgstr[0] line forced a ~10-million-element list (~80 MB, reproduced at
    ~340 ms versus ~0.1 ms for a real catalog) before reporting the mismatch. The
    validation now checks the index set without ever materializing the
    header-sized sequence — it requires the present indices to be a dense 0-based
    prefix (sized by the bytes actually in the file) whose length equals
    Nplurals — so the same malicious input is rejected in bounded time. The
    structured {plural_count_mismatch, Msgid, Nplurals, Indices} error a genuine
    count mismatch returns is byte-for-byte unchanged; only the resource bound is
    fixed.

Fixed

  • erli18n_po:parse/1,2 continuation-line accumulation is now genuinely
    O(total).
    A msgid/msgstr/msgctxt/msgid_plural/msgstr[N] field
    spread across many continuation lines was accumulated by appending a growing
    binary held inside the parser's per-entry record
    (<<Prev/binary, Bin/binary>>); because that accumulator had more than one
    reference, the runtime's in-place binary-append optimization did not apply and
    the build degraded to super-linear on a many-continuation field. Each
    continuation segment is now prepended onto a reversed list in O(1) and the
    whole field is joined exactly once at finalization (iolist_to_binary/1), so
    the per-field build is linear in the total byte count by construction rather
    than depending on a runtime heuristic. The parsed bytes are unchanged.

Packaging

  • erli18n is now a self-contained Hex package inside the umbrella. Its
    README.md, CHANGELOG.md, and LICENSE were relocated from the repo root
    into apps/erli18n/ so the published tarball ships them, and the package's
    ex_doc / {hex, [{doc, #{provider => ex_doc}}]} configuration moved from
    the root rebar.config into apps/erli18n/rebar.config. The root keeps only
    umbrella-wide and shared-community files. Required because rebar3_hex
    computes the package file set strictly inside the app directory: with the
    package files at the repo root, the 0.4.0 tarball shipped only
    include/erli18n.hrl, rebar.config, and src/*.erl — no
    README/CHANGELOG/LICENSE. No runtime module behavior changed.

Changed

  • The test suite no longer makes a runtime eqwalizer:dynamic_cast/1
    call.
    The nine property/fuzz/CT modules that bridged PropEr's
    statically-term() generator boundaries
    (erli18n_po_props, erli18n_negotiate_props, erli18n_lookup_props,
    erli18n_plural_props, erli18n_interp_props, erli18n_po_fuzz,
    erli18n_server_SUITE, erli18n_pt_store_SUITE, erli18n_loader_SUITE)
    now reconcile those boundaries with a static
    -eqwalizer({nowarn_function, F/A}). annotation on each affected function —
    the same zero-runtime-dep pattern already used in the runtime modules
    erli18n_server and erli18n_pt_store — instead of calling the
    eqwalizer:dynamic_cast/1 helper at run time. The previous runtime call
    undef-crashed under Common Test because the eqwalizer_support
    git_subdir checkout lands the helper's beam at a double-nested path that
    rebar3's ct provider never adds to the code path. The suites are green again
    with no skips, coverage stays at 100% on every touched module, and no
    runtime/published module was edited for this change.
  • eqwalizer_support is RETAINED as the eqwalizer toolchain dependency
    (not dropped).
    It is the required git_subdir dep every eqwalizer project
    declares per the official getting-started instructions; it anchors the
    OTP/stdlib type overlays elp eqwalize-all needs. Removing it was tried and
    rejected: without it, elp eqwalize-all cannot narrow stdlib results and
    reports incompatible_types against term() across every src module
    (a locally-reproduced 174-error degrade of an otherwise-green type gate). It
    is now justified solely as the build-time type-checker anchor — it is no
    longer on the test suites' runtime code path (see the previous entry), so its
    git_subdir double-nesting no longer causes {undef, dynamic_cast}.
  • bin/quality-gate.sh --full now hard-requires elp. A new
    require_elp step records a real FAIL (counted in the gate total, forcing a
    non-zero exit) when elp is not found, instead of letting the eqwalizer and
    elp lint steps silently SKIP-to-green. In --full those two steps now run
    strictly (a missing elp is a FAIL, not a SKIP); only the cheap --fast
    lane keeps the soft-skip with an install hint. This closes the SKIP-passes
    hole so a machine without elp can no longer pass the strict gate.
  • Repository converted to a rebar3 umbrella. The runtime library now
    lives in apps/erli18n/ (its src/, test/, and erli18n.app.src moved
    verbatim) instead of the repo root. This is a layout-only change with no
    runtime module edits
    : the published erli18n package's modules and public
    API are byte-for-byte unchanged. The Hex publish path is
    cd apps/erli18n && rebar3 hex publish package (each package is published
    from its own self-contained app directory, not via --app from the umbrella
    root). Contributors should note that the lib's runtime dependency
    (telemetry ~> 1.3), compile options, doc config, and its own
    {project_plugins, [rebar3_hex, rebar3_ex_doc]} now live in
    apps/erli18n/rebar.config; the root rebar.config carries only
    umbrella-wide tooling (dev/test plugins, the test profile, and the
    dialyzer/xref/hank/erlfmt policy).
  • Documentation swept to the two-package umbrella reality. README.md,
    CONTRIBUTING.md, the plugin's apps/rebar3_erli18n/README.md, and
    .github/workflows/release.yml now describe the shipped layout consistently:
    the umbrella project tree (apps/erli18n/, apps/rebar3_erli18n/,
    examples/erli18n_demo/); the Erlang-native rebar3 erli18n extractor as a
    separate, opt-in {plugins, [rebar3_erli18n]} package depending on the
    library in the plugin → lib direction; the proven cross-package
    _checkouts/{erli18n, rebar3_erli18n} load-path requirement; the scoped xref
    host-seam ignore and why (the rebar3 host modules are escript-internal, not a
    fetchable Hex dep); and the --full gate's hard elp requirement (soft-skip
    only in --fast). The release workflow publishes both packages from
    per-package prefixed tags (erli18n-vX.Y.Z, rebar3_erli18n-vX.Y.Z),
    erli18n first. Prose is en-US throughout. Documentation only; no runtime or
    published-module edits.

Added

  • Catalog tooling promoted to a separate publish-ready plugin package,
    rebar3_erli18n
    (apps/rebar3_erli18n/). The four catalog providers
    (rebar3 erli18n extract|merge|check|report) now ship as their own rebar3
    plugin Hex package rather than being bundled into the runtime library — the
    dominant rebar3 idiom for a tool with a real runtime consumer (the
    gpb/rebar3_gpb_plugin pattern). The plugin declares a real dependency on
    this library ({deps, [{erli18n, "~> 0.5"}]}) and reuses the published PO
    API across that boundary. Consumers opt in with
    {plugins, [rebar3_erli18n]}. The plugin carries its own
    README/CHANGELOG/LICENSE (Apache-2.0) and is published as a separate
    Hex package, after this library, against {erli18n, "~> 0.5"}. See
    apps/rebar3_erli18n/CHANGELOG.md.
  • Real downstream-consumer example, examples/erli18n_demo/. A separate
    rebar3 project (deliberately under examples/, NOT apps/, so the umbrella
    does not auto-discover it) that consumes BOTH umbrella packages exactly as a
    real downstream app would: its rebar.config declares
    {deps, [{erli18n, "~> 0.5"}]} and {plugins, [rebar3_erli18n]}, and its
    production modules (erli18n_demo_greeting, erli18n_demo_errors,
    erli18n_demo_accounts) contain genuine compile-time-literal
    erli18n:gettext/ngettext/pgettextf/npgettext/dgettext/gettextf
    call sites
    across the default, errors, and accounts domains. Running
    rebar3 erli18n extract → merge --locale pt_BR → check against it produces
    the committed baseline .pot templates and the translated pt_BR .po
    catalogs under examples/erli18n_demo/priv/gettext/, which the
    rebar3 erli18n check gate compares against (it FAILS on drift, PASSES in
    sync — the non-vacuous CI gate the library repo itself cannot host, because
    the facade never calls itself and extraction there yields zero .pot). The
    example also documents the dynamic-msgid caveat: dynamic_label/1 calls
    erli18n:gettext/1 with a runtime (non-literal) key, so it is NOT extracted
    and never causes a false drift failure, while still translating at runtime.
    Because the example is developed in-tree (against the umbrella sources, not a
    Hex fetch) and rebar3 has no native {path, ...} resource, it surfaces both
    in-repo apps through rebar3's documented _checkouts/ override
    (_checkouts/erli18n, _checkouts/rebar3_erli18n); those links and the
    example's _build/ are git-ignored recreatable artifacts, while the baseline
    catalogs are tracked.
  • Executed proof of the cross-package plugin → lib load path. An
    ERLI18N_DIAG_LOADPATH-gated diagnostic in rebar3_erli18n_common logs the
    loaded location of erli18n_po at provider-run time. Driven from
    examples/erli18n_demo/, the extract → merge → check run confirms
    code:which(erli18n_po) resolves under the consumer's
    _build/default/checkouts/erli18n/ebin/erli18n_po.beam — the unpublished
    runtime library is reached through the consumer's checkout (not a Hex fetch)
    across the {deps, [erli18n]} boundary, with no undef erli18n_po:dump/1.
    So the runtime erli18n_po:escape_string/1 reuse is reachable cross-package,
    and the relocated rebar3 erli18n check gate can meaningfully pass/fail
    rather than undef-crash. The contingency private escaper/dumper was not
    vendored. No runtime/published module behavior changed in this step.
  • The translation-freshness gate now runs inside the consumer example.
    bin/quality-gate.sh runs rebar3 erli18n check from inside
    examples/erli18n_demo/ (via a new run_step_in <dir> <name> -- <cmd...>
    helper that executes in a ( cd <dir> && … ) subshell, so the gate's own
    working directory is never mutated and a failure is accounted exactly like
    any other step). The check re-extracts the demo's real erli18n:gettext
    call sites and FAILS the build on drift against the committed catalogs, in
    the same load context where erli18n_po is on the plugin path through the
    demo's _checkouts/erli18n. Before the step, ensure_demo_checkouts
    idempotently (re)creates both examples/erli18n_demo/_checkouts/erli18n and
    …/_checkouts/rebar3_erli18n so the git-ignored links are always present.
    This replaces the previous in-library invocation, which was vacuous (the
    facade never calls itself, so extraction in the library repo yields zero
    .pot and the check protected nothing). A deliberate negative-drift
    integration test (providers_SUITE:check_drift_cycle_in_load_context)
    encodes the FAIL-on-drift → PASS-when-fresh cycle through the real provider
    entry points and asserts up front that code:which(erli18n_po) is reachable,
    so a cross-package load-path regression fails the test explicitly instead of
    masquerading as drift. No runtime/published module behavior changed in this
    step.
  • PO-metadata edge-case assertions in po_meta_SUITE (the plugin's
    metadata serializer suite). Four serialize-side cases were added to pin
    contracts the existing golden tests did not assert: an explicit empty-binary
    context emits msgctxt "" while the absent-context undefined omits the
    line (the no-context invariant, on the write side); an obsolete PLURAL entry
    #~ -prefixes every line of its multi-line block (msgid, msgid_plural,
    each msgstr[N]); an obsolete entry with a translator comment keeps the
    # comment un-prefixed while #~ -prefixing the body (including
    #~ msgctxt); and a plural entry carrying both an #. extracted comment and
    #: references emits them in canonical GNU order before the plural block.
    Coverage of rebar3_erli18n_po_meta stays at 100%. These edge cases were
    mined from the discarded runtime preserve-mode CT work (see the design note
    below); no runtime/published module was edited.

Decided (design)

  • The runtime erli18n_po:parse/1 preserve-mode WIP is abandoned. An
    earlier in-progress design added an opt-in erli18n_po:parse(Po, #{preserve => true}) mode that retained the full GNU metadata channel (translator and
    extracted comments, #: references, #, flags incl. fuzzy, #|
    previous-msgid, and #~ obsolete entries) on the runtime READ/parse side.
    That mode is deliberately NOT shipped: erli18n_po:parse/1 stays lossy by
    design, collapsing all metadata and dropping fuzzy (PSD-001) and obsolete
    (PSD-007) entries, so the runtime API surface stays minimal. PO metadata is
    structurally owned by the plugin's WRITE/serialize side
    (rebar3_erli18n_po_meta), not the runtime parser. This matches the Gettext
    merge contract the plugin implements: #: references and comments are
    authoritative from the freshly extracted POT, and only msgstr is preserved
    from the old PO, so the runtime read side has no need to round-trip the
    metadata channel. The discarded suite's edge-case assertions worth keeping
    (the PSD-001/PSD-007 fuzzy/obsolete and no-context cases) were ported to the
    plugin-side po_meta_SUITE instead (see above). No runtime/published module
    was edited for this decision.

Removed

  • The host-beam extraction workaround that previously let the in-repo
    plugin satisfy the root project's xref as a project app (a vendored escript
    that extracted the rebar3 host modules into a generated beam directory, plus
    the matching root rebar.config project-app-dirs / extra-paths wiring). As a
    normal rebar3 plugin, rebar3_erli18n receives the rebar3 host modules at
    plugin-load time; xref resolution for the host seam is now a scoped
    -ignore_xref/{xref_ignores} confined to the eight host {M, F, A} edges.
    No runtime module edits.