Skip to content

v0.4.0

Choose a tag to compare

View all tags
@earthsciml-bot earthsciml-bot released this 04 Nov 18:18
· 14 commits to main since this release
v0.4.0
4093ace
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

EarthSciASTDiff v0.4.0

Diff since v0.3.0

EarthSciASTDiff v0.4.0

A structure-only mode for jacobian_bands, derived documents that keep
data-fed parameters as inputs, and much faster band computation on large
stencil operators. Every binding now also admits EarthSciAST 0.9. A minor
release: the Rust SysView gains a public field (see Breaking changes), and
the derivative of Pre changes (see Fixed).

Breaking changes

  • Rust: SysView has a new public field
    data_sources: Option<&IndexMap<String, DataSource>>.     Code that builds a
    SysView with a struct literal must add it (None, or the document's
    data sources); sysview fills it from the document, and
    sysview_flat(&flat)?.with_data_sources(..) sets it for a flattened
    system.
  • Pre(a) in a right-hand side is a (spec §3.2). The rule table gave
    Pre the derivative 0, so jacobian_bands dropped a dependence that
    EarthSciAST's right-hand side has. d[Pre(a)] = d[a] now, and the
    inliner lowers Pre(a) to a before differentiation, so no band
    coefficient, evaluation document or product document carries a Pre.
    Bands of a document that reads a state through Pre gain those entries.

New

  • Structure-only mode of jacobian_bands (spec §2.1). Regions are
    clipped and sites found as in the full mode, but no derivative is taken
    and no coefficient simplified; bands merge on structure alone. The mode
    refuses exactly what the full mode refuses (an op with no §3 rule, a
    windowed extremum over more than one contracted index).
    • Julia and Python: jacobian_bands(...; coefficients = false) and
      jacobian_pattern(...; coefficients = false); each band.coef is
      nothing / None, and serialize_band / parse_band omit and accept
      a missing coef.
    • Rust: jacobian_structure, jacobian_structure_view,
      StructureEntry / StructureBand and serialize_structure_band.
  • Derived documents keep data-fed parameters as inputs (spec §8.1,
    §9.1). The evaluation document and the product documents used to declare
    a parameter fed from a data source by its shape and default only, so
    EarthSciAST built it as a constant. Each data-fed parameter now keeps its
    data rules, and the document carries the data_sources entries they
    name, so EarthSciAST builds the parameter as a forcing slot of
    rhs_program. A flattened system has no data_sources, so pass the
    flattened document's: a data_sources keyword in Julia and Python
    (sysview, evaluation_document, the product documents and Julia's
    jacobian_document), SysView::with_data_sources in Rust. A named source
    that cannot be found is a BandError. An observed that only a kept rule
    reads (an expression unit_conversion, say) is kept.
  • Summand groups are clipped on their own axes (spec §2.2). A body that
    is a sum of terms cut along different dimensions (a dimension-split flux
    divergence) used to be clipped into the product of their regions,
    7 × 7 × 7 = 343 bodies for a 3-D PPM transport term. Each group of
    summands that share a cut dimension is now clipped on its own
    (7 + 7 + 7), summands cut along no dimension form a group of their own,
    and a - b is split into [a, -b]. The band set and values are
    unchanged; the band order and grouping of such bodies change (see
    Conformance).
  • scripts/bench-jacobian-bands.jl --stages doc1.esm doc2.esm … times load,
    flatten, sysview and jacobian_bands per document.

Fixed

  • A faq node's operand list reads nothing (spec §3.5). The site walk
    descended into a faq node's args, the serializer's copy of the arrays
    its body reads, so a whole array named there was refused as a
    "whole-array reference inside a faq body".
  • makearray region values in the inliner (Rust and Python). A
    full-rank region value over a pinned region (a 3-D field's boundary face,
    esm-spec §4.3.2) was refused with makearray value rank 3 ≠ region free rank 2. Every binding now reads a region value by position, at
    idx − lo + origin, and takes its rank from the value itself, so a value
    whose range does not start where its region does no longer reads the
    wrong cell, and a full-rank array variable or makearray value no longer
    raises an arity error (or, in Rust, panics).
  • Julia: jacobian_document of a flattened system declares the
    data_sources its data-fed parameters name (a data_sources keyword);
    it used to fail data_source_undefined.
  • Rust: non-finite literals are keyed by their bits in the structural
    memo, so x * -Inf no longer receives the simplification of x * Inf,
    and bands whose coefficients differ only in such a literal are not
    merged.
  • Structure-only bands merge the same in every binding: Python and Rust
    now coalesce a one-hot column that clipping splits into regions, as the
    Julia reference does, and through a recurrence they merge bands by
    (equation, column) rather than (row, column).

Performance

On ReSEACT's lowered advection term (one faq of 4.2e5 nodes),
jacobian_bands took about 1400 s in Julia. All three bindings now keep one
occurrence index and one derivative memo per band computation, shared
across the regions of a body; memoize the coefficient simplifier on
structural id; group bands for merging by structural id instead of
re-serializing each coefficient; and intern every coefficient and guard so
equal subtrees are one object (on that term, retained coefficient memory
drops from 209 MiB to 2.1 MiB in Julia, and from 805,377 to 5,899 distinct
operator nodes in Rust). Julia releases each clipped region's memo entries
when the region is done; Rust keys its memos weakly. Output is byte-identical
apart from the changes listed above.

Dependencies

  • Every binding admits EarthSciAST 0.9: Julia 0.6, 0.7, 0.8, 0.9, Rust
    >=0.5, <0.10, Python >=0.4,<0.10. The floors are unchanged.

Conformance

  • New fixtures, each with band goldens checked in Julia, Python and Rust:
    pre_operator (with block, evaluation-document, value, product,
    second-order and scatter goldens), faq_operand_list, data_fed_flat,
    split_sum_2d, split_diff_2d and the structure-only onehot_clipped.
  • New structure-only goldens <name>.structure.bands.json for every fixture
    with a state band golden (those of the two recurrence fixtures are written
    by the Rust binding).
  • Changed goldens: coupled_chem_advection (band order of its x- and
    y-split boundary bands, now group by group; same bands and values) and
    the forcing_bracket derived documents (they now carry its data-fed
    parameters' rules and data sources).

Breaking changes

None marked in the commit messages (type!: or BREAKING CHANGE:); see the changelog below.

Changelog

  • chore(release): v0.4.0 (e2d0916)
  • chore(deps): admit EarthSciAST 0.9 in every binding (4cf0960)
  • style(julia): drop the trailing blank line Runic rejects in hashcons.jl (8bc7a92)
  • test(goldens): structure-only goldens for faq_operand_list and pre_operator (2663c87)
  • perf(rust): weak structural memo, exact non-finite keys, lazy rebuilds (9870aca)
  • fix(bindings): read makearray region values by position, at their own rank (d8e9b24)
  • perf(rust): key a bare variable's occurrence set once in OccIndex (81ee446)
  • perf(python): memoize the occurrence short-circuit in drule_check (9b1a1ed)
  • perf(julia): release each clipped region's index and memo entries when done (b8de171)
  • fix(bands): structure-only entries merge on structure alone in every binding (f2b1f91)
  • perf(bands): intern every band list in one bottom-up pass (Julia, Rust) (9433a13)
  • fix(bands): split a - b summands, give uncut summands their own group (spec §2.2) (d0826e9)
  • fix(bindings): declare data sources in the flat jacobian_document and keep observed a carried rule reads (d1875f3)
  • fix(rules): lower Pre to its operand before differentiation (spec §3.2) (191f14c)
  • style(julia): runic formatting of drule_check (b8ed436)
  • fix(bands): structure-only mode refuses what the full mode refuses (f4d2dd0)
  • perf(rust): weak-keyed occurrence index, shared second-order memos (18ea36f)
  • perf(bands): one object per distinct coefficient subtree (Julia, Rust) (060ff88)
  • split_sum_2d fixture: the §2.2 band form, byte-compared in all three bindings (21c6eb1)
  • feat(bands): clip each summand group on its own axes (spec §2.2) (03ae8bb)
  • structure-only goldens of the recurrence fixtures (written by Rust), api-surface entries, lint (4eed1c3)
  • spec §2.1: structure-only entries through a recurrence (716577f)
  • feat(bands): structure-only mode of jacobian_bands (spec §2.1) (b8ff340)
  • spec §2.1: structure-only entries, and how their pattern relates to the full mode's (6688c7a)
  • perf(rust): one occurrence index and derivative memo per band computation (5e6fba1)
  • fix(bindings): inline a full-rank faq value over a pinned makearray region (86b11dd)
  • bench: --stages mode times load, flatten, sysview and jacobian_bands per document (bb8dfe0)
  • feat(bindings): carry data-fed parameters' rules and data sources in derived documents (f04ffc6)
  • perf(julia): one occurrence index and derivative memo per band computation (b579cbd)
  • fix(rules): Pre is the identity in a right-hand side (spec §3.2) (60366bd)
  • fix(bands): a faq node's operand list reads nothing (spec §3.5) (77e5950)

Merged pull requests:

Installation

using Pkg; Pkg.add("EarthSciASTDiff")   # Julia
pip install earthsci-astdiff             # Python

Contributors

ctessum-claude
Assets 2
Loading