Skip to content

catena v1.5.0

Latest

Choose a tag to compare

@github-actions github-actions released this 10 Oct 15:21
· 5 commits to main since this release

The largest release since the API froze: the places a chain had to stop
are closed, and three packages now sit beside the core. Nothing existing
changes behaviour.

This reverses three earlier decisions, on purpose. The original design
ruled out parallel operators ("a different library"), capped the
operator count, and kept everything in one package. Concurrency is still
a different package — the core starts no goroutine it did not start
before — but it is no longer someone else's problem.

Added

  • Seq.Then applies a function to the sequence, so a function that
    takes one sits inside the chain instead of wrapping it. Every
    one-argument package function fits, with its types inferred:
    s.Then(catena.Distinct).Then(catena.Sorted).Take(10). It is also how
    the stages in catena/parallel attach.

  • Ordered, chainable grouping. Seq.FoldedBy and catena.GroupedBy
    are FoldBy and GroupBy that yield a Seq2 of pairs, keys in the
    order they were first seen, instead of returning a map: the result is
    deterministic and the chain keeps going. GroupedBy is a package
    function because Seq2[K, []T] leads back to Seq[[]T], the same
    instantiation cycle that makes Chunked one.

  • Joins that drop nothing. Seq.LeftJoinBy keeps a left element with
    no match and tells combine so; Seq.GroupJoinBy yields one result per
    left element with all of its matches together.

  • Five ideas from lo, in catena's shape.

    • Seq.FirstOr and Seq.LastOr return a fallback when the sequence is
      empty, so a lookup can sit inside an expression.
    • IsSorted, Seq.IsSortedBy and Seq.IsSortedWith check order in one
      pass with no buffer, stopping at the first pair out of order.
    • Duplicates and Seq.DuplicatesBy yield each repeated value once, at
      its second occurrence.
    • Seq.Sample draws n elements uniformly at random in O(n) memory by
      reservoir sampling — 803 KB → 456 B against collecting 100k elements
      to shuffle them, and 4.4× faster. A seeded generator makes the choice
      repeatable.
  • Try.UntilErr stops at the first error, as Collect does, but as
    a sequence: the chain continues with the full Seq API and the error
    is left where you asked, to be checked once the pipeline has run. It is
    the policy exit Try was missing — Ignore keeps going and Must
    panics — and it is how a fallible source feeds a stage that takes a
    plain sequence, the concurrent ones included.

  • catena/parallel: concurrency. Six operators, each a stage for Then:

    • Map, MapUnordered and ForEach run a callback up to n at a
      time. The source stays on the consumer's goroutine; only the callback
      runs elsewhere. A failed call is a Try element, so Collect,
      CollectAll and Ignore choose the policy as they always have.
    • Buffer reads ahead, ChunkedWithin batches by size or by time, and
      Merge interleaves several sequences by arrival; these run their
      sources on goroutines of their own.

    Every one joins the goroutines it started before its iteration
    returns — on a normal end, an early break, a cancelled context and a
    panic alike. That is tested under testing/synctest, in virtual time,
    against callbacks that are deliberately slow to stop.

  • catena/source: producers that ship. Lines for the lines of a
    reader, of any length; Paginate for a paginated API, fetching a page
    only once the one before it has been yielded in full; and
    sqlsource.Rows for a query, in a package of its own so the others do
    not link database/sql. Each opens its resource inside the iteration
    and releases it on every exit — the pattern the error-handling guide
    taught, now code.

  • catena/lint: a linter, as a separate module so the library's own
    still depends on nothing. catenalint runs as a go vet tool and
    reports the pipelines that are correct but wasteful —
    Sorted(s).Take(n) where TopN holds n elements,
    Sorted(s).First() where Min holds one, Distinct(Sorted(s)) where
    Dedupe holds none, a GroupBy used only to count — and the lazy
    sequence that is built and never consumed. It offers a fix only where
    the replacement returns the same values of the same type. It carries
    the library's version, under its own prefix: lint/v1.5.0.

  • Guides for concurrency and
    linting, and
    a lookup from lo, Kotlin and LINQ names to
    catena's, every entry of which a test checks against the library.
    Reference pages for the two new packages, and concurrency and
    sources examples.

Documentation

  • The v1.4.x series is archived at /1-4/, per the rule that a minor
    series is snapshotted when the next minor ships.

Changed

  • Terminals and grouping operators are faster, some of them much.
    Every operator consumed its source through an adapter that made a nil
    sequence behave as an empty one, and that made every source an opaque
    function value. The call now goes through a method small enough to
    inline, so a source built in the calling function is one the compiler
    can see through. Measured interleaved against 1.4, over 100k elements:
    Sum and Contains 71µs → 24µs, the speed of a hand-written loop;
    FoldBy 16% faster; GroupBy and JoinBy 10–13%; single-stage Map
    5%. Multi-stage pipelines are unchanged. No API or behaviour change.

  • The published hand-loop baselines were too slow, and are corrected.
    They were written inline in the benchmark body, where nothing may be
    optimised away, which made a plain loop look up to four times slower
    than it is. Measured as functions of their own, the hand-written
    4-stage pipeline takes 25µs, not 39µs — so a 4-stage catena pipeline is
    about 10× a hand loop on trivial arithmetic, not 6.6× as the README and
    the performance guide said. The library did not get slower; the earlier
    comparison flattered it.

  • The 100% coverage gate now covers parallel, source and sqlsource as
    well as the core package, and the linter's module is gated to 100%
    separately. CI also runs the linter over the examples, which must come
    back clean.

  • The selection example's leaderboard sorted every player to show four.
    The linter found it; it uses TopNBy now, with identical output.

  • The join example rolls up with FoldedBy in place of collecting map
    keys and sorting them, and shows the two new joins.

  • The operator reference is generated for the sub-packages too, and the
    README's "exactly one goroutine" is now stated of the core package,
    which is where it is true.