v0.3.0

A span can now say how far its work has got, and the displays draw it.
Work in items, bytes or a percentage, reported by a library, a copy or a
pool of targets, is rolled up the tree and drawn as an amount, a share,
a rate and the time left. The displays can draw in five themes the
program picks. The live clocks tick in tenths of a second. A new package,
progress/cliprogress, holds the glue between a command line and the
progress packages, which clusterctl and sind each carried a copy of until
now.

The text the live tree and the counter draw with the zero Theme
changes for every program, including one that reports no work. That is a
breaking change, which v0 allows in a minor release, so the minor version
goes up. The new API and the new package progress/cliprogress would
make this a minor release in any case.

Changes a program or its tests may notice

progress/display:

• The live clocks read tenths of a second, cut rather than rounded
  (decision 18). The command's time on the counter's line, on the tree's
  command row and on the counter line the tree falls back to on a small
  terminal reads 0:41.3 or 1:02:03.4, where it read 0:41 or
  1:02:03. How long a running target, step, call or wait has run reads
  4.2s or 3m12.4s, against a request's bound 1.0s/10m, and from an
  hour on 1h02m, as before. A test that compares frames sees 0:41.0
  and 4.0s where it saw 0:41 and 4s. What is not a live clock stays
  as it was: the [0:03] in front of a plain line, how long a span that
  has ended took (✓ powering on 2.0s, done in 25s, the summary), a
  pause's countdown (20s left, rounded up) and request bounds (/10m).
• The tree lists running targets by the tenth of a second each started
  in, counted from when the tree was made on the monotonic clock, those
  that started in the same tenth in the order they were queued (decision
  18). It used to sort them by the whole seconds each had run at the time
  of the frame. Targets that start within the same second can now be
  listed in another order: four queued exe1 to exe4 that a pool starts in
  the order exe4, exe2, exe1, exe3, 0.20 s, 0.41 s, 0.45 s and 0.55 s in,
  are listed exe4, exe1, exe2, exe3 (exe1 and exe2 started in the same
  tenth, so they keep their queue order), where v0.2.0, at a frame 2.0 s
  in, listed them exe1, exe2, exe3, exe4. A row no longer changes places
  from one frame to the next.
• While anything runs, every frame differs from the one before, so the
  tree and the counter write to the terminal up to ten times a second
  rather than about once: about 12 KB a second for a region of 12 rows
  of 100 columns. A test that drives a display from a ticker on the real
  clock sees a line at every tick, 0:01.0 and then 0:01.1, where
  v0.2.0 drew 0:01 once. A frame drawn again within the same tenth still
  writes nothing.
• A clock that steps back reads 0:00.0 rather than a negative time such
  as -60:00.
• Nothing else changes in what a program that reports no work draws with
  the zero Theme: plain lines and the summary are byte for byte as in
  v0.2.0. The Terminal cuts a row that holds colour sequences by the
  columns it shows, but cuts a row without one, which is every row drawn
  with the zero Theme, with termtext.Truncate as before.

progress:

• The event log stays at version 1 and gains the type advance and the
  keys amount, size, unit and leftOut (decision 23). It writes an
  advance of a span at most once a second by default, so seq is no
  longer gapless: the advances it leaves out leave gaps, which leftOut
  counts. A reader of the log that checks seq for gaps, or refuses an
  unknown type, key or unit, must allow these before the programs it reads
  report work; a program can set a negative LogOptions.AdvanceEvery to
  write every advance. The log of a program that reports no work is byte
  for byte what v0.2.0 wrote. Event.Seq on the Bus still has no gaps.
• BusOptions.Now must be safe for concurrent use once a span reports
  work: Advance and SetAmount read it on the caller's goroutine,
  outside the Bus's lock. A test clock moved by hand without a mutex of
  its own races under -race as soon as a span calls Advance,
  SetAmount, CountWriter or CountReader.
• An advance that comes too soon after the one before is sent later from
  a timer's goroutine, still under the Bus's lock and in Seq order, so a
  sink's Handle may run on a goroutine the program did not start, up to
  100 ms after the call. Spans that report no work start no timers.

progress/progresstest:

• Check enforces a new rule 7, the work of a span: an advance names a
  span that is running and changes its Amount; two advances of a span
  come at least 100 ms apart by their times; Amount and Size are
  never negative; a span in Percent has a Size of 100; and a span's
  Unit never changes once given. Decision 15 makes a new rule a breaking
  change. This one applies only to the event and fields this release
  adds, which no source of events sent before, so a stream that passed
  Check under v0.2.0 passes it still (decision 23). A hand-made event of
  type 8, now TypeAdvance, is no longer reported as having no valid
  type; rule 7 checks it instead, and refuses one that does not change its
  span's Amount or names a span that is not running.
• A Capture, and the one behind Watch, keeps every advance the Bus
  sends, up to one a span every 100 ms; a test that counts events or
  compares their types sees them once its spans report work.
• Tree draws a span's work after its message and before its flags, as
  amount=2147483648 size=2147483648 unit=bytes, each left out when zero,
  from the span's newest event, advances included. A tree of spans without
  work is unchanged, byte for byte.

fanout, progress and progress/display:

• fanout.Item gains the fields Unit and Size, progress.Fields
  gains Amount, Size and Unit, progress.LogOptions gains
  AdvanceEvery, and display.TreeOptions, CounterOptions and
  PlainOptions gain Theme, inserted among their fields. Code that
  builds any of these with an unkeyed composite literal no longer
  compiles.

New

progress (decision 23):

• Work(unit, size), an option of Start, Update and End, gives a
  span's work in a Unit, Items, Bytes or Percent, out of a size; a
  size of 0 or less is not known, and the size of work in Percent is
  always 100. A unit once given stays: a later Work changes the size
  alone.
• Span.Advance adds to the amount and Span.SetAmount sets it, for
  work that reports how far it has got; the amount may go down. Both do
  nothing on a nil span or one that is not running, and cost an atomic
  operation and a clock read; the Bus's lock is taken at most once every
  100 ms a span. A span given no Work that advances counts Items.
• TypeAdvance: the Bus sends a span's newest amount at most every
  100 ms, by its clock. The first goes out at once, the newest is never
  lost, and the span's End carries the final amount. Fields, and so
  Event, gain Amount, Size and Unit: the span's own work, never
  that of the spans below.
• CountWriter and CountReader advance the innermost span in a context
  by the bytes each write or read moved. They return the writer or reader
  itself, at no cost, when the context carries no span. They hide the
  io.ReaderFrom of the writer and the io.WriterTo of the reader they
  wrap, so io.Copy cannot use those fast paths through them; the other
  side's still apply.
• Meter, Reading and Bound roll the work of a Bus's spans up the
  tree for a sink. Amounts add within one unit. A span's share done comes
  from its own size, else from the targets of a Fold step, or of a batch
  whose Total is known, once one of them is bounded, short of done until
  their count is complete, else from the sizes of the work below. A
  Reading also gives the rate over the last five seconds and, once a
  span has run two seconds, the time left over the same window, and says
  when the work has stalled for five seconds. The zero Meter is ready to
  use, and not safe for concurrent use. Reading may gain fields, and
  Unit values, in a minor release.
• LogOptions.AdvanceEvery: the least time between two advances of a
  span the log writes. 0 means a second, and a negative value writes them
  all.
• Without a Bus, the new calls allocate nothing, which tests hold them to.

progress/display:

• Themes (decisions 19 and 20). Theme is a comparable value whose zero
  value is no theme and draws what v0.2.0 drew. There are five themes,
  Classic, Aurora, Ember, Neon and Tide. Themes lists them,
  ParseTheme reads a name in any case, none for the zero Theme, and
  String, MarshalText and UnmarshalText let a Theme back a
  --theme flag through flag.TextVar. Theme.In draws a theme in its
  256 colours, the terminal's own 16, or none, its art alone, for
  NO_COLOR; Theme.Colours says which. The kit reads no environment:
  the program decides from its flags, NO_COLOR, TERM, COLORTERM and
  whether standard error is a terminal.
• TreeOptions, CounterOptions and PlainOptions take a Theme, and
  Summary a Theme and ASCII. The tree and the counter draw a theme's
  marks, and, where the theme has them, a spinner on each running target
  and in front of the counter's line, a bar on counted steps that gives
  way when the row would not fit, and a guide at the innermost indent;
  Classic's running mark stands still, and it draws no bar or guide.
  Plain lines take a theme's colours and one mark after the time, and the
  summary one mark in front, and both keep their words, so a log reads the
  same once the colours and marks are taken out. With a theme, ASCII
  keeps its colours and draws ASCII marks, a [###...] bar, and no
  spinn...

v0.2.0

The live tree costs a fraction of what it did on steps of tens of
thousands of targets. It handles every event and draws every frame under
a lock that the Bus waits for, and every worker that reports an event
waits for the Bus, so on such steps the tree slowed the work itself.

Changes a program or its tests may notice

progress/display:

• The live tree folds a set of more than 256 targets again at most once a
  second, rather than at every frame, so such a row may leave out the
  targets that ended in the last second. The count in the step's row, and
  the line a step leaves when it ends, stay exact. A test that draws
  frames less than a second apart over more than 256 targets sees the
  earlier fold (decision 16). This changes the text the tree draws, so the
  minor version goes up.

Faster

On one machine, best of three runs:

	v0.1.0	v0.2.0
Ending 16,000 targets	489 ms	98 ms
A run that draws a frame every 100 events, 16,000 targets	1.64 s	0.29 s
The same, 30,000 targets	6.0 s	0.83 s
A frame while 16,000 targets run	5.8 ms	0.72 ms
A frame while targets end, after 16,000 ended well	3.8 ms	13 µs

• A target that ends leaves its step in constant time, rather than with a
  pass over the step's other targets.
• A failure finds its row by its text, and a batch that ends adds its
  targets to its step's in place rather than copying the step's set.
• A frame builds only the rows of a list that fit the region, rather than
  one for every running target.
• The tree counts each event once, rather than a second time for the
  counter line it draws on a terminal too small for it.

The Go API, the counter, the plain lines and the event log are unchanged.

v0.1.0

The first release of go-clikit, the runtime that GSI-HPC's command-line
tools share. A library reports what it does as spans carried in a
context.Context, at no cost when no command is listening, and runs work
on many targets in bounded pools; the command decides how that is shown:
a live tree or a counter on a terminal, plain lines for a CI log, or a
JSONL event log for programs.

The packages were written for clusterctl, where they shipped in v0.4.0
under internal/, and moved here with their history.

Packages

• termtext: escaping untrusted text for a terminal, and the columns text
  takes there.
• progress: spans in a context.Context, the Bus that orders their
  events, the JSONL event log and the W3C trace context.
• progress/display: the live tree, the counter, plain lines, the summary
  line, and the Terminal they share with the command's own output.
• progress/progresstest: capturing the events a command reports,
  checking them against the contract every source of events keeps, and
  Screen, a terminal for tests.
• fanout: Each, Map and Batches, bounded pools that report their
  items as progress.

API reference: https://pkg.go.dev/github.com/GSI-HPC/go-clikit

Requirements

• Go 1.26 or newer. The module requires the oldest Go release the Go
  project still supports, and follows it up after each Go release.
• github.com/GSI-HPC/go-nodeset v1.0.0 and golang.org/x/text v0.42.0,
  and nothing else.
• Licensed under Apache-2.0.

Stability

The module is at v0: a minor release may change the API, and its notes say
how. Beyond the Go API, four things are interfaces, and changing any of
them is a breaking change:

• the six span kinds: command, step, batch, target, call and wait;
• the JSONL event log, version 1, described in doc/event-log.md: keys
  may be added, never renamed or removed;
• the text the displays draw, which programs compare in their tests,
  but for the characters termtext escapes, which may grow in a minor
  release (decision 14);
• the rules progresstest.Check enforces: a new or stricter rule is a
  breaking change (decision 15).

Coming from clusterctl v0.4.0

Rewrite the imports:

clusterctl v0.4.0	go-clikit v0.1.0
github.com/GSI-HPC/clusterctl/internal/termtext	github.com/GSI-HPC/go-clikit/termtext
github.com/GSI-HPC/clusterctl/internal/progress	github.com/GSI-HPC/go-clikit/progress
github.com/GSI-HPC/clusterctl/internal/progress/display	github.com/GSI-HPC/go-clikit/progress/display
github.com/GSI-HPC/clusterctl/internal/progress/progresstest	github.com/GSI-HPC/go-clikit/progress/progresstest
github.com/GSI-HPC/clusterctl/internal/clikit/fanout	github.com/GSI-HPC/go-clikit/fanout
github.com/GSI-HPC/clusterctl/nodeset	github.com/GSI-HPC/go-nodeset

fanout.Batches and fanout.Batch take go-nodeset's *nodeset.NodeSet,
so a program that passes node sets to them moves to go-nodeset as well.

The event log writes the same version 1 lines, and the span kinds are
the same. The API was reviewed before this release for what an internal
package can afford and a public one cannot, and reshaped:

clusterctl v0.4.0	go-clikit v0.1.0
termtext.EscapeCell, termtext.EscapeText	termtext.Escape, termtext.EscapeLines; the escaped bytes are the same
termtext.RuneWidth(r)	unexported; termtext.Width(string(r))
progress.Options	progress.BusOptions
Options.Trace, Parent, TraceFlags, TraceState	BusOptions.Trace, the TraceContext that ParseTraceContext returns
TraceSink.Begin(TraceContext)	TraceSink.Begin(Run); the log takes the program's name from the Bus
LogOptions.FlushEvery, LogOptions.CloseWait, Log.WantsLines	removed
Count.Targets	Count.Started
fanout.Skip, fanout.IsSkipped(err)	progress.Skip, errors.Is(err, progress.ErrSkipped); Span.End ends a span skipped for it
fanout.Options	fanout.MapOptions
fanout.DefaultMax	fanout.DefaultLimit
outcomes := fanout.Map(…)	outcomes, err := fanout.Map(…), the error the step ended with
Describe func(T) (node, host, role string)	Describe func(T) fanout.Item
Summarize func(n int, names []string, errs []error, interrupted bool) error, fanout.Failure(noun, n, names, errs, interrupted)	Summarize func(fanout.Summary) error, fanout.Failure(noun, s); Summary.Canceled
BatchOptions.After	removed; test the pause in a testing/synctest bubble
Batch.Ran	Batch.Started
PanicError.Value string	PanicError.Value any, and PanicError.Target
display.NewTerminal(w, size) and the fields PanicLog, Program, Foreground	display.NewTerminal(w, display.TerminalOptions{…})
progresstest.Checked	removed; progresstest.Watch
ctx, tree := progresstest.Watch(…); tree()	ctx, w := progresstest.Watch(…); w.Finish(), with w.Events() and w.Tree()
progresstest.Option, a func(*progress.Options)	opaque; progresstest.Classify and the new progresstest.Sinks

Added: Bus.Program, Bus.PanicLog and Bus.Classify, which answer for
a nil Bus too; progress.Run; progress.ErrSkipped and progress.Skip;
fanout.Item, fanout.Summary and fanout.Failed; MapOptions.Noun;
display.TerminalOptions; progresstest.Watcher and progresstest.Sinks;
progress.Log.Done.

The behaviour differs where the review asked for it or found bugs, as
listed below.

Changes a program or its tests may notice

termtext:

• Width and Truncate count a character of one column followed by the
  emoji variation selector U+FE0F as two columns, as terminals draw it, so
  a red heart or a keycap measures 2, not 1.
• The soft hyphen U+00AD and the prepended concatenation marks, such as
  U+0600, take one column, not none.
• Characters made wide after Unicode 15.0, such as U+1FAE9, take two
  columns: the widths follow Unicode 18.0, where golang.org/x/text knows
  15.0. U+1171E, a spacing mark since Unicode 16.0, takes one.
• Text holding any of these measures wider, and Truncate cuts it
  shorter.

progress:

• Classify takes a fallback's ClassNone for an error as
  ClassTarget, so a span no longer ends "failed (none)".
• The event log names the program from BusOptions.Program when
  LogOptions.Program is empty.

progress/display:

• Plain counts "1 target" and "480 targets" when PlainOptions.Noun is
  nil, not hosts. A program that names its targets is unchanged.
• Start starts a display once, and never after Close. A Terminal
  carries one display, and a second panics.
• Plain lines and the counter no longer print an empty part for a step
  with no name, as fanout.Map given no Step reports: its lines name
  the span above it, and a plain line with nothing to name it reads
  "start, 2 hosts", not ": start, 2 hosts".
• In the tree, a target's name is read as {} in an error only where it
  stands alone. A name inside a word or an address is left as it is, so a
  target named "e" no longer turns "timeout" into "tim{}out"; names beside
  "-" and "_" are read as before.
• The summary counts the nodes of a batch left out once, not again on top
  of the targets already counted.
• A time from 9.95 s up to 10 s reads "10s", not "10.0s".
• A write of the command's that ends a line and then resets the colours no
  longer leaves the line open, which used to keep the display off the
  terminal and hold the lines written beside it.
• A display that panics takes its region off at once, and writes the lines
  it held before its stack.

fanout:

• Map takes the program's name, its panic log and its class rule from
  the Bus in its context when MapOptions leaves them empty.
• A batch whose work returns a skip ends skipped, and Batches goes on
  with the next one rather than stopping.
• Each given a limit below one runs DefaultLimit at a time, as Map
  does, not one.
• Map ends the pool under a deadline as it does under an interrupt: the
  items it left out, and those whose work failed once the context had
  ended, end their targets canceled rather than failed as timeouts, and
  the step ends canceled when every item that failed ended so. The
  outcomes keep the context's error; tell a deadline from an interrupt by
  ctx.Err().
• An item Options.Acquire refused ends its target at once, rather than
  once the whole pool has finished: skipped for an error of Skip,
  canceled when the context had ended, and with the refusal otherwise.
  Its outcome keeps the refusal even when the context ends later.
• The release function Acquire returned runs before the item's target
  ends, so the next item can be reported running before that target's
  end.
• A panic in the release fails the item, with its stack in
  Options.PanicLog, instead of ending the process.
• A panic in the work or in Acquire fails its item even after the
  context has ended, rather than passing for a cancellation.
• Work, Acquire or a release that calls runtime.Goexit, as t.FailNow
  does, fails its item with an error that names which of the three it
  was, instead of leaving it reported as done.
• Failure lists names that repeat as given, so that the list names as
  many items as the count, and takes time linear in the number of names:
  an interrupt of a pool over 16000 hosts no longer holds the command for
  seconds.
• Recovered escapes the target it names.

progress/progresstest:

• Watch asks for the lines of output, so Check also checks their
  texts, and it checks the events at the test's cleanup when Finish is
  not called.
• Tree writes a target's name as {} only where it stands alone, not
  inside other words.
• Tree lists the names of targets that name a node twice, alone or in
  another target's node set, rather than folding them into one, so a
  target that ran twice shows.
• Tree orders siblings...