Repository navigation
v1.51.0 moves the foreman to dwarf v0.16.0 and a project's SQL-backed microservices to sequel v1.20.2. The engine's
background work runs as separate jobs on each shard, a wide forEach no longer stalls at its fan-in, and new flows
go around a shard the replica cannot read. The foreman lists the flows that closed but are not archived yet, and
reads one flow's summary by its key. On the short-circuit transport, a request that overlapping subscriptions in one
queue match now reaches one of them, as it does over NATS. The foreman's databases need a schema change, or new
databases.
Highlights
- dwarf v0.16.0. Per-shard background jobs, wide fan-outs kept in check, and placement around an unreadable
shard. A database the v1.50.0 foreman created needs dwarf's v0.15.0 and v0.16.0 schema changes by hand. ListRecentlyClosedandSummaryon the foreman, and Agent Studio's Closed tab lists the flows that just
closed.- One handler per queue. Subscriptions whose routes overlap in a queue no longer each run a request that the
short-circuit transport delivers. - sequel v1.20.2.
db.Connreturns a*sequel.Conn, a migration file of several statements runs in one
transaction, and MySQL statements are no longer cut off after 8 seconds.
dwarf v0.16.0
The foreman moves to dwarf v0.16.0, by way of v0.15.0, and sequel v1.20.2. The engine's background work (archiving,
deletion, retention, expiration and crash recovery) runs as separate jobs on each shard, so one slow or blocked job
holds back no other, and a tree of any size is archived and deleted. A wide forEach no longer stalls at its fan-in
or takes the replica's workers with it, and new flows go around a shard the replica cannot read. dwarf's release
notes for v0.15.0 and
v0.16.0 have the full list.
The foreman's databases need a schema change. dwarf edits its migrations in place before v1.0, so a database
v1.50.0's foreman created does not pick the changes up, and nothing detects it: the foreman starts, then fails on
its first write of a flow or step. Start each environment's foreman on new, empty databases, or apply the
statements in the Breaking sections of dwarf's v0.15.0 and then v0.16.0 release notes with every replica stopped,
every archive day included. Tests need nothing.
ListRecentlyClosedlists the closed flows not archived yet. A flow that closes leavesListOpenat once but
reachesListClosedonly when it is archived, about a minute later, so for that minute it was in neither list. To
list every closed flow, callListRecentlyClosedand thenListClosed, and drop duplicates byFlowKey.Summaryreads one flow's summary by its key, open or closed, archived or not. EveryFlowSummarynow names
itsRootFlowKey, whichResume,Cancel,Terminate,DeleteandContinuetake, and itsParentStepKey,
the step that called a subgraph child.- Agent Studio's Closed tab lists the flows that closed in the last minute, beside the archived ones. On a
subgraph child's page it links to the root flow, and no longer offers the actions that a child's key cannot take.
Fork is offered on a flow's page and a step's page only once the tree's root is closed, whichForkrequires: a
completed child or step of a root still running or interrupted showed Fork, and the fork failed with a 409. - Durations in the foreman's JSON are integer milliseconds.
FlowOptions.TimeBudgetistimeBudgetMillis
(wastimeBudget, nanoseconds), andShardSummary.LatencyislatencyMillis. Go callers pick the names up with
the bump; a body written by hand that still sendstimeBudgetis ignored, and its flow runs on the default budget. - Text PostgreSQL cannot store is refused on every database. A NUL in a string, or invalid UTF-8 in JSON passed
through as given, fails the step that writes it, and is a 400 atCreate,Run,Resume,ContinueandFork.
It used to pass on SQLite. Carry binary data base64-encoded. - An expired flow is deleted, not archived. A flow
FlowExpirationends is terminated as before and deleted
within minutes. The check runs hourly, or every half of the expiration when that is shorter. - A subgraph's error in the carrier is a JSON object. Code using the generated clients needs no change, but a
foreman and a task-bearing microservice on different releases misread a subgraph's error: deploy them together. - Metrics.
dwarf_operation_secondsis renameddwarf_shard_operation_seconds.dwarf_archive_daysand
dwarf_reap_pass_secondsare removed: the background work is timed per job bydwarf_janitor_batch_seconds,
whether each job keeps moving isdwarf_janitor_progress_age_seconds, and the retention kept is
dwarf_archive_oldest_day_age_seconds.dwarf_archive_lag_secondsanddwarf_archive_failing_secondsgain a
joblabel, so aggregate them withmax. New:dwarf_ingress_operations, one count per API call by status, and
dwarf_placement_skipped, the flows placed around a shard the replica cannot read. The Grafana workflow dashboard
reads the new metrics and adds panels for the API calls, placement, and each background job.
sequel v1.20.2
A project's own SQL-backed microservices move to sequel v1.20.2 with the bump, from v1.18.0.
db.Connreturns a*sequel.Conn(v1.19.0), which holds one connection for a sequence of statements and
transactions and runs them with sequel's handling: placeholders, virtual functions and tracing. The*sql.Connit
returned before bypassed sequel. Code that stores the connection under thedatabase/sqltype, or callsBeginTx
orRawon it, no longer compiles;db.DB.Conn(ctx)returns the raw one.- A migration file of two or more statements runs in one transaction on PostgreSQL, SQL Server and SQLite
(v1.20.0), so a failure applies nothing and the next start runs the file again. A file not applied yet that holds a
statement that cannot run in a transaction, such asCREATE INDEX CONCURRENTLYorVACUUM, or that begins or
commits a transaction itself, now fails: move such a statement to a file of its own.sequel.txtstates the rule. - MySQL and MariaDB statements are no longer cut off after 8 seconds (v1.20.1). sequel set an 8-second read and
write timeout on every connection, so a lock wait, a largeDELETEor a slow migration failed with
invalid connectionwhile the server went on running it. The driver's timeouts now apply, none by default. - A cancelled SQL Server transaction no longer breaks the next statement on its connection (v1.18.1).
- Driver security fixes (v1.20.2): the MySQL driver v1.10.2, pgx v5.13.0 and go-mssqldb v1.12.0. On PostgreSQL, a
statement whose connection fails after it is sent now returns an error rather than being run again on another
connection. - The foreman registers a sequel virtual function,
SEEK_FROM. sequel's virtual functions are shared by
everything in the process, so a microservice bundled with the foreman must not register its own under that name.
Overlapping Subscriptions in One Queue
A request to a subject that more than one subscription in a queue matches now reaches one of them, as it does over
NATS. The short-circuit transport picked one subscriber per queue at each subject that matched, so subscriptions
whose routes overlap, such as /{p...} and /orders/{p...} in a microservice's default queue, each got a request
to /orders/1 when the caller was in the same process. Publish returned the first response and dropped the
others, but every handler ran, so its side effects happened once per overlapping route. The short-circuit now groups
a queue's subscribers across every subject that matches and picks one, each equally likely. Neither transport picks
the most specific route, so a catch-all such as /{p...} takes a share of the requests to the more specific routes
it overlaps in its queue.
Other Changes
- Email parsing moves to
github.com/go-letters/letters, the successor of the deprecated
github.com/mnako/letters, with the same API.smtpingressapi.Emailaliases itsEmailtype, so a project that
imports the old package to name a type of a parsed email renames that import. - Dependencies are updated, among them nats-server v2.15.3, nats.go v1.56.0 and grpc v1.85.0.
- The short-circuit trie no longer reallocates on long subjects. A lookup where two or more subscriptions
overlap at every segment of a long subject allocated twice per segment past the sixteenth; it now allocates only
its result. - A subject with a literal
*or>segment matches a wildcard subscription once. On the short-circuit
transport, such a segment matched both as the wildcard and as a literal, so the handler ran twice. It now matches
only through the wildcard, as in NATS. - CI runs govulncheck, failing on a known vulnerability the code can reach, scanned on the latest Go release.
- Task tools fail
Chatwith a 400. A tool URL that names a task failsChat, orInitChatand with it the
ChatLoopflow, with a 400 rather than a 500, the same as a tool URL that is unknown or that the caller is not
authorized for.
Migration
From inside a Microbus project, ask Claude Code to upgrade Microbus:
Get the latest version of Microbus.
The upgrade-microbus skill first tells you that the foreman's databases need dwarf's schema changes, and asks
whether each environment takes new, empty databases or the statements from dwarf's v0.15.0 and v0.16.0 release
notes. It offers to delete the local SQLite files. It then tells you to deploy the foreman and the microservices
that carry tasks together, renames timeBudget and latency in hand-written foreman JSON, and renames
dwarf_operation_seconds in dashboards and alerts, pointing out the queries that read a removed metric. It offers
to add ListRecentlyClosed where the project lists closed flows, and checks the project's own SQL-backed
microservices for sequel's changes: db.Conn, migration files with a statement that cannot run in a transaction,
and the SEEK_FROM name. It renames the email package import where a project uses it, and tells you about
the change to overlapping subscriptions and the status Chat returns for a task tool.