Repository navigation
Flow and step keys carry the flow's creation hour, and a step key names its flow. This is groundwork for retaining
ended flows cheaply. This release edits migrations that earlier releases shipped, so existing databases cannot be
upgraded in place: start v0.13.0 on fresh databases (see Upgrading).
Breaking changes
-
Flow keys are
{shard}-{hour}-{flowID}-{token}.{hour}is the flow's creation hour, in whole hours since
2000-01-01 00:00 UTC, taken from the database clock. This applies everywhere a flow key appears:- the keys
Create,Run,ContinueandForkreturn; FlowSummary.FlowKeyandFlowSummary.ThreadKey;FlowStep.SubgraphKey;FlowOptions.ThreadKeyandQuery.ThreadKey;Flow.FlowKey().
A three-part key from an earlier release is rejected with 400.
- the keys
-
Step keys are
{shard}-{hour}-{flowID}-{stepID}-{token}: a step key names its flow. This applies to
Flow.StepKey(), toFlowStep.StepKey,PredecessorKey,SuccessorKey,PrevKeyandNextKey, and to the step
keysStepandForkaccept. A step key is still stable across every re-execution of one step. A key stored
under an earlier release, including an idempotency key built fromStepKey(), does not carry over. -
Keys are accepted only as dwarf spells them. A key is rejected with 400 if any of these hold:
- the token is not exactly 16 lowercase hex digits;
- a number in the key has a
+sign or a leading zero; - the hour is above 2^31-1.
Earlier, a database collation that ignores case or trailing spaces (MySQL's and SQL Server's defaults) could match
another spelling of a key to its flow, and anAwaitparked on that spelling was never woken. An out-of-range hour
now returns 400 instead of 500. -
A key with the right token but the wrong hour is not found (404), the same as a wrong token. So is a step key
that names a flow the step is not in. -
Migrations were edited in place.
dwarf_flowsgainscreated_hour, set by the database clock, and
dwarf_stepsgainsflow_created_hour. Existing databases cannot be upgraded in place, and flows in them do not
carry over.
Behavior changes and fixes
- Keys now reveal when their flow was created, to the hour (see Data handling). The
correlation id on traces, logs and metrics is unchanged:{shard}-{flowID}, with no hour or token. Terminate,Delete,Purgeand the recovery sweeps no longer fail on a flow deleted concurrently. If the
flow's row was removed by the reaper or a racingDelete, the operation finishes with an empty final state
instead of returning an error.- Fewer spurious steals on a healthy fleet. Under saturation, two dispatchers sharing a shard no longer report
dwarf_steps_stolenanddwarf_steps_claim_lostwhen neither is slow. - Deleted flows are removed in smaller batches: 512 root flows per transaction, not 4,096. On PostgreSQL, a
deletion of large-state flows then holds back vacuum for seconds rather than tens of seconds.Purgestill marks
up to 4,096 flows per call. Query.Searchmatches keys in the new spelling, soListandPurgeby a full key work as before.
Performance
- MySQL and MariaDB: creating a flow costs one more statement, which reads back the creation hour.
Fork: one more read per flow it clones, on every database.- Otherwise no change. The engine's internals were restructured with no change in behaviour, and the local
benchmarks show no change in throughput.