df.seq silently emits its own serialized expression tree as SQL beyond ~100 chained steps

[pinodeca]

Component: DSL graph construction (df.seq / ~>)
Versions: pg_durable 0.2.1, duroxide 0.1.28, PostgreSQL 17.9 (Azure HorizonDB)

Summary

Building a sequential chain of more than roughly 100 steps does not fail at
construction time. df.start returns an instance ID normally, and the instance
is then guaranteed to fail at execution with:

SQL execution failed: error returned from database: syntax error at or near "{"

The cause is visible in the stored graph: past a certain point df.seq stops
materializing node handles and instead writes its own serialized expression tree
into df.nodes.query, where the worker subsequently executes it as SQL.
Inspecting df.nodes.query for the failing node shows literal JSON:

{"node_type":"THEN","left_node":{ …

This is a correctness bug rather than a capacity limit. A chain that exceeds the
builder's limit produces a corrupt graph, not a rejected one, and the corruption
is only discovered after the instance has been accepted and started.

Reproduction

-- h := df.sql(step_1); then repeatedly:
select df.seq(<h>, <next step expr>);   -- -> h
select df.start(<h>, label => <label>);

Repeat the df.seq fold N times.

N	Outcome
50	completes
100	completes
200	fails: syntax error at or near "{"
1000	fails: syntax error at or near "{"

Per-step cost up to the boundary is 102–110 ms per checkpointed step, so the
failure is not a timeout.

Node materialization at the break

Counting rows in df.nodes for the failing instances shows construction stops
well short of the requested depth:

Requested depth	SQL nodes	THEN nodes
200	73	72
1000	111	110

The counts are not a clean function of the requested depth, which suggests the
boundary is a size/recursion limit inside the builder rather than a step count.

Impact

Any workload that constructs a long sequential pipeline — a per-row ETL fold, a
generated migration chain, a loop unrolled at build time — will be accepted,
persisted, started, and then fail. Because the failure surfaces only at execution,
callers that check df.start for success will believe the workflow was durably
enqueued when the stored graph cannot run.

Suggested handling

Fail loudly at the construction boundary. Raising an error from df.seq when the
tree can no longer be materialized would be strictly better than emitting a graph
that is guaranteed to fail, and would let callers chunk or restructure before
anything is persisted. If a legitimate depth ceiling exists, documenting it and
enforcing it explicitly would also resolve this.

Provenance

Found during a stress campaign against pg_durable 0.2.1 on Azure HorizonDB
(PostgreSQL 17.9), 3 August 2026. Phases covered submission storms, chain depth,
fan-out, signals, cancellation, and durable-state growth. The full report is
internal to Microsoft; the reproduction above is self-contained.