One primary entity per flow instance: Empire study and authoring simplification

[yazzaoui]

Summary

This is a study of whether Swarm should simplify the entity model around this principle:

One flow instance should have one primary state entity.

Said differently:

flow instance = state machine + one primary state object

This is not the same as saying a flow type can only ever have one runtime instance. A template flow can still have many instances:

operating[vertical_id=a]
operating[vertical_id=b]
operating[vertical_id=c]

The simplification is that each of those flow instances has one primary state object. Cross-flow routing targets flow instances, not arbitrary entities hidden inside a static flow.

After auditing Empire, I think this principle holds strongly. The current complexity is not that Empire has many entity types per flow. It does not. The complexity is that many static flows host many entity instances and then every handler has to recover the right entity with create_entity, select_entity, select_or_create_entity, and query_entities.

The core finding:

Empire already wants one primary entity per logical process. Several flows are authored as static singleton services only because the platform makes that the easier shape today. Those should likely become template flows keyed by the process identity.

---

Why this matters

The composition model we have been discussing tries to separate concerns cleanly:

output pin = what the producer publishes
input pin = what the receiver accepts
parent connect = how flows are wired
analyzer = prove and lower the route

But receiver-side entity lookup adds another hidden routing layer:

event -> receiver flow -> which entity inside that flow?

That is where a lot of the design complexity comes from:

• address_keys
• input.address.by
• select_entity
• select_or_create_entity
• target.match
• query_entities(...).count == 1
• cardinality/resolution ambiguity
• static service flows with internal entity registries

If we instead make the normal model:

event -> receiver flow instance

then receiver entity selection mostly disappears. The flow instance itself is the addressed unit.

---

Empire evidence

I inspected the Empire contracts under:

/Users/youmew/swarm/empire/contracts

The package has 10 child flows:

flows:
- id: signal-search
  mode: static
- id: discovery
  mode: static
- id: scoring
  mode: static
- id: lifecycle
  mode: static
- id: holding-devops
  mode: static
- id: treasury
  mode: static
- id: spec-repo
  mode: static
- id: repo-scaffold
  mode: static
- id: validation
  mode: static
- id: operating
  mode: template

Only operating is currently a template flow.

Every child flow defines exactly one entity type:

Flow	Mode today	Entity type	Observation
operating	template	Product	Already matches the proposed model.
validation	static	validation_case	Really wants one instance per validation case.
scoring	static	vertical	Really wants one instance per scored vertical.
lifecycle	static	lifecycle_case	Really wants one instance per marginal review case.
treasury	static	opco_budget	Really wants one instance per OpCo/vertical budget.
holding-devops	static	holding_resources	Really wants one instance per resource work request/job.
spec-repo	static	artifact_repo	Likely wants one instance per artifact repo identity.
repo-scaffold	static	repo_scaffold_artifact_repo	Likely wants one instance per scaffold repo/job identity.
discovery	static	scan_campaign	Likely wants one instance per scan campaign.
signal-search	static	signal	Mixed: campaign should be a flow instance; raw signals may be records/list items.

This is important: Empire is not proving that flows need many entity types. It is proving that we currently use static flows as ad hoc instance managers.

---

Current friction pattern

Validation

validation is a static flow with many validation_case entities.

Entry events create a new entity:

vertical.shortlisted:
  create_entity: true
  data_accumulation:
    writes:
      - target_field: validation_case_id
        expression: entity.entity_id
      - source_field: vertical_id
        target_field: vertical_id
      - source_field: vertical_name
        target_field: vertical_name
      ...

Later callbacks must reselect that entity:

spec_repo.repo_ready:
  select_entity:
    by:
      validation_case_id: payload.source_validation_case_id

This is really saying:

route this callback to validation[validation_case_id]

If validation were a template flow keyed by validation_case_id, the handler would not need select_entity for normal in-topology callbacks. The routing layer would deliver to the correct flow instance.

Proposed shape:

flows:
  validation:
    flow: validation
    mode: template
    instance:
      by: validation_case_id
      on_missing: create

Then:

connect:
  - from: scoring.vertical_shortlisted
    to: validation.vertical_shortlisted

  - from: spec-repo.repo_ready
    to: validation.spec_repo_ready

The analyzer proves that spec-repo.repo_ready carries source_validation_case_id or maps it to validation_case_id.

---

Scoring

scoring is a static flow with many vertical entities.

It creates a vertical on vertical.discovered:

vertical.discovered:
  create_entity: true
  data_accumulation:
    writes:
      - source_field: vertical_name
        target_field: vertical_name
      - geography
      - raw_signals
      - signal_strength
      - source_field: mode
        target_field: discovery_mode
      ...

Derived verticals also create more entities:

scoring.derived_requested:
  create_entity: true
  data_accumulation:
    writes:
      - source_field: opportunity_name
        target_field: vertical_name
      - source_field: parent_id
        target_field: parent_id
      ...

This is naturally:

one scoring flow instance per vertical/scoring case

Proposed shape:

flows:
  scoring:
    flow: scoring
    mode: template
    instance:
      by: vertical_id
      on_missing: create

The vertical.derived case may create a child scoring instance, with parent_id as lineage rather than same-flow entity lookup.

---

Treasury

treasury currently creates one opco_budget per vertical:

treasury.budget_open_requested:
  create_entity: true
  guard:
    checks:
    - id: unique_budget_vertical
      check: query_entities(vertical_id == payload.vertical_id).count == 0

Then most operational requests reselect it:

opco.spend_requested:
  select_entity:
    by:
      vertical_id: payload.vertical_id

opco.spend_recorded:
  select_entity:
    by:
      vertical_id: payload.vertical_id

opco.budget_adjustment_requested:
  select_entity:
    by:
      vertical_id: payload.vertical_id

This wants to be:

flows:
  treasury-budget:
    flow: treasury
    mode: template
    instance:
      by: vertical_id
      on_missing: reject

Then operating can wire to treasury directly:

connect:
  - from: operating.spend_requested
    to: treasury.spend_requested

  - from: operating.spend_recorded
    to: treasury.spend_recorded

The target is the treasury[vertical_id] flow instance. No receiver entity lookup needed.

---

Holding DevOps

holding-devops creates a new entity for each service request:

opco.infra_provisioning_requested:
  create_entity: true
  guard:
    checks:
    - id: unique_infra_provisioning_request_id
      check: query_entities(request_id == event.id).count == 0

Completions then validate against the current entity and also query uniqueness:

holding.infra_work_completed:
  guard:
    checks:
    - id: infra_completion_matches_unique_request
      check: entity.request_id == payload.request_id && ... && query_entities(request_id == payload.request_id).count == 1

This looks like a job/process:

requested -> awaiting_execution -> fulfilled/failed/denied/escalated

That is a strong template-flow candidate:

flows:
  holding-devops-job:
    flow: holding-devops
    mode: template
    instance:
      by: request_id
      on_missing: create

Each infra/CI/CD/DNS/deploy/rollback request becomes a flow instance. Completion events route back to holding-devops-job[request_id].

The output to operating then becomes a normal composition edge:

connect:
  - from: holding-devops.deploy_completed
    to: operating.deploy_completed

No per-emit target block:

# remove this from the handler happy path
emit:
  event: deploy.completed
  target:
    flow: operating
    match:
      vertical_id:
        ref: payload.vertical_id

---

Spec Repo and Repo Scaffold

These are service flows that currently use select_or_create_entity by repo identity.

spec-repo:

spec_repo.repo_registration_accepted:
  select_or_create_entity:
    by:
      scope: payload.scope
      scope_id: payload.scope_id
      artifact_type: payload.artifact_type

Callbacks select the repo entity by the same identity:

spec_repo.commit_completed:
  select_entity:
    by:
      scope: payload.provenance.scope
      scope_id: payload.provenance.scope_id
      artifact_type: payload.provenance.artifact_type

repo-scaffold has the same pattern with repo_scaffold_scope, repo_scaffold_scope_id, and repo_scaffold_artifact_type.

These likely want template instances keyed by composite repo identity:

flows:
  spec-repo:
    mode: template
    instance:
      by:
        - scope
        - scope_id
        - artifact_type
      on_missing: create

The only caution is that these flows wrap runtime-owned artifact repository behavior. If a single physical artifact repo can receive many commits over time, the flow instance should represent the repo, not the individual commit.

---

Operating already shows the target model

operating is already a template flow and has one primary entity, Product.

It stores contained data as lists inside the primary entity:

Product:
  vertical_id: uuid
  planned_components:
    type: "[PlannedComponent]"
    initial: []
  document_artifacts:
    type: "[DocumentArtifactState]"
    initial: []
  components:
    type: "[ProductComponentState]"
    initial: []

This is the right pattern for contained state:

Product owns components and document artifacts as local data.
Those items are not independently routable flow state.

If a component later needs its own lifecycle, agents, retries, timers, or independent routing, it should become a child/template flow instance.

---

Proposed rule

The platform default should be:

A flow instance has one primary state entity. Cross-flow routing addresses flow instances. Nested lists/maps inside that state are for contained data. If a contained item needs lifecycle or routing, promote it to a child flow instance.

This gives a clean ladder:

field = scalar state
list/map = contained local state
child flow instance = independently addressable lifecycle
sibling/service flow = reusable capability

It also gives a simpler explanation to authors:

A flow is a state machine. A template flow creates many state-machine instances. Each instance has one state object.

---

What this simplifies

1. Receiver-side addressing

Current model:

event reaches flow -> select which entity inside flow -> handler runs

Proposed model:

event reaches flow instance -> handler runs

This eliminates most normal-path select_entity usage.

2. Composition

Current:

emit:
  event: deploy.completed
  target:
    flow: operating
    match:
      vertical_id:
        ref: payload.vertical_id

Proposed:

connect:
  - from: holding-devops.deploy_completed
    to: operating.deploy_completed

The analyzer routes by instance key.

3. Uniqueness guards

Current:

guard:
  checks:
  - id: unique_budget_vertical
    check: query_entities(vertical_id == payload.vertical_id).count == 0

Proposed:

instance:
  by: vertical_id
  on_conflict: reject

The runtime/analyzer owns uniqueness for flow instances.

4. Authoring language

Instead of teaching authors:

static flow
entity table
create_entity
select_entity
select_or_create_entity
query_entities
pin routing target
receiver address keys

we can teach:

flow type
flow instance
primary state
connect
instance key
child flow

That is much easier.

---

What should remain as lists/maps

The simplification should not turn every record into a flow instance.

Use lists/maps when the item is contained and does not need independent routing:

Product:
  components:
    type: "[ProductComponentState]"
  document_artifacts:
    type: "[DocumentArtifactState]"

Good list candidates:

• planned components inside an operating product;
• document artifacts inside a product;
• dimension scores inside a scoring case;
• freeze categories inside a budget;
• evidence items or summaries;
• file manifests;
• static template file lists.

Use child/template flows when the item needs any of:

• its own lifecycle/state machine;
• timers;
• retries;
• independent agent assignment;
• independent event subscriptions;
• cross-flow addressability;
• durable idempotency identity;
• separate audit trail;
• parallel execution with callbacks.

That means these should likely be flow instances, not list entries:

• validation case;
• scoring case/vertical;
• lifecycle review case;
• budget account;
• deployment/resource work job;
• repo artifact identity;
• scan campaign.

---

The main downside: agent context across entities

The strongest downside is real:

If every case/job becomes its own flow instance, agents lose the accidental shared context they currently get from living inside one static flow that owns many entities.

Today:

same static flow
same agent role/session
many entities

After the change:

same flow type
many flow instances
separate instance contexts by default

This matters for agents such as:

• analysis-agent learning across many scored verticals;
• holding-devops-agent seeing repeated deploy failures across components;
• treasury learning spending patterns across OpCos;
• discovery/dedup agents comparing candidates across a campaign;
• validation agents reusing review heuristics across cases.

I do not think this is a reason to keep multi-entity static flows as the default. It means agent memory scope should become explicit.

Proposed split:

flow instance state = deterministic local case/job state
agent memory = configurable shared knowledge scope

Example:

agents:
  analysis-agent:
    context_scope:
      kind: flow_type
      flow: scoring

Possible scopes:

instance         # default: isolated to one flow instance
flow_type        # shared across all instances of this flow type
parent_instance  # shared across children under the same parent instance
workspace        # shared across the whole run/workspace
service          # named service memory, e.g. holding-devops

Default should be conservative:

agent context is instance-scoped unless explicitly widened

This preserves determinism and avoids accidental cross-case leakage, while still allowing useful shared memory.

The important design principle:

Do not use multi-entity static flows as a backdoor for shared agent memory. Shared memory should be a first-class, explicit capability.

---

Relationship to flow interfaces

This model works well with structural flow interfaces.

A service interface can require instance identity and pins:

name: swarm.deploy_service.v1

inputs:
  deploy_requested:
    payload: DeployRequest
    instance:
      by: request_id

outputs:
  deploy_completed:
    payload: DeployCompleted
    key: vertical_id
    replies_to:
      - deploy_requested

A concrete implementation can satisfy it with a template flow:

implements:
  - swarm.deploy_service.v1

instance:
  by: request_id

pins:
  inputs:
    deploy_requested:
      event: opco.deploy_capacity_requested

  outputs:
    deploy_completed:
      event: deploy.completed
      key: vertical_id

The parent composes against the interface, not a static entity table.

---

Relationship to addressed input pins

If we adopt one primary entity per flow instance, input pin addressing becomes much simpler.

Earlier model:

pins:
  inputs:
    deploy_completed:
      event: deploy.completed
      address:
        by: vertical_id
        resolution: one

Refined model for common cases:

instance:
  by: vertical_id

pins:
  inputs:
    deploy_completed:
      event: deploy.completed

The input pin does not need to say how to select an entity inside the flow. The flow instance key already defines that.

Input-level address metadata remains useful only when:

• a flow intentionally supports multiple addressable subresources;
• an external/no-target event must late-bind to a static singleton service;
• a legacy/migration flow still owns many entities;
• a specific input routes to a child resource rather than the primary state.

So input.address should become advanced/escape-hatch, not the happy path.

---

Suggested platform model

Flow instance declaration

mode: template

instance:
  by: vertical_id
  on_missing: create
  on_conflict: reject

Composite key:

instance:
  by:
    - scope
    - scope_id
    - artifact_type
  on_missing: create

Primary state entity

The entity file defines the primary state shape:

Product:
  vertical_id: uuid
  components: "[ProductComponentState]"
  document_artifacts: "[DocumentArtifactState]"

Potential future sugar:

entity: Product

or infer it if exactly one entity type exists.

Connect routing

connect:
  - from: operating.spend_requested
    to: treasury.spend_requested

Analyzer expansion:

operating.spend_requested carries vertical_id
treasury instance.by = vertical_id
route to treasury[vertical_id]

Field rename:

connect:
  - from: validation.spec_repo_requested
    to: spec-repo.repo_registration_requested
    using:
      instance:
        source: vertical_id
        target: scope_id

Explicit collection update remains local

Product:
  components: "[ProductComponentState]"

Handler update could remain a state write/update expression. It should not require creating a component entity unless component becomes independently routable.

---

Migration implications for Empire

Suggested migration shape:

Current flow	Proposed shape	Key
operating	keep template	vertical_id
validation	template	validation_case_id or vertical_id depending desired uniqueness
scoring	template	vertical_id / scoring case id
lifecycle	template	lifecycle case id or vertical_id
treasury	template	vertical_id
holding-devops	template job	request_id
spec-repo	template service instance	(scope, scope_id, artifact_type)
repo-scaffold	template service/job instance	(scope, scope_id, artifact_type) plus maybe component id if multiple components share artifact type
discovery	template campaign	campaign_id / scan_id
signal-search	template campaign; signal records as contained/output data	batch_id / campaign id

This migration would remove much of the normal-path select_entity/select_or_create_entity from Empire and shift uniqueness/idempotency to flow instance keys.

---

Potential objections

Objection: What about global service state?

Some services may need global aggregate state. Example: treasury might need portfolio-level budget caps.

Answer: model that as a parent/service singleton flow or explicit aggregate flow, not as many unrelated entities inside one flow.

treasury-portfolio = singleton aggregate
treasury-budget[vertical_id] = per-OpCo budget instance

Objection: What about querying across many instances?

A static flow with many entities makes cross-entity query feel natural.

Answer: cross-instance query should be explicit analytics/read-model capability, not the core routing model. The runtime can still index instances by fields for dashboards/analyzer/runtime, but event delivery should not depend on ad hoc entity queries.

Objection: What about dedup across many cases?

Dedup can be handled by:

• a parent aggregate flow;
• a service singleton flow;
• a read model/index;
• explicit agent memory scope;
• explicit uniqueness key on flow instances.

It should not require every process flow to be static and multi-entity.

Objection: Does this make too many flow instances?

Yes, it increases instance count. But it reduces internal entity-routing complexity. This is likely a good trade if flow instances are the runtime's primary unit of composition, lifecycle, trace, and isolation.

The platform must make flow instances cheap and first-class.

---

Analyzer responsibilities

Under this model, verify should prove:

1. Every template flow has a declared instance key or a valid default.
2. The primary entity has fields compatible with the instance key.
3. Output pins declare or infer the key they carry.
4. Connect edges map producer output keys to receiver instance keys.
5. Missing instance behavior is explicit: create, reject, or maybe resolve_only.
6. Duplicate instance creation is handled by instance uniqueness, not hand-authored query_entities guards.
7. Normal in-topology handlers do not need select_entity; if present, verify warns or rejects unless the flow opts into multi-entity/legacy mode.
8. Static flows with entity creation are flagged as advanced or migration-era.
9. Agent context scope is explicit when wider than the instance.
10. Collection fields are not cross-flow addressable unless promoted to child flows or explicit subresource interfaces.

---

Recommended design principle

I would promote this as a core authoring principle:

The normal unit of durable workflow state is the flow instance, not an arbitrary entity inside a static flow.

And the normal hierarchy should be:

root flow
  child service/capability flow
    template flow instances
      one primary state object
      contained lists/maps
      child flow instances for independent sub-lifecycles

This keeps composition understandable:

parent connects flow instances
handlers update local state
lists hold contained data
child flows handle independent work
agents can opt into shared memory explicitly

---

Open questions

1. Should the platform allow multiple entity types per flow at all, or only behind an explicit advanced.multi_entity: true flag?
2. Should static flows be stateless/singleton by default, with create_entity discouraged or warned?
3. Should mode: template become the default whenever a flow has states and a primary entity?
4. How should instance keys be declared and inferred from the primary entity?
5. Should select_entity become legacy/escape-hatch for external callbacks and static service records only?
6. How do we model cheap flow instances so this does not become operationally heavy?
7. What is the right explicit agent memory model to recover cross-instance learning?
8. How should aggregate/query use cases be modeled: parent aggregate flow, read model, or service singleton?
9. For repo services, should the instance represent the repo identity or each commit/job?
10. For validation, should the instance key be validation_case_id or vertical_id?

---

Recommendation

Adopt this direction as the default design target:

one flow instance = one primary state entity

Keep list fields for contained local data. Promote list items to child/template flows when they need independent lifecycle, routing, timers, retries, or agent work. Keep multi-entity static flows as an advanced/legacy escape hatch, not the normal model.

This would simplify Empire materially and also simplify the composition model we are designing. The main missing counterpart is explicit agent memory scope, because we should not rely on multi-entity static flows to get shared agent context by accident.

[yazzaoui]

On the agent-memory open question: it's an authoring concern, not a platform feature

One of the strongest concerns surfaced in this proposal is that moving away from static-multi-entity flows loses the accidental shared agent context that current-shape flows provide — and that "agent context scope" would have to become a first-class platform feature to compensate.

I want to argue the opposite: the agent-memory case dissolves into an authoring pattern that uses existing primitives. No new platform feature needed.

The pattern: static singleton coordinator + template-per-process

For each current static-multi-entity flow, split into two flows:

• Static singleton coordinator — owns aggregate state, cross-instance learning, contention tracking, shared policies. Has an agent if the cross-cutting reasoning needs one. Holds the durable shared knowledge.
• Template-per-process — handles individual lifecycles, one primary entity each (per this proposal's main rule). Typically deterministic; may not need its own agent at all.

They communicate via events. Per-process instances stay bounded. Singleton holds the cross-cutting knowledge explicitly.

Worked example: holding-devops

Today:

holding-devops (static)
├── holding_resources entities (one per request)
└── holding-devops-agent
    └── sees all entities; reasons across them (implicit shared context)

Proposed split:

holding-devops-coordinator (static singleton)
├── ContentionState entity (aggregate capacity, learned patterns, throttle policy)
└── holding-devops-coordinator-agent
    └── reasons over aggregate state, not individual jobs

holding-devops-job (template per request_id)
├── primary entity per job lifecycle
├── no agent (deterministic state machine)
└── subscribes to coordinator policy events

Communication:

• Job emits job.started → coordinator updates ContentionState
• Job emits job.completed → coordinator updates ContentionState + accumulated patterns
• Coordinator emits policy.updated → active jobs update local mirror (Pattern C from the OPCO draft)
• New jobs check ContentionState via coordinator routing before proceeding

Why this works

• Per-job state is bounded — one entity per instance, terminates when the job completes. No entity pile-up.
• Coordinator state is explicit — inspectable in the DB, writable in tests, replayable from events. No hidden agent-session sidecar.
• Agent context is focused — coordinator's agent sees aggregate state, not 5000 historical job entities pretending to be its memory.
• Replay works cleanly — rebuild the coordinator from event history; rebuild jobs from their inception. No special agent-context replay machinery.
• Uses existing primitives only — static singleton, template flow, events, entity state. Nothing new.

Generalizes across Empire

Most current static-multi-entity flows have a natural split. Coordinator is optional — many flows don't need one (only ship a coordinator when there's genuine shared state).

Today	Singleton coordinator (static, optional)	Per-process (template)
holding-devops	holding-devops-coordinator (contention, policy)	holding-devops-job (per request_id)
treasury	treasury-portfolio (cross-OpCo caps, aggregate spend)	treasury-budget (per vertical_id)
validation	validation-coordinator (cross-case patterns, optional)	validation-case (per validation_case_id)
scoring	scoring-meta (cross-vertical learning, optional)	scoring-vertical (per vertical_id)
lifecycle	lifecycle-coordinator (cohort decisions)	lifecycle-case (per case id)
discovery	discovery-coordinator (cross-campaign learning, optional)	discovery-campaign (per campaign_id)
signal-search	signal-search-coordinator (corpus state, dedup index)	signal-search-campaign (per batch_id)
spec-repo	(probably no coordinator needed)	spec-repo-instance (per repo identity)
repo-scaffold	(probably no coordinator needed)	repo-scaffold-job (per request)

The platform doesn't need to know about agent memory. Consumers decide per-flow whether shared state warrants a coordinator. The choice of where the agent lives IS the choice of its memory scope — no separate scoping primitive required.

Why this is materially cleaner than a platform "agent context scope" primitive

A platform-level context_scope: flow_type | parent_instance | workspace | service would require:

• Designing how each scope physically works (shared session? shared knowledge file? shared embedding store?)
• Defining replay/fork semantics when agent context depends on peer-instance state
• Surfacing context source in agent.diagnose for debugging
• Trust/isolation semantics for multi-tenant deployments
• Documentation about which scope to pick when

The singleton-coordinator pattern avoids all of this. Memory becomes data (the coordinator's entity state). Authoring discipline replaces platform machinery.

The one subtle area worth naming

The pattern is clean when shared state is information (capacity counts, learned patterns, aggregate spend). It's less clean when shared state is tacit reasoning (an agent that has reviewed 100 cases has implicit understanding hard to crystallize as entity fields).

For the tacit case, the singleton coordinator's long-running agent session covers it naturally — within the singleton's flow instance lifetime, the agent has continuous session context. Across restarts or across coordinator boundaries, tacit knowledge has to be crystallized as entity state (rules, templates, patterns written explicitly).

This is fine for most Empire cases. The "agent learns implicit patterns across thousands of cases" case probably ends up needing external memory primitives anyway (vector DB, fine-tuning, etc.) — but that's true today and isn't a regression.

Implication for this proposal

The "agent context scope as parallel platform workstream" concern dissolves. The proposal becomes:

• Core: one primary entity per flow instance; static-multi-entity becomes legacy escape hatch.
• Companion authoring pattern: use static singleton coordinators for genuinely shared state / cross-instance learning / aggregate policy.
• Migration: each former multi-entity static flow splits into coordinator (optional) + per-process template.

No new platform features for agent memory. Authors decide per-flow whether they need a coordinator. The platform stays small.

This makes the broader proposal materially more shippable — the platform investment shrinks to "make flow instances cheap and first-class" (still real; still needs the load-test validation), without committing to design and ship a new agent-context primitive in parallel.

Pilot implication

If holding-devops is the first pilot of one-primary-entity-per-instance, pilot the full pattern — split into coordinator + job, not just the job half. Validates both halves of the shape and the communication pattern between them. Migration cost estimate covers the full refactor.

The companion pattern probably deserves a paragraph in the OPCO draft's authoring guidance section so consumers know "if you find yourself wanting cross-instance learning, the answer is a singleton coordinator, not a multi-entity static flow."

[yazzaoui]

Systematic study of downsides — singleton coordinator + template-per-process pattern

Following up on the previous comment with a comprehensive inventory of the costs this pattern carries, organized by category. The win is still real, but the costs are real too and should be understood before migration starts.

Each entry: brief description, severity (Low / Medium / High), and whether the cost is intrinsic (structural to the pattern) or mitigatable (addressable through tooling, convention, or design).

---

Category A: Contract / authoring surface costs

A1. Flow count growth (~70% in Empire). Medium. Intrinsic.

Empire goes from 10 flows to ~16-17 (10 templates + ~6-7 coordinators). Each flow has its own contract files (schema, nodes, events, agents, prompts). More files to maintain, navigate, version. Per-consumer cost is unchanged (don't ship a coordinator if you don't need one), but total contract surface grows.

A2. Cross-flow event surface expands. Medium. Intrinsic.

What were internal events between handlers in one static flow become typed cross-flow events between coordinator and per-process instances. Each crossing needs pin declarations, payload schemas, verifier checks, conformance tests. The event topology becomes denser and the analyzer has more to verify at boot.

A3. Coordinator agent prompts need rewrites. Medium. Mitigatable.

Today's holding-devops-agent prompt assumes it sees all entities. After the split, the coordinator agent reasons over aggregate ContentionState, not individual cases. This is bounded authoring work but real migration effort per coordinator. Worse if the prompt currently bakes in "look at recent entities for context" patterns.

A4. Naming and discoverability requires convention. Low. Mitigatable.

Authors need to learn: "look for *-coordinator for shared state; look for the template for per-process work." Without documentation, the split is opaque to new readers ("why are there two holding-devops flows?"). Mitigatable via naming conventions + OPCO draft authoring guidance.

A5. Documentation / teaching burden grows. Medium. Mitigatable.

The model is conceptually clean but authoring it requires understanding both halves of the pattern. Onboarding new authors becomes a longer journey: "first you learn flow + entity; then you learn that template flows are per-process and singletons are for shared state; then you learn the coordinator/template pairing pattern."

A6. Pattern proliferation risk: false coordinators. Medium. Mitigatable.

Once the pattern is canonical, authors may add coordinators where they don't belong — for flows that genuinely don't need shared state, just because the pattern exists. Cargo-culting risk. Mitigation: authoring guidance that says "coordinator is optional; only add when there's real cross-instance work."

A7. Test fixture complexity grows. Low. Mitigatable.

Testing per-instance behavior used to be "set up one entity in the static flow." Now it's "set up coordinator state + per-instance state + the events between them." More fixture surface, more setup boilerplate per test. Mitigation: shared fixture helpers per coordinator/template pair.

---

Category B: Coordinator state management

B1. Coordinator state grows unbounded without explicit lifecycle. High. Intrinsic.

ContentionState, learned patterns, accumulated aggregates, historical decisions — these all accrue on the coordinator entity. Today's multi-entity static flow hides this (old entities just sit there); the new pattern forces explicit decisions:

• What gets archived vs kept?
• What windowing applies (last N days? last N events?)?
• When does the coordinator entity get GC'd or rolled over?
• If the coordinator entity is sharded, how?

Every coordinator needs a state-lifecycle design. Cannot be deferred without operational pain.

B2. Coordinator becomes a serialization point. Medium. Intrinsic.

Many per-instance flows emitting events to the coordinator concurrently means the coordinator's entity has lots of write concurrency. Today's per-entity serialization helps correctness but means the coordinator's throughput is bounded by its single-entity write rate. For high-volume coordinators this matters.

B3. Coordinator as "god flow" risk. Medium. Mitigatable.

Operators may pack too much into the coordinator over time — every new shared concern accretes there. Coordinator entity grows wide; coordinator agent's prompt grows complex; coordinator becomes a kitchen sink. Discipline needed to keep coordinator scope narrow. Mitigation: review discipline + "split coordinator if it covers >1 concern" guideline.

B4. Cold-start problem for new OpCos. Medium. Mitigatable.

When a new OpCo first comes online, the coordinator may not have learned anything specific about that OpCo's workload yet. Per-instance flows start with no specific policy guidance. Mitigation: coordinator ships with default policies; learns OpCo-specific patterns over time. But behavior of early OpCo lifecycle is materially different from later OpCo lifecycle (less learned context).

---

Category C: Operational dependencies

C1. Per-instance flows depend on coordinator availability. High. Intrinsic.

Today's static-multi-entity flow has no cross-flow runtime dependency — the flow is self-contained. After split, per-instance flows that need coordinator policy events have a runtime dependency on the coordinator being healthy. Coordinator down → policy events stop → instances proceed with stale local mirrors or stall.

This is a real new operational coupling. Not catastrophic (Pattern C's local mirrors give graceful degradation) but real.

C2. Coordinator availability becomes a first-class SLO. Medium. Intrinsic.

Today's flow availability is implicit in the runtime's health. After split, each coordinator becomes a distinct availability surface that ops needs to monitor, alert on, and recover. Number of coordinators × availability targets = real ops surface growth.

C3. Bottleneck risk for high-volume cases. High. Mitigatable.

For high-volume coordinators (signal-search seeing thousands of signals; treasury-portfolio across many OpCos), the coordinator may become a throughput limit. Per-instance work is naturally parallel; coordinator work is naturally serial-on-its-entity. Mitigation: shard the coordinator (multiple coordinator instances by some partitioning key), or restructure (signals stay as records inside campaign template, not events to a coordinator — which IS the proposal's intended answer for that case).

C4. Cross-coordinator coordination is not native. Medium. Intrinsic.

If two coordinator domains overlap (e.g., treasury-portfolio AND holding-devops-coordinator both need to know about an OpCo's resource usage), the pattern doesn't natively compose. You need either a third coordinator above them, or events between coordinators (which is more event surface). The pattern is clean for orthogonal concerns but degrades for overlapping ones.

---

Category D: Lifecycle and migration

D1. Migration data: what happens to historical entities? High. Intrinsic.

Today's static-multi-entity flows have accumulated entity rows representing real history. The migration doesn't have a clean place to put that history:

• Stay in legacy flow forever → two patterns coexist permanently
• Replay/migrate into coordinator's state → real migration work, risk of data loss or transformation errors
• Discard → lose historical context

No clean answer. Most flows will end up with "legacy entities preserved for read-only inspection; new work flows through the new pattern."

D2. Backward compatibility for run history. Medium. Intrinsic.

Old run.fork operations expect entity_state rows under the static flow's identity. After migration, the entity model differs. Historical run inspection / fork-from-old-runs may not work cleanly. May require a versioning/compatibility shim for run history that crosses the migration boundary.

D3. Tacit reasoning bounded by session lifetime. Low. Intrinsic.

The singleton's long-running agent session covers continuity within the singleton's flow instance lifetime. Across restarts or session resets, tacit knowledge has to be crystallized as state. Fine for most cases. The "agent learns implicit patterns across years of operation" case still needs external memory primitives (vector DB, fine-tuning) — but that's true today too and isn't a regression.

D4. Coordinator session lifetime IS its operational lifetime. Medium. Intrinsic.

Related to D3 but worth naming: the coordinator's flow instance is a singleton, but flow instances do terminate (intentionally or via cleanup). When a coordinator instance is terminated and recreated, its agent session restarts. Authors who assume "the coordinator always remembers everything" will be surprised. Documentation / lifecycle clarity needed.

---

Category E: Replay, debugging, observability

E1. Replay across coordinator + instances is more complex. High. Intrinsic.

run.fork semantics today reconstruct one flow's state at time T. After split, forking means reconstructing BOTH coordinator state at T AND per-instance state at T, with consistency (events from peer instances that influenced coordinator state must have replayed in order). The reconstruction is well-defined but the runtime machinery is more involved.

E2. Operational visibility splits across two flows. Medium. Mitigatable.

Today you look at one flow's entity table to see all activity. After: dashboards / debugging / metrics have to cover both coordinator state AND per-instance states. Tooling needs to span both. Mitigation: convention (always pair coordinator + template in dashboards) + tooling support (e.g., "show me everything in the holding-devops domain" surfaces both).

E3. Debugging requires understanding the split. Medium. Mitigatable.

When something goes wrong in holding-devops-job processing, you may need to look at the coordinator state to understand why (e.g., "the policy event the job is waiting for never arrived because the coordinator entity is in a weird state"). The debugging path is longer and crosses flows. Mitigation: tooling that surfaces the coordinator/template relationship explicitly.

E4. Event chain length grows. Low. Intrinsic.

Today's "request → process → response" is mostly in-flow. After split, it's "request → emit-to-coordinator → coordinator decides → emit-policy → instance proceeds → emit-result → coordinator updates." More hops, more events to trace through, longer causal chains in run logs.

---

Summary by severity

High-severity costs (5):

• B1: Unbounded coordinator state growth — every coordinator needs explicit lifecycle
• C1: Runtime dependency on coordinator availability
• C3: Bottleneck risk for high-volume coordinators
• D1: Historical entity migration with no clean answer
• E1: Replay/fork complexity across coordinator + instances

Medium-severity costs (9):

• A1, A2: Flow count and event surface growth
• A3: Coordinator agent prompt rewrites
• A5, A6: Documentation burden + pattern-proliferation risk
• B2, B3: Concurrent writes + god-flow risk
• B4: Cold-start
• C2: SLO surface growth
• C4: Cross-coordinator coordination
• D2: Run history backward compat
• D4: Coordinator session lifetime semantics
• E2, E3: Visibility splits + debugging complexity

Low-severity costs (3):

• A4: Naming convention requirement
• A7: Test fixture complexity
• D3: Tacit reasoning bounded by session lifetime
• E4: Event chain length

---

Intrinsic vs mitigatable

Intrinsic (cannot be eliminated by tooling/convention):

• A1, A2 (flow count + event surface — structural to splitting one flow into two)
• B1, B2 (coordinator state growth + concurrency — structural to having one shared entity)
• C1, C2, C4 (cross-flow runtime dependencies — structural)
• D1, D2, D3, D4 (migration history + lifetime semantics — structural)
• E1, E4 (replay complexity + event chain length — structural)

Mitigatable (can be reduced by tooling/convention/design):

• A3, A4, A5, A6, A7 (authoring concerns — addressable via docs + tooling)
• B3, B4 (god-flow + cold-start — addressable via review discipline + default policies)
• C3 (bottleneck — addressable via sharding or restructuring)
• E2, E3 (observability/debugging — addressable via tooling that spans the pair)

About 60% of the costs are intrinsic. The other 40% are addressable with discipline / tooling investment.

---

Implications for the broader study

For the pilot phase (whichever flow goes first):

1. Confront the high-severity costs explicitly. B1 (state lifecycle), C1 (availability dependency), C3 (bottleneck risk), D1 (migration history), E1 (replay) all need answers BEFORE committing — not after the pilot starts. Each one can sink the pilot if hit unexpectedly.

2. Don't ship a coordinator without a state-lifecycle design. B1 is the most insidious because it's slow-burn. State grows, performance degrades, eventually it's a crisis. Coordinator schemas should declare lifecycle policy at design time (e.g., "ContentionState fields archived after 30 days; aggregate counters preserved").

3. Choose pilot flow carefully. holding-devops is the strongest case (job-shape, modest volume, clean coordinator scope). signal-search would be the worst pilot (high-volume, bottleneck risk, cross-coordinator coordination with discovery). Pilot the cleanest case first; learn from it before tackling the harder ones.

4. Tooling investment needs to be sized. The mitigatable costs (~40%) all require platform investment in tooling, conventions, and authoring guidance. If we ship the pattern without the tooling, the costs that are theoretically mitigatable become real costs in practice.

5. The pattern's appeal is partly that it uses existing primitives — but the costs above show that "existing primitives" doesn't mean "no new work." The platform doesn't need new primitives, but consumers (and the docs/tooling layers) need substantial work to make the pattern usable at scale.

---

Net

The pattern is the right answer. The costs above don't change that conclusion. But they DO mean:

• Migration takes longer and costs more than the simple "split each multi-entity static into coordinator + template" framing suggests
• High-severity costs need explicit answers per coordinator before pilot
• Tooling investment is a real parallel workstream, not free
• Some coordinators may end up not getting built (because the costs outweigh the benefit for their specific shared-state case), and that's correct outcome — coordinator is optional

The systematic study before pilot should answer for each prospective coordinator:

1. What shared state does it own? (Bound the scope to prevent god-flow)
2. What's the state lifecycle policy? (Prevent unbounded growth)
3. What's the availability target? (Force the SLO question)
4. What's the bottleneck risk? (Volume-shape analysis)
5. What's the migration path for historical entities? (Don't lose data)
6. What's the replay/fork story? (Cross-flow consistency)
7. What's the bare-minimum tooling needed to operate this in production? (Sizing the parallel work)

If those answers exist for the pilot flow, the pilot is real. If they don't, the pilot is research disguised as engineering.

[yazzaoui]

My read: adopt the direction, but make the pilot prove the hard parts

I agree with the core direction: one flow instance should normally own one primary state entity. I would adopt it as the default authoring model, not as an absolute ban on multi-entity/static shapes.

The main reason is semantic ownership. The current static-flow + many entity rows pattern creates a hidden second routing layer:

event -> flow -> which entity?

That forces authors and the runtime to keep re-solving identity through select_entity, select_or_create_entity, query_entities(...).count == 1, target.match, and uniqueness guards. Most of that logic is really instance routing in disguise. If the process identity is the flow instance identity, the common path becomes simpler:

event -> flow instance -> primary state entity

That is a better mental model for authors and a better semantic owner for the platform.

What we materially win

• Simpler authoring story: flow type, flow instance, primary state, contained fields, child flow when independently routable.
• Less receiver-side entity selection and fewer local uniqueness guards.
• Cleaner idempotency: instance.by / on_conflict can own process identity instead of hand-authored query_entities checks.
• Better local debugging for process-shaped work: one request/case/job maps to one traceable flow instance.
• Better lifecycle isolation: retries, timers, terminality, agent assignment, and audit attach naturally to the process instance.
• Cleaner composition: connect can route by instance key rather than every producer hand-authoring target selectors.

I agree with the user's intuition here: flow instantiation should be cheap. If it is not cheap enough to model real process instances, the platform is pushing authors toward worse semantic shapes.

What we lose or pay

The biggest costs are not instance count by itself. The real costs are:

1. Coordinator state lifecycle. If we split holding-devops into coordinator + jobs, the coordinator's aggregate state can grow forever unless it has a lifecycle/retention/rollup policy.
2. Coordinator availability and bottlenecks. A coordinator becomes a real runtime dependency and possible serialization point. Local mirrors and stale-policy semantics need to be designed deliberately.
3. Replay/fork complexity. Replaying a process now may require both the per-process instance and the coordinator facts that influenced it at that time.
4. Observability split. Debugging crosses two flows: the job/case and its coordinator. Tooling must show a domain view, not just isolated flow views.
5. Historical migration. Old static-flow entity rows will not cleanly become new flow instances. The honest default is probably read-only legacy history plus new-shape work going forward.
6. False coordinators. Once the pattern exists, authors may create coordinators for everything. That would recreate singleton sprawl under a new name.

These are serious, but they are better problems than hidden semantic routing. They are explicit architecture/ops problems and can be designed around.

Coordinator pattern: good, with a constraint

I agree with the follow-up comment that shared agent memory should usually be modeled as a static singleton coordinator plus template-per-process instances, not as a new platform-level agent context_scope primitive.

But I would add one hard rule:

Coordinator memory must become typed, inspectable state or event-reconstructable facts. The coordinator agent's conversation history cannot be the semantic owner.

Tacit LLM experience is useful, but if the system depends on it, it must crystallize into coordinator state: rules, policies, learned thresholds, counters, templates, suppression lists, etc. Otherwise replay, fork, diagnosis, and review become mushy.

Platform/authoring acceptance criteria before broad adoption

I would not migrate Empire broadly until the pilot proves these invariants:

• Template flows can declare instance.by, on_missing, and on_conflict as first-class contract surface.
• Exactly one primary entity is declared or inferred for normal template flows.
• Verify proves primary entity fields are compatible with the instance key.
• Verify proves connect edges map producer keys to receiver instance keys.
• Normal in-topology handlers do not need select_entity; if present, verify warns or rejects unless the flow is external/legacy/advanced.
• Static flows that create many entities are marked as advanced/migration-era, not normal happy path.
• Coordinators declare state lifecycle policy: retention, rollup/archive, and cleanup behavior.
• CLI/dashboard can show a domain view that joins coordinator + child instances.
• Replay/fork story is explicit for coordinator-influenced child work.
• Legacy history strategy is explicit: read-only old rows vs migration vs replay.

Pilot recommendation

holding-devops is the right pilot if we want to prove the whole pattern:

holding-devops-coordinator
  owns aggregate contention/capacity/policy state

holding-devops-job[request_id]
  owns one request/job lifecycle

This pilot is strong because it exercises both halves: per-request lifecycle plus real cross-instance aggregate policy. signal-search would be a poor first pilot because the volume and dedup/read-model concerns are likely to dominate before the core model is proven.

Net decision

I would proceed with this as the default design target:

normal unit of durable workflow state = flow instance + one primary entity

Keep lists/maps for contained local data. Promote to child/template flow only when the item needs lifecycle, routing, retries, timers, agent work, or independent audit.

Keep multi-entity static flows as an explicit escape hatch for legacy, external ingress, read models, or true singleton aggregate services. Do not keep them as the normal process-instance model.

The trade is worth it: we move complexity out of hidden receiver-side entity selection and into explicit process instances, coordinator state, and tooling. That is a better place for the complexity to live.

[yazzaoui]

Refinement: not all flows want template; some want static singleton + map-as-contained-data

Following up on the previous comments. The pressure-test "what actually forces holding-devops to split?" surfaced something important: the proposal's "migrate everything to template" framing over-applies its own principle. For some flows, the right shape isn't template-per-business-identity at all — it's static singleton with a typed map.

The pattern

For holding-devops specifically:

holding-devops (static singleton)
  entity: HoldingDevopsState
    verticals: map<vertical_id, VerticalContext>
      VerticalContext:
        active_jobs: [{request_id, status, started_at, ...}, ...]
        capacity_allocated: {...}
        history: [...]
        learned_patterns: [...]
    aggregate:
      total_capacity, contention, throttle_policy, ...
  agent: holding-devops-agent
    # session sees the whole entity → all verticals + aggregate
    # cross-vertical pattern detection naturally preserved

Event arrives carrying vertical_id → handler reads entity.verticals[payload.vertical_id] for the relevant context. Agent has the rest of the entity in session for cross-vertical reasoning. No select_entity, no flow proliferation, no coordinator split.

Why this is consistent with the proposal's own ladder

The proposal explicitly says:

Use lists/maps when the item is contained and does not need independent routing.
Use child/template flows when the item needs own lifecycle/state machine, timers, retries, independent agent assignment, cross-flow addressability...

For per-vertical context in holding-devops:

Indicator	Applies?
Independent state machine per vertical	No — no per-vertical lifecycle with explicit states
Timers per vertical	No — timers are per-job (list item), not per-vertical
Independent agent per vertical	No — one agent serves all (feature, not bug)
Cross-flow addressable per vertical	No — other flows talk to holding-devops as a whole
Independent event subscriptions per vertical	No

By the proposal's own rule, per-vertical context is contained data. Map field is correct. Promoting each vertical to a flow instance would be over-promotion — it goes past the line the proposal itself draws.

The unit-of-routing observation

The right addressing unit isn't necessarily per-request or per-vertical or per-anything-specific. It's whatever has its own state machine and lifecycle requiring platform machinery.

For holding-devops:

• Each request has a lifecycle (requested → working → completed/failed) → simple enough to be a list item with a status field, not a flow instance
• Per-vertical context accumulates over the OpCo's lifetime → contained data (map value)
• The holding-devops service itself is the unit with the agent session → static singleton

Simplest mapping that works:

• holding-devops = static singleton (one flow instance, one agent session)
• request lifecycle = list item with status field (no platform state machine needed)
• per-vertical context = map value (organizational; contained data)

When map-as-data is right vs when template-per-instance is right

The map-as-contained-data pattern works when:

• Per-key state is mostly data (capacity, history, accumulated learning)
• No need for platform state-machine machinery (advances_to, sets_gate, timer cancellation, gate accumulation across handlers) per key
• Agent session benefits from seeing across keys (cross-key learning case)

Template-per-instance is right when:

• Each key has a complex multi-state machine with platform states
• Per-key timers, gates, retries are needed
• Per-key agent assignment is meaningful
• Cross-flow events need to address a specific key as a recipient (e.g., callback target)

Revised migration table for Empire

Applying this discrimination to the proposal's original table:

Flow	Per-identity state shape	Right pattern
operating	per-OpCo rich state machine (states, gates, transitions)	Template per vertical_id (already done)
validation	per-case state machine (4 gates, multi-phase)	Template per validation_case_id
scoring	per-vertical scoring state machine (gates, dimensions, accumulate)	Template per vertical_id
operating child component-scaffold	per-component g1..g5 gate matrix	Template per component_id (already done via PR #107)
lifecycle	per-case marginal review (simple state, mostly data)	Singleton + map<case_id, review> likely better
holding-devops	per-vertical context (data + agent learning)	Singleton + map<vertical_id, ctx>
treasury	per-vertical budget (counters + history)	Singleton + map<vertical_id, budget>
spec-repo	per-repo state (data + commit history; runtime-owned semantics)	Singleton + map<repo_identity, state> likely
repo-scaffold	per-job lifecycle (artifact creation pipeline)	Template per request might fit; or singleton+map if state is simple
discovery	per-campaign state (some state machine)	Template per campaign_id likely
signal-search	thousands of signals per campaign	Template per campaign + signals as list items in campaign

About 5-6 flows naturally want template (where per-key state machines need platform machinery). About 3-4 want singleton + map (where per-key state is contained data + shared agent context is valuable). One or two are ambiguous and depend on the specific state shape.

This is a materially different migration than "migrate all 10 flows to template." It's:

• Cheaper (fewer migrations)
• More honest (preserves cross-key agent learning where it's genuinely valuable)
• More nuanced (each flow gets the shape that fits its state)

Refined design principle

The principle from the original proposal should be stated more precisely:

One primary entity per flow instance. Per-business-identity state lives as map values when it's data + shared agent context; promotes to template flow when it needs platform state-machine machinery.

This preserves the "one entity per flow instance" rule (which is right) while making the map-vs-template choice the design decision, with clear criteria for when each fits.

One load-bearing technical question

For map-as-contained-data to work, Swarm's CEL needs to support indexed map access with a variable key:

# Reading in guards:
guard:
  check: entity.verticals[payload.vertical_id].capacity_allocated.cpu < policy.max_cpu_per_vertical

# Writing nested values:
data_accumulation:
  writes:
    - target_path: entity.verticals[payload.vertical_id].active_jobs
      operation: append
      value: {request_id: event.id, status: 'requested', started_at: now()}

We hit verifier limits before with list helpers — Swarm #1031 rejected entity.list.all() / .filter() / .map() as unsupported segments. Map indexing by variable key (entity.map[variable]) might hit similar restrictions.

If the verifier doesn't fully support map indexing with variable keys, that becomes the load-bearing platform ask for this pattern — and it's a much smaller and more surgical ask than "make flow instances cheap and first-class for thousands of instances per OpCo."

Worth a quick spike against current platform-spec / verifier before this pattern can be confirmed as viable.

Net implication for the proposal

The original "one primary entity per flow instance" principle stands. The migration framing needs refinement:

• Default for new flows: pick template-per-instance OR singleton-with-map-as-data based on whether per-key state machine machinery is needed
• For existing static-multi-entity flows: migration target depends on per-key state shape; about half want template, half want singleton+map
• The companion authoring pattern (singleton coordinator + per-process template) from my previous comment is still right for cases where the per-key state genuinely warrants its own state machine + cross-key aggregation is also needed
• CEL map indexing support becomes the load-bearing platform question; should be verified before committing to the pattern

This makes the proposal materially more shippable: smaller migration scope, preserves valuable cross-key agent context where it exists, smaller platform investment (map indexing in verifier is a much smaller change than "cheap flow instances at scale"), and a cleaner mental model for authors ("does per-key need a state machine? template. Otherwise: map value").

[yazzaoui]

Authoring impact: simpler mental model, but only if the platform is opinionated

I want to call out the authoring/DX angle explicitly, because it is probably the strongest reason to pursue this direction.

Overall, this model should make authoring easier and more natural. The current model asks authors to juggle too many overlapping mechanisms:

static flow
many entity rows
create_entity
select_entity
select_or_create_entity
query_entities(...).count == 1
producer target.match
payload fields that are really routing keys
receiver-side late binding

That is hard to teach because the author has to decide whether identity belongs to the producer, the receiver, the payload, the selected entity, or the query guard. A lot of our recent routing issues came from that ambiguity.

The proposed model gives a cleaner ladder:

flow instance = one process / lifecycle
primary entity = state of that process
list/map = contained local data
child/template flow = contained item promoted because it has its own lifecycle/routing
singleton coordinator = shared policy / aggregate learning across instances

That is closer to how people naturally think about workflows: cases, jobs, campaigns, products, budgets, requests.

The simple authoring rubric

I would teach authors this decision tree:

1. Does this thing have independent states, timers, retries, callbacks, terminality, or audit?

   • Yes → make it a template flow instance.

2. Does another flow need to address it directly as a recipient?

   • Yes → make it a template flow instance.

3. Is it just data owned by the current process?

   • Yes → keep it as a typed field/list/map on the primary entity.

4. Does it summarize, coordinate, or learn across many process instances?

   • Yes → use a singleton coordinator, but only if there is real shared state or policy.

This is much easier than asking authors to choose among select_entity, select_or_create_entity, query guards, target selectors, and implicit payload conventions.

Flow composition becomes easier too

This also aligns well with the flow-composition work.

For process-shaped receivers, composition should be:

producer output pin -> connect -> receiver instance key -> flow instance

For contained data, composition should stop at the owning flow instance:

producer output pin -> connect -> singleton/template owner -> typed local state update

So we should keep two concepts distinct:

• connect routes across flow instances.
• typed keyed-map/list operations update contained state inside the current primary entity.

If a map key starts behaving like a routable recipient, that is the promotion signal: it should become a flow instance.

Where authoring can get harder

The risk is not the principle. The risk is exposing too much design choice without good primitives.

Authoring gets harder if:

• every small record becomes a flow instance;
• every shared concern gets a coordinator;
• authors must hand-write all cross-flow event plumbing;
• maps require arbitrary CEL expressions like entity.verticals[payload.vertical_id].jobs.filter(...);
• there is no generated scaffold for common template/coordinator patterns;
• dashboards show only isolated flows instead of the domain pair.

So the platform should be opinionated. It should make the happy path obvious and make escape hatches feel advanced.

What the platform should provide to keep this easy

Minimum DX surface I would want before broad migration:

• instance.by, on_missing, and on_conflict as first-class template-flow syntax.
• One declared/inferred primary entity per normal flow instance.
• connect edges that route by instance key without hand-authored target blocks in the common case.
• Typed keyed-map operations for contained state, not arbitrary path strings or list traversal hacks.
• Verifier warnings/errors when normal in-topology handlers use select_entity unnecessarily.
• A generator/scaffold for:
  • template-per-process flow;
  • singleton coordinator;
  • coordinator + job pair.
• Domain-level diagnostics: "show me holding-devops" should show coordinator + related job instances together.

Net authoring verdict

Yes, this makes authoring more natural if implemented as an opinionated model:

If it has lifecycle, make it a flow instance.
If it is contained state, keep it in the primary entity.
If it is shared policy/learning, use a coordinator.

It is a clearer mental model than today's static-multi-entity shape. The biggest DX danger is over-modeling. We should not make authors constantly choose among three equally valid patterns. The platform should guide them toward the right one with syntax, verifier feedback, and scaffolds.

My bias: make flow instances cheap, make connect own cross-instance routing, make keyed maps safe for contained data, and push select_entity into the escape-hatch bucket. That gives authors a smaller, more coherent language.

[yazzaoui]

Decision: lock the authoring model

Let's lock the model around this principle:

The normal unit of durable workflow state is a flow instance with one primary state entity.

Everything else follows from deciding whether state is local contained data, an independently routable lifecycle, or shared coordination state.

---

Locked mental model

flow definition        = contract/type
flow instance          = one running process/case/job/service instance
primary entity         = durable state object for that flow instance
list/map field         = contained local state owned by that primary entity
child/template flow    = contained item promoted because it has lifecycle/routing
singleton coordinator  = shared policy, aggregate state, or cross-instance learning
connect                = parent-owned flow-instance wiring
interface              = reusable structural contract over pins/payloads/keys
analyzer               = expands compact authoring and rejects ambiguity

Solidity analogy:

contract type          = flow definition
deployed contract      = flow instance
contract storage       = primary entity
mapping/list storage   = typed map/list inside the primary entity
child contract         = child/template flow instance
factory/coordinator    = parent/coordinator flow
ABI/interface          = flow interface
compiler/typechecker   = analyzer

---

Authoring decision rubric

Authors should not choose among many overlapping routing/entity primitives. They should answer these questions:

1. Does this thing have independent states, timers, retries, callbacks, terminality, direct routing, or audit?

   Use a template flow instance.

2. Is it just data owned by the current process/service?

   Use a typed field/list/map on the primary entity.

3. Does it coordinate, summarize, or learn across many instances?

   Use a singleton coordinator, but only if there is real shared state or policy.

4. Does a map/list item start behaving like a routable recipient?

   Promote it to a child/template flow.

This keeps the happy path small:

flow instance -> primary entity -> contained fields

and keeps promotion explicit:

contained state becomes child flow only when it becomes an actor/process

---

Composition model

Cross-flow delivery routes between flow instances, not arbitrary entity rows inside static flows.

Template-to-template happy path:

# producer
pins:
  outputs:
    spend_requested:
      event: opco.spend_requested
      key: vertical_id

# receiver
instance:
  by: vertical_id

pins:
  inputs:
    spend_requested:
      event: opco.spend_requested

# parent
connect:
  - from: operating.spend_requested
    to: treasury.spend_requested

Analyzer expansion:

operating.spend_requested carries vertical_id
treasury instance.by = vertical_id
therefore route operating[vertical_id=X] -> treasury[vertical_id=X]

If names differ, the parent declares the adapter:

connect:
  - from: validation.spec_repo_requested
    to: spec-repo.repo_requested
    using:
      instance:
        source: vertical_id
        target: scope_id

The old composition/bind work still matters, but its public target should be revised:

keep: pin aliasing, import boundaries, resolver plumbing, fail-closed verification
revise: parent-owned correlate_by/cardinality as the main authoring syntax
prefer: connect + instance key mapping + explicit delivery

---

Delivery vs contained-state update

Keep these separate:

connect routes across flow instances
map/list operations update contained state inside the current primary entity

For process-shaped receivers:

producer output pin -> connect -> receiver instance key -> flow instance

For contained data:

event reaches owning flow instance -> typed map/list update on primary entity

Do not use connect to target arbitrary nested map entries. If a nested item needs cross-flow addressability, it has crossed the promotion line and should become a flow instance.

---

What stays as escape hatch

These should remain possible, but not be the normal authoring path:

• multi-entity static flows;
• select_entity / select_or_create_entity for external ingress, legacy migration, or explicit advanced database-like flows;
• producer emit.target for genuinely exotic dynamic routing;
• singleton coordinators for real shared state;
• typed maps/lists for contained data;
• custom adapters where interface or key names differ.

The analyzer should warn or reject normal in-topology use of select_entity once the new model is available.

---

Immediate platform surface to build toward

High-ROI primitives:

mode: template
instance:
  by: vertical_id
  on_missing: create | reject
  on_conflict: reject | reuse

Primary entity inference:

if exactly one entity type exists, infer it as the primary entity
otherwise require explicit entity: <Type>

Output pin identity:

pins:
  outputs:
    deploy_completed:
      event: deploy.completed
      key: vertical_id
      carries:
        - vertical_id
        - component_id
        - request_id

Parent composition:

connect:
  - from: holding-devops.deploy_completed
    to: operating.deploy_completed

Typed contained-state operations:

data_accumulation:
  writes:
    - target: entity.verticals[payload.vertical_id].active_jobs
      op: append
      value:
        request_id: event.id
        status: requested

We should avoid making authors hand-write arbitrary CEL/path hacks for maps/lists. If singleton+map is part of the model, typed map operations need to be real contract surface and verifier-owned.

---

Empire migration framing

Do not migrate every static flow blindly to template.

Use this classification:

Flow	Target shape
operating	Already template per vertical_id. Keep.
validation	Template per validation case. Strong yes.
scoring	Template per scored vertical/case. Strong yes.
discovery	Template per campaign likely.
signal-search	Template per campaign; signals as contained records/list.
treasury	Singleton + map or template per budget depending whether budget needs platform lifecycle. Spike needed.
holding-devops	Singleton + map if jobs stay simple; coordinator + job template if jobs need platform lifecycle. Spike needed.
lifecycle	Likely singleton + map unless marginal review needs real platform lifecycle.
spec-repo	Singleton + map or template per repo identity; depends on repo lifecycle/audit needs.
repo-scaffold	Ambiguous: template per request if artifact creation needs lifecycle; otherwise singleton + map.

The migration target is not a slogan. It depends on whether the keyed item needs platform machinery.

---

Analyzer obligations

The analyzer should own the boring and dangerous parts:

• infer primary entity when unique;
• verify instance.by fields exist and type-check;
• verify output key/carries fields exist in emitted payloads;
• verify connect maps producer keys to receiver instance keys;
• fail on ambiguous key matches;
• generate explicit route plans for runtime;
• warn when select_entity is used in normal in-topology delivery;
• verify typed map/list updates against declared shapes;
• expose expand and minimize views so compact authoring remains debuggable.

---

First pilots

Do two small pilots before broad Empire migration:

1. Template pilot: validation or scoring.

   Proves instance.by, primary entity inference, connect-to-template, and removal of normal select_entity.

2. Singleton + map pilot: treasury or holding-devops.

   Proves typed keyed-map access/update, shared agent context preservation, and whether contained-state authoring is ergonomic.

This pair proves both sides of the model. A single migration cannot validate the whole design.

---

Locked principle

Final wording I would use in the spec/design docs:

Swarm's default authoring model is flow-instance-centered. A normal flow instance owns one primary state entity. Cross-flow composition routes to flow instances through connect and instance keys. Lists and maps are contained state inside the primary entity. Authors promote contained state to child/template flows only when it needs independent lifecycle, routing, timers, retries, agents, or audit. Static multi-entity flows remain an advanced/legacy escape hatch, not the common path.

This keeps the model composable, analyzer-friendly, and teachable.