Skip to content

feat(examples): software_factory example; drop Postgres/Redis from framework#244

Merged
miguelgfierro merged 2 commits into
issue-147-pipeline-evolutionfrom
feat/software-factory-example
May 28, 2026
Merged

feat(examples): software_factory example; drop Postgres/Redis from framework#244
miguelgfierro merged 2 commits into
issue-147-pipeline-evolutionfrom
feat/software-factory-example

Conversation

@miguelgfierro
Copy link
Copy Markdown
Contributor

What

Adds a self-contained software-factory example folder and pulls the Postgres/Redis checkpointer + PostgresAuditLog out of the framework.

New: examples/software_factory/

A small example package showing the headline state-mode features in one place:

Feature Where it shows up
State + reducers state.pyextend on qa_feedback so feedback accumulates across loop iterations
Branching with router pipeline.py — single .branch(\"qa\", qa_router) call
Cycle with recursion_limit QA fail → codegen with recursion_limit=3
Checkpoint + resume builder raises on its first call; invoke(run_id=...) resumes
Observability handler progress.py — prints per-node events
Plug-and-play backend templates checkpointers/{postgres,redis}.py, audit/postgres.py — flat ~50 LOC each, no shared base class

__main__.py defaults to FileCheckpointer; `FIREFLY_CKPT=postgres` (with `PG_DSN`) or `FIREFLY_CKPT=redis` (with `REDIS_URL`) swaps in the templates. All agents are deterministic stubs — runs offline.

Why codegen and builder are separate nodes (not collapsed)

In stub form they look redundant. They're kept distinct because they model two different failure-recovery patterns the state-mode API supports:

  • Builder fails = transient (dep install timeout, flake) → engine checkpoints the failure → `invoke(run_id=...)` retries builder in place. No cycle.
  • QA fails = substantive (tests don't pass) → `qa_router` returns `"codegen"` → cycle re-enters codegen with `qa_feedback` in state.

Collapsing them loses one of the two recovery stories.

Framework cleanup (same PR)

Removed because there are zero internal callers, apps already have their own connection pools, and the Checkpointer Protocol is small enough (three methods) that a concrete implementation is ~50 LOC of example code:

  • `PostgresCheckpointer`, `RedisCheckpointer` (pipeline/checkpoint.py)
  • `PostgresAuditLog` (pipeline/audit.py)
  • `PsycopgBackend` helper (pipeline/_psycopg_backend.py — no remaining consumers, deleted)
  • `psycopg[binary]` dropped from the `[postgres]` extra in pyproject.toml
  • Mocked test classes for the three removed backends

`LoggingAuditLog` and `OtelAuditLog` stay in the framework: they wrap non-trivial stdlib/OTel APIs and serve the "audit goes to my log aggregator without thinking" default path. `QueryableAuditLog` Protocol split stays (`FileAuditLog` in framework + the Postgres example template both implement it).

Example tidy-up

  • `examples/pipeline_state.py`: drops the shallow `run_software_factory` and `run_software_factory_postgres` scenarios. Keeps the branching, map-reduce, and HITL+audit scenes as quickstart snippets. Doc-string and `main()` updated accordingly.
  • `examples/README.md`: adds the new entry, links to the folder.

Diff stats

```
22 files changed, 846 insertions(+), 871 deletions(-)
```

Net negative LOC overall.

Verification

```bash
source ~/.venvs/firefly/bin/activate
pytest tests/unit/ # 1531 passed
pytest examples/software_factory/tests/ # 1 passed
ruff check / ruff format --check # clean
pyright fireflyframework_agentic/pipeline/ examples/software_factory/ # 0 errors
python -m examples.software_factory # runs through QA loop + resume; release v2026.05.28
python examples/pipeline_state.py # all three quickstart scenes still work
```

…ramework

Adds a self-contained software-factory example package under
examples/software_factory/ that exercises the headline state-mode features:

* state + reducers (extend on qa_feedback)
* branching with router + cycle (qa fail → codegen, recursion_limit=3)
* checkpoint + resume on a transient builder failure
* StatePipelineEventHandler progress output

Keeps codegen and builder as separate nodes deliberately: builder models a
transient failure recovered via checkpoint, qa models a substantive failure
recovered via cycle back to codegen — two different recovery patterns.

Includes plug-and-play Checkpointer Protocol implementations under
checkpointers/{postgres,redis}.py and a QueryableAuditLog implementation
under audit/postgres.py. Each is a flat ~50–80 LOC class against a
caller-supplied connection. FIREFLY_CKPT env var swaps backends.

Same PR removes the matching in-framework backends and their machinery:

* PostgresCheckpointer, RedisCheckpointer (pipeline/checkpoint.py)
* PostgresAuditLog (pipeline/audit.py)
* PsycopgBackend helper (pipeline/_psycopg_backend.py) — no remaining consumers
* psycopg[binary] dep dropped from [postgres] extra in pyproject.toml
* Postgres/Redis mocked test classes (tests/unit/pipeline/test_*.py)

The Checkpointer and AuditLog Protocols, FileCheckpointer, FileAuditLog,
LoggingAuditLog, and OtelAuditLog stay in the framework — concrete backends
that wrap non-trivial APIs (stdlib logging, OTel logs SDK) keep their place.

Also drops the run_software_factory and run_software_factory_postgres
scenarios from examples/pipeline_state.py (the deep walkthrough now lives in
the dedicated folder) and links the new folder from examples/README.md.

Verification: pytest tests/unit/ → 1531 passed; pytest examples/software_factory/tests/ → 1 passed; ruff + pyright clean.
Match the layout used by other example packages (tests/examples/corpus_search/
is the established pattern). README links to the new test location.
@miguelgfierro miguelgfierro merged commit f0640e2 into issue-147-pipeline-evolution May 28, 2026
@miguelgfierro miguelgfierro deleted the feat/software-factory-example branch May 28, 2026 11:05
ancongui pushed a commit that referenced this pull request May 31, 2026
…xample

feat(examples): software_factory example; drop Postgres/Redis from framework
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant