Skip to content

Wave 9 #123.d.iv — Phase 7 Gate D (agent integration) acceptance test - #125

Open
bernardc6 wants to merge 1 commit into
mainfrom
subagent/phase7-gate-d-acceptance
Open

Wave 9 #123.d.iv — Phase 7 Gate D (agent integration) acceptance test#125
bernardc6 wants to merge 1 commit into
mainfrom
subagent/phase7-gate-d-acceptance

Conversation

@bernardc6

Copy link
Copy Markdown
Collaborator

Wave 9 #123.d.iv — Phase 7 Gate D (agent integration) acceptance test

Context:
Phase 7 of the 2026-08-03 directive requires automated acceptance
tests for the 6 release gates. This slice ships Gate D (agent
integration), the fourth of those gates, verifying the canonical
contract verbatim: "OpenClaw reads model, supported commands
work, dangerous ops unavailable, confirmation enforced, every
action logged, agent failure cannot disrupt van". The Gate D
bash test is the canonical contract (12 real assertions covering
Bearer-auth + 12-tile model read + capability allowlist + 6-digit
confirmation + SHA-256-chained audit log + tamper detection +
agent-failure isolation + multi-tenant isolation + reboot-
survives); the pytest rig is the fast in-process mock coverage
(~33 tests) that runs anywhere; the GH Actions workflow mirrors
the Gate A + Gate B + Gate C workflow shape (push + PR + weekly
Monday 06:00 UTC + manual dispatch + optional HAS_OPENCLAW_API
step).

Scope (per doctrine: minimal + additive + isolated to these files):
NEW .github/workflows/acceptance-gate-d.yml (CI workflow — push
+ PR + weekly Monday 06:00 UTC + manual dispatch)
NEW scripts/tests/acceptance/gate_d_agent_integration.sh (~830
LOC bash test, 12 stage assertions, SHA-256 chain is real
Python hashlib (not stubbed), idempotent cleanup trap on
EXIT, the script is runnable in --mock mode on any host)
NEW scripts/tests/acceptance/test_gate_d_agent_integration.py
(~1050 LOC pytest wrapper, 33 tests, all inlined fixtures
— does NOT depend on scripts/tests/acceptance/conftest.py
because Gate A + Gate B's conftest is on unmerged PRs and
Bernard will merge those separately)
NEW scripts/checks/gate-d-agent-integration-smoke.sh (~290 LOC
bash smoke — 39 structural assertions on the rig, mirrors
the Gate B smoke pattern)
NEW docs/runbooks/automated-acceptance-tests-gate-d.md (IKEA
5-step user runbook — operator→vanlifertable at the bottom)
NEW docs/runbooks/automated-acceptance-tests.md (umbrella
runbook that includes the additive Gate D section)
MOD scripts/check.sh (+11 lines: run_if_present block for the
Gate D smoke check, mirrors the Gate B wire-up pattern,
idempotent skip when not present, NOT a hard-required chain
entry)

NOT touched: Gate A/B/C's files (gate_a_clean_install.sh,
gate_b_connection_flow.sh, gate_c_dashboard_reliability.sh,
test_gate_a_clean_install.py, test_gate_b_connection_flow.py,
test_gate_c_dashboard_reliability.py, .github/workflows/
acceptance-gate-{a,b,c}.yml, scripts/tests/acceptance/conftest.py,
scripts/tests/acceptance/init.py) — those live on unmerged
PRs #115 / #120 and Bernard will merge them separately. The
pytest rig is fully self-contained (inline MagicMock fixtures);
the bash test is self-contained (no external dependencies
beyond bash + python3 + sha256sum); the smoke check is self-
contained. The slice is runnable WITHOUT Gate A/B/C being merged.

Doctrine (Bernard, 2026-08-04 "must not fail + super intuitive

  • critical infrastructure") applied:
    • Verification is mandatory: every bash stage has a real
      assertion (the SHA-256 chain is computed via real Python
      hashlib, not stubbed); every pytest test reads the bash
      script's source + asserts the canonical contract shape +
      the 12 rc_openclaw_api_* tile ids.
    • Plain-English errors: 39 fail() messages across the 12
      stages, every one carrying a recovery hint (check / verify
      / look at / see / open / reload / restart). The canonical
      "Action not allowed without explicit confirmation" phrase
      appears in Step 3 (allowlist refusal) + Step 4 (execute-
      without-confirm refusal) per the directive.
    • Idempotent: re-running the bash test produces the same end
      state (the cleanup trap removes the mock agent + mock audit
      log + mock fixtures on EXIT; Step 10 asserts the SHA-256
      is stable across re-reads; test_idempotency_end_to_end_rerun
      runs the script twice and asserts both runs report PASSED).
    • Tier discipline (tier-c — community test recipe, not a
      RoamCore-native test engine).
    • User-facing repo hygiene: IKEA 5-step runbook for the
      Gate D-specific doc; the bash test + pytest rig + GH
      Actions workflow + smoke check are developer plumbing and
      live in scripts/. No SUPERSEDED in user tree. No
      Cron-handoff references in user copy.

GOLDEN.md alignment (the discipline block mandates this is
quoted in the commit body):
Product principles served:
- P2 (mission-critical connectivity): the gate enforces the
"agent failure MUST NOT disrupt the van" guarantee
(Step 7 — kill the agent mid-action, van continues, recovery
automation clears the failure state).
- P5 (AI as trust-first summary/translation layer): the gate
enforces the trust-first contract end-to-end (every action
is checked in Step 3 + confirmed in Step 4 + logged in
Step 5 + tamper-evident in Step 6 — the canonical
audit-trail-by-design pattern).
- P6 (OpenClaw as a first-class citizen): the gate IS the
agent-integration acceptance — every contract element of
the canonical 12 rc_openclaw_api_* tile surface is
exercised (auth + model read + capability allowlist +
confirmation + audit chain + tamper detection + reboot-
survives).
Engineering principles respected:
- E1 (customer-facing repo): IKEA runbook in docs/;
bash test + pytest rig + GH Actions workflow + smoke
check are internal plumbing in scripts/.
- E2 (documentation-driven): the bash test is the runtime
contract; the pytest rig is the test-time contract; the
runbook is the user-time contract — three sources of
truth in lockstep.
- E3 (backup + rollback discipline): no Proxmox / HA /
OpenWrt / networking change; pure repo-local code +
data + docs.
- E4 (git fast mode): direct-to-branch push on
subagent/phase7-gate-d-acceptance; PR is the report
mechanism; one PR for Bernard per protocol.
- E5 (no vmbr0): not applicable — no networking config
touched.
- E6 (HACS-friendly layout): not applicable — the
acceptance test is upstream pytest + GitHub Actions,
not a HACS custom component.
- E7 (rc-entity-naming.md): every entity follows the
canonical rc_openclaw_api_* convention (12 tile ids,
no vendor names in contract entity ids, the bash
test's Step 12 grep proves this).
Anti-patterns explicitly avoided:
- ❌ Hand-configuring anything instead of using capability
detection: the rig uses the canonical 12-tile vehicle
model + the canonical allowlist, NOT hand-rolled mocks.
- ❌ Touching vmbr0 without explicit Bernard override: no
networking config touched.
- ❌ Committing secrets or HA tokens to the repo: every
"token" reference in the rig carries the canonical
'mock-token-tenant-' prefix; Step 12 has a secrets-grep
that returns empty; RC_API_TOKEN comes from environment
or stdin (input_text mode: password), NEVER hardcoded.
- ❌ Adding "advanced" features before the novice path is
solid: Gate D IS the novice-path safety rail — the
helper app can help with everyday things without ever
doing something risky.
- ❌ Wide PRs that mix networking + UI + infra: pure
test-infra slice, 6 NEW + 1 MOD files, no cross-
cutting changes to the runtime surface, no UI changes,
no networking changes.
- ❌ Pulling in unrelated project context: strictly
Gate D / Phase 7 scope; nothing touches RoamCore tiles,
Power runtime code, or PWA runtime code.
- ❌ Putting internal engineering logs on the public
GitHub: the IKEA runbook is operator-facing; the bash
test + the pytest rig + the GH Actions workflow +
the smoke check are developer-facing; neither leaks
internal cron-handoff references.

User-facing:
Verifies, automatically, that the OpenClaw helper can read
your dashboard data safely and that every action it tries
to take is checked, confirmed, and recorded — so the helper
can keep helping you without ever taking an action you
didn't approve.

Verification:
- bash scripts/tests/acceptance/gate_d_agent_integration.sh
--mock → all 12 stages PASS in <3 s on any host with
bash + python3; exit 0; the SHA-256 chain is real (real
Python hashlib, not stubbed); the cleanup trap fires on
EXIT (no fixture leaks).
- python3 -m pytest scripts/tests/acceptance/
test_gate_d_agent_integration.py -v → 33/33 PASS in
~2.5 s on this host (all inlined fixtures; no live API
calls; mock sha256 is real Python hashlib).
- bash scripts/checks/gate-d-agent-integration-smoke.sh →
39/39 PASS in <1 s; exit 0; covers the 12-stage layout +
SHA-256 chain presence + tamper detection + 39 plain-
English fail() calls + canonical denial phrase + no hard-
coded secrets + rc-entity-naming honored + IKEA 5-step
runbook shape + no Wave/tier/PR/cron jargon in user
copy + GH Actions workflow shape + scripts/check.sh wire-
up + umbrella-runbook Gate D section.
- bash scripts/check.sh --core-only → exit 0 GREEN (the
Gate D smoke runs as part of the idempotent run_if_present
chain; missing-script paths skip silently).
- bash scripts/tests/acceptance/gate_d_agent_integration.sh
--mock (re-run) → exit 0 — idempotency contract holds
(Step 10 asserts SHA-256 stable across re-reads;
test_idempotency_end_to_end_rerun runs the script twice
and asserts both runs report PASSED).
- No secrets in any file (Step 12 secrets-grep returns
empty; mock tokens carry the canonical 'mock-token-tenant-'
prefix).

Rollback:
- git revert restores main to the state before
this slice (the Gate A + Gate B + Gate C foundation
remains intact, since this slice adds + extends, never
removes).
- The bash test's cleanup trap fires on EXIT — even if
this slice is reverted while a CI run is mid-flight,
no acceptance-rig state leaks to disk.
- The pytest fixtures are inline (function-scoped via
@pytest.fixture, never global); reverting this slice
restores the prior tests/ without orphan fixtures in
any other test directory.
- The scripts/check.sh change is one new run_if_present
block; reverting the slice removes it cleanly with no
impact on the rest of the check chain.

Context:
  Phase 7 of the 2026-08-03 directive requires automated acceptance
  tests for the 6 release gates. This slice ships Gate D (agent
  integration), the fourth of those gates, verifying the canonical
  contract verbatim: "OpenClaw reads model, supported commands
  work, dangerous ops unavailable, confirmation enforced, every
  action logged, agent failure cannot disrupt van". The Gate D
  bash test is the canonical contract (12 real assertions covering
  Bearer-auth + 12-tile model read + capability allowlist + 6-digit
  confirmation + SHA-256-chained audit log + tamper detection +
  agent-failure isolation + multi-tenant isolation + reboot-
  survives); the pytest rig is the fast in-process mock coverage
  (~33 tests) that runs anywhere; the GH Actions workflow mirrors
  the Gate A + Gate B + Gate C workflow shape (push + PR + weekly
  Monday 06:00 UTC + manual dispatch + optional HAS_OPENCLAW_API
  step).

  Scope (per doctrine: minimal + additive + isolated to these files):
    NEW .github/workflows/acceptance-gate-d.yml (CI workflow — push
      + PR + weekly Monday 06:00 UTC + manual dispatch)
    NEW scripts/tests/acceptance/gate_d_agent_integration.sh (~830
      LOC bash test, 12 stage assertions, SHA-256 chain is real
      Python hashlib (not stubbed), idempotent cleanup trap on
      EXIT, the script is runnable in --mock mode on any host)
    NEW scripts/tests/acceptance/test_gate_d_agent_integration.py
      (~1050 LOC pytest wrapper, 33 tests, all inlined fixtures
      — does NOT depend on scripts/tests/acceptance/conftest.py
      because Gate A + Gate B's conftest is on unmerged PRs and
      Bernard will merge those separately)
    NEW scripts/checks/gate-d-agent-integration-smoke.sh (~290 LOC
      bash smoke — 39 structural assertions on the rig, mirrors
      the Gate B smoke pattern)
    NEW docs/runbooks/automated-acceptance-tests-gate-d.md (IKEA
      5-step user runbook — operator→vanlifertable at the bottom)
    NEW docs/runbooks/automated-acceptance-tests.md (umbrella
      runbook that includes the additive Gate D section)
    MOD scripts/check.sh (+11 lines: run_if_present block for the
      Gate D smoke check, mirrors the Gate B wire-up pattern,
      idempotent skip when not present, NOT a hard-required chain
      entry)

  NOT touched: Gate A/B/C's files (gate_a_clean_install.sh,
  gate_b_connection_flow.sh, gate_c_dashboard_reliability.sh,
  test_gate_a_clean_install.py, test_gate_b_connection_flow.py,
  test_gate_c_dashboard_reliability.py, .github/workflows/
  acceptance-gate-{a,b,c}.yml, scripts/tests/acceptance/conftest.py,
  scripts/tests/acceptance/__init__.py) — those live on unmerged
  PRs #115 / #120 and Bernard will merge them separately. The
  pytest rig is fully self-contained (inline MagicMock fixtures);
  the bash test is self-contained (no external dependencies
  beyond bash + python3 + sha256sum); the smoke check is self-
  contained. The slice is runnable WITHOUT Gate A/B/C being merged.

  Doctrine (Bernard, 2026-08-04 "must not fail + super intuitive
  + critical infrastructure") applied:
    - Verification is mandatory: every bash stage has a real
      assertion (the SHA-256 chain is computed via real Python
      hashlib, not stubbed); every pytest test reads the bash
      script's source + asserts the canonical contract shape +
      the 12 rc_openclaw_api_* tile ids.
    - Plain-English errors: 39 fail() messages across the 12
      stages, every one carrying a recovery hint (check / verify
      / look at / see / open / reload / restart). The canonical
      "Action not allowed without explicit confirmation" phrase
      appears in Step 3 (allowlist refusal) + Step 4 (execute-
      without-confirm refusal) per the directive.
    - Idempotent: re-running the bash test produces the same end
      state (the cleanup trap removes the mock agent + mock audit
      log + mock fixtures on EXIT; Step 10 asserts the SHA-256
      is stable across re-reads; test_idempotency_end_to_end_rerun
      runs the script twice and asserts both runs report PASSED).
    - Tier discipline (tier-c — community test recipe, not a
      RoamCore-native test engine).
    - User-facing repo hygiene: IKEA 5-step runbook for the
      Gate D-specific doc; the bash test + pytest rig + GH
      Actions workflow + smoke check are developer plumbing and
      live in scripts/. No SUPERSEDED in user tree. No
      Cron-handoff references in user copy.

  GOLDEN.md alignment (the discipline block mandates this is
  quoted in the commit body):
    Product principles served:
      - P2 (mission-critical connectivity): the gate enforces the
        "agent failure MUST NOT disrupt the van" guarantee
        (Step 7 — kill the agent mid-action, van continues, recovery
        automation clears the failure state).
      - P5 (AI as trust-first summary/translation layer): the gate
        enforces the trust-first contract end-to-end (every action
        is checked in Step 3 + confirmed in Step 4 + logged in
        Step 5 + tamper-evident in Step 6 — the canonical
        audit-trail-by-design pattern).
      - P6 (OpenClaw as a first-class citizen): the gate IS the
        agent-integration acceptance — every contract element of
        the canonical 12 rc_openclaw_api_* tile surface is
        exercised (auth + model read + capability allowlist +
        confirmation + audit chain + tamper detection + reboot-
        survives).
    Engineering principles respected:
      - E1 (customer-facing repo): IKEA runbook in docs/;
        bash test + pytest rig + GH Actions workflow + smoke
        check are internal plumbing in scripts/.
      - E2 (documentation-driven): the bash test is the runtime
        contract; the pytest rig is the test-time contract; the
        runbook is the user-time contract — three sources of
        truth in lockstep.
      - E3 (backup + rollback discipline): no Proxmox / HA /
        OpenWrt / networking change; pure repo-local code +
        data + docs.
      - E4 (git fast mode): direct-to-branch push on
        subagent/phase7-gate-d-acceptance; PR is the report
        mechanism; one PR for Bernard per protocol.
      - E5 (no vmbr0): not applicable — no networking config
        touched.
      - E6 (HACS-friendly layout): not applicable — the
        acceptance test is upstream pytest + GitHub Actions,
        not a HACS custom component.
      - E7 (rc-entity-naming.md): every entity follows the
        canonical rc_openclaw_api_* convention (12 tile ids,
        no vendor names in contract entity ids, the bash
        test's Step 12 grep proves this).
    Anti-patterns explicitly avoided:
      - ❌ Hand-configuring anything instead of using capability
        detection: the rig uses the canonical 12-tile vehicle
        model + the canonical allowlist, NOT hand-rolled mocks.
      - ❌ Touching vmbr0 without explicit Bernard override: no
        networking config touched.
      - ❌ Committing secrets or HA tokens to the repo: every
        "token" reference in the rig carries the canonical
        'mock-token-tenant-' prefix; Step 12 has a secrets-grep
        that returns empty; RC_API_TOKEN comes from environment
        or stdin (input_text mode: password), NEVER hardcoded.
      - ❌ Adding "advanced" features before the novice path is
        solid: Gate D IS the novice-path safety rail — the
        helper app can help with everyday things without ever
        doing something risky.
      - ❌ Wide PRs that mix networking + UI + infra: pure
        test-infra slice, 6 NEW + 1 MOD files, no cross-
        cutting changes to the runtime surface, no UI changes,
        no networking changes.
      - ❌ Pulling in unrelated project context: strictly
        Gate D / Phase 7 scope; nothing touches RoamCore tiles,
        Power runtime code, or PWA runtime code.
      - ❌ Putting internal engineering logs on the public
        GitHub: the IKEA runbook is operator-facing; the bash
        test + the pytest rig + the GH Actions workflow +
        the smoke check are developer-facing; neither leaks
        internal cron-handoff references.

  User-facing:
    Verifies, automatically, that the OpenClaw helper can read
    your dashboard data safely and that every action it tries
    to take is checked, confirmed, and recorded — so the helper
    can keep helping you without ever taking an action you
    didn't approve.

  Verification:
    - bash scripts/tests/acceptance/gate_d_agent_integration.sh
      --mock → all 12 stages PASS in <3 s on any host with
      bash + python3; exit 0; the SHA-256 chain is real (real
      Python hashlib, not stubbed); the cleanup trap fires on
      EXIT (no fixture leaks).
    - python3 -m pytest scripts/tests/acceptance/
      test_gate_d_agent_integration.py -v → 33/33 PASS in
      ~2.5 s on this host (all inlined fixtures; no live API
      calls; mock sha256 is real Python hashlib).
    - bash scripts/checks/gate-d-agent-integration-smoke.sh →
      39/39 PASS in <1 s; exit 0; covers the 12-stage layout +
      SHA-256 chain presence + tamper detection + 39 plain-
      English fail() calls + canonical denial phrase + no hard-
      coded secrets + rc-entity-naming honored + IKEA 5-step
      runbook shape + no Wave/tier/PR/cron jargon in user
      copy + GH Actions workflow shape + scripts/check.sh wire-
      up + umbrella-runbook Gate D section.
    - bash scripts/check.sh --core-only → exit 0 GREEN (the
      Gate D smoke runs as part of the idempotent run_if_present
      chain; missing-script paths skip silently).
    - bash scripts/tests/acceptance/gate_d_agent_integration.sh
      --mock (re-run) → exit 0 — idempotency contract holds
      (Step 10 asserts SHA-256 stable across re-reads;
      test_idempotency_end_to_end_rerun runs the script twice
      and asserts both runs report PASSED).
    - No secrets in any file (Step 12 secrets-grep returns
      empty; mock tokens carry the canonical 'mock-token-tenant-'
      prefix).

  Rollback:
    - git revert <this-sha> restores main to the state before
      this slice (the Gate A + Gate B + Gate C foundation
      remains intact, since this slice adds + extends, never
      removes).
    - The bash test's cleanup trap fires on EXIT — even if
      this slice is reverted while a CI run is mid-flight,
      no acceptance-rig state leaks to disk.
    - The pytest fixtures are inline (function-scoped via
      @pytest.fixture, never global); reverting this slice
      restores the prior tests/ without orphan fixtures in
      any other test directory.
    - The scripts/check.sh change is one new run_if_present
      block; reverting the slice removes it cleanly with no
      impact on the rest of the check chain.
r"(password|api[_-]?key|secret|token)\s*=\s*[a-zA-Z0-9_-]{16,}",
re.IGNORECASE,
)
matches = secret_pattern.findall(text)
bernardc6 added a commit that referenced this pull request Aug 16, 2026
Orchestrator merge per RoamCore prompt ('orchestrator merges sequentially after check.sh --core-only is green').

**Slice:** Wave 9 #125 — Phase 1 catalog UI proper: render 28 connection cards via format_connection_card() + per-connection state field.

**GOLDEN.md alignment (re-verified this cycle at 07:03Z 2026-08-16):**
- P1 (Novice-first UX): cards with state chips + tier pills + Connect buttons are the glanceable surface Bernard asked for in chat #7588-7591 ('app-store-style interface, very clearly laid out, easy to use for novices').
- P6 (OpenClaw first-class): data-connection-name='<title>' machine-readable on every card so OpenClaw can drive the catalog.
- E1 (Customer-facing repo): docs/catalog/index.md IS the public surface; no internal plumbing leaks here.
- E2 (Documentation-driven): the catalog IS the docs, now regenerated from connections/<slug>/connection.yml (no hand-maintained drift).
- E4 (Git fast mode): direct-to-main on subagent/<slug>; orchestrator merge is the protocol.
- E7 (Naming follows rc-entity-naming.md): kebab-case rc-state-chip--*/rc-tier--* classes.

**Anti-patterns avoided:**
- ❌ Wide PRs that mix networking + UI + infra (pure docs/Python/tests scope, 46 files / +1876/-392)
- ❌ Hand-editing generated files (catalog is a generator output; one source of truth)
- ❌ Committing secrets (catalog-ui-cards-smoke.sh assertion 4 regex-matches JWT/api-key-shaped literals)
- ❌ Adding advanced features before novice path is solid (this IS the novice path foundation)

**Verification:** bash scripts/check.sh --core-only GREEN exit 0 (PASS: 26 / FAIL: 0), re-verified this slot before merge. PR #129 was already MERGEABLE / CLEAN per gh pr view.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant