Skip to content

Stabilize live validation - #42

Merged
dimavrem22 merged 1 commit into
mainfrom
fix/live-ci-stability
Aug 7, 2026
Merged

Stabilize live validation#42
dimavrem22 merged 1 commit into
mainfrom
fix/live-ci-stability

Conversation

@dimavrem22

Copy link
Copy Markdown
Contributor

Executive Summary

This PR makes live Codex validation strict, repeatable, and safe to diagnose.

  • Correlates one current call record per identity and verifies each transcript on its authoritative owner.
  • Uses a compact run-specific hosted-action marker while retaining exact action, recipient, and single-delivery gates.
  • Bounds setup retries and reports content-free failure state without publishing live logs.

Description

Voice scenarios now sweep interrupted matching calls before each trigger, reject delayed or ambiguous records using a request-time lower bound, require a stable driver/AUT pair, and clean up every post-snapshot record on both identities. Driver-owned records prove caller speech; AUT-owned records must contain both parties and return the AUT-local agent speech. Speech-mode and hosted-action metadata are read from that exact AUT call.

The hosted action uses three distinct speech-safe words derived from the full workflow run and attempt. Host installation retries are limited to setup, and live messages, calls, and agent turns remain single-attempt. Failure workflows emit bounded state predicates instead of raw runtime logs or artifacts.

Reason

Scheduled validation could fail after querying the non-authoritative call transcript or after a long spoken marker was shortened while being persisted as an action. Interrupted calls and unbounded setup failures also made repeated validation less reliable.

Decisions

  • Call ownership: Validate caller speech on the driver record and require two-way conversation plus agent-local speech on the exact AUT record because transcript parties are identity-relative.
  • Correlation: Keep plugin-owned snapshot, request-time, timestamp, uniqueness, and stability checks because they reject stale or ambiguous rows without changing a product contract.
  • Retry boundary: Retry only host installation; each live message, call, action, and agent turn remains single-attempt so failures stay observable.
  • Diagnostics: Publish only bounded state and counts because raw live logs can contain message and call content.

Testing

  • Full offline suite: 452 passed, 23 skipped with skips limited to live-environment scenarios.
  • Codex host contract with codex-cli 0.147.0: 5 passed.
  • Voice and CI helper regressions: 21 passed.
  • Changed Python files: Ruff checks passed.
  • Workflow YAML parsing, installer shell syntax, and git diff --check: passed.

@dimavrem22

Copy link
Copy Markdown
Contributor Author

Validation is complete on fixed commit 76c8d16feb155cbca33212c6c92adcc47522509b.

Three consecutive complete cycles passed:

  1. Fast checks + full live stack
  2. Fast checks + full live stack
  3. Fast checks + full live stack

Every cycle passed both channel modes, all four A2A scenarios, all three voice scenarios, external events, and the full-stack aggregator. The schedule-only failure notifier was intentionally skipped on these PR/manual events; no test lane was skipped.

The live stack still runs on PRs but is not a required merge check. Required checks remain unit (3.11), unit (3.12), and contract-pr, with strict up-to-date enforcement disabled.

@dimavrem22
dimavrem22 merged commit f5e77d1 into main Aug 7, 2026
45 checks passed
@dimavrem22
dimavrem22 deleted the fix/live-ci-stability branch August 7, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant