Skip to content

[codex] Track automation-applied outcome feedback#226

Merged
ScriptedAlchemy merged 3 commits into
masterfrom
codex/hermes-parity-r10-outcomes
Jul 2, 2026
Merged

[codex] Track automation-applied outcome feedback#226
ScriptedAlchemy merged 3 commits into
masterfrom
codex/hermes-parity-r10-outcomes

Conversation

@ScriptedAlchemy

Copy link
Copy Markdown
Owner

Summary

  • track post-approval managed skill outcomes from usage telemetry
  • track applied fact proposal recall/deletion outcomes from the memory store
  • include outcome feedback in automation artifacts/generated eval payloads and expose a read-only dashboard API
  • feed ignored post-approval outcomes into stale/improvement recommendations

Validation

  • cargo fmt --check
  • CARGO_TARGET_DIR=/home/zack/.cache/tracedecay-target-r10-outcomes cargo test --lib outcomes -- --nocapture
  • CARGO_TARGET_DIR=/home/zack/.cache/tracedecay-target-r10-outcomes cargo test --lib skill_usage::recommendations -- --nocapture
  • CARGO_TARGET_DIR=/home/zack/.cache/tracedecay-target-r10-outcomes cargo test --test agent_suite approval_stamps_approved_at_and_usage_baselines -- --nocapture
  • CARGO_TARGET_DIR=/home/zack/.cache/tracedecay-target-r10-outcomes cargo test --test dashboard_api_test automation_outcomes_endpoint_returns_live_read_only_outcomes -- --nocapture
  • CARGO_TARGET_DIR=/home/zack/.cache/tracedecay-target-r10-outcomes cargo check

Notes

  • Branch was merged with current origin/master before push.
  • cargo check/test emitted existing vendor libsql warnings only.

ScriptedAlchemy added 3 commits July 2, 2026 08:28
Add an outcomes module that measures whether approved changes were
actually good: approved managed skills get adoption verdicts
(adopted/ignored/too_early) from usage-ledger deltas anchored on a new
additive approved_at timestamp plus approval-time view/use baselines,
and applied fact proposals get recall verdicts (recalled_and_helpful/
recalled/never_recalled/deleted) from the memory store's post-apply
trajectory.

Outcomes persist as a per-project snapshot refreshed at the start of
session-reflector and skill-writer runs, feed the artifact chain (an
applied_change_outcomes section in the feedback payload and outcome-
derived generated-eval entries), strengthen stale/improvement skill
recommendations when an approved skill is ignored, and are exposed
read-only at GET /api/automation/outcomes.
@changeset-bot

changeset-bot Bot commented Jul 2, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 33123a3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy ScriptedAlchemy marked this pull request as ready for review July 2, 2026 09:02
@ScriptedAlchemy ScriptedAlchemy merged commit 2894baf into master Jul 2, 2026
14 checks passed
@ScriptedAlchemy ScriptedAlchemy deleted the codex/hermes-parity-r10-outcomes branch July 2, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant