Skip to content

[2/5] Observe locked SDK baselines accurately#52

Draft
ebarti wants to merge 1 commit into
agent/sdk-evolution-demo-reliability-planfrom
agent/sdk-evolution-locked-baselines
Draft

[2/5] Observe locked SDK baselines accurately#52
ebarti wants to merge 1 commit into
agent/sdk-evolution-demo-reliability-planfrom
agent/sdk-evolution-locked-baselines

Conversation

@ebarti

@ebarti ebarti commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • treat uv.lock as the authoritative SDK baseline during explicit candidate inspection
  • isolate locked baselines when the matching SDK is absent or the ambient environment has drifted
  • keep non-opt-in runs honest by recording the actual ambient version instead of stamping it as locked
  • convert isolated API snapshot setup, install, execution, timeout, and malformed-output failures into bounded evidence

Stack

Review boundary

This layer repairs baseline provenance and snapshot collection only. Behavior-probe subprocess failures and the final fail > incomplete > changed > pass assessment are intentionally implemented together in the next layer so incomplete evidence cannot temporarily appear green.

Validation

  • PYTHONPATH=. .venv/bin/python -m pytest -q — 421 passed, 12 skipped
  • .venv/bin/ruff check .
  • .venv/bin/mypy
  • uv lock --check
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant