Skip to content

ASSERT v0.3.0 Release

Latest

Choose a tag to compare

@tangym tangym released this 04 Sep 22:07
04b713f

ASSERT v0.3.0

ASSERT v0.3.0 improves dependency isolation, evaluation authoring, sandbox security, and result exploration. This release includes breaking changes to optional dependency names and the default multi-turn inference budget.

Highlights

  • Evidence-backed eval config generation in the run-assert-eval skill, including source research, citation review, and one-risk-per-suite output.
  • Host-authoritative sandbox action mediation, with case-correlated decisions and host-owned evidence.
  • Atomic behavior presets with broader built-in risk coverage and CI enforcement.
  • Cleaner dependency ownership between ASSERT capabilities and target frameworks.
  • Improved viewer tables and metric dropdown behavior on constrained screens.

Breaking changes

Optional dependencies

ASSERT extras now represent product capabilities instead of repository examples:

Previous Replacement
assert-ai[otel] assert-ai[phoenix] plus the target framework's OpenInference instrumentor
assert-ai[azure-aad] assert-ai[azure-auth]
assert-ai[langgraph] Install target-owned dependencies
assert-ai[dspy] Install target-owned dependencies
assert-ai[examples] Use each example's requirements.txt
assert-ai[regression] Use the repository development dependency group

The all extra now installs all ASSERT capabilities: Phoenix, analysis, embeddings, Azure authentication, and ACS integration.

Multi-turn inference

The default pipeline.inference.max_turns value changes from 10 to 6.

Set the value explicitly to retain previous behavior:

pipeline:
  inference:
    max_turns: 10

Results generated before and after this change may not be directly comparable for multi-turn behaviors.

Security and reliability

  • Sandbox tool decisions can be evaluated and recorded by an authenticated host service.
  • Network evidence is stored outside the evaluated target's writable mount.
  • Sandbox mediation now correlates policy, mocks, and evidence by test case.
  • Real Docker containment tests run in CI.
  • LiteLLM and frontend dependencies were updated to patched versions.
  • The HTTP endpoint target now declares its direct aiohttp dependency.

Additional improvements

  • assert-ai --version now reports the installed package version.
  • Traced callable targets emit an actionable warning when no spans are captured.
  • Analysis, Phoenix, embeddings, and framework dependencies are isolated into compatible environments.
  • Documented example environments receive clean-install and pip check coverage.

Upgrade

python -m pip install --upgrade "assert-ai==0.3.0"

Review the breaking changes above before upgrading existing evaluation environments.