asibench 0.2.0

ASI-Bench 0.2.0 includes framework improvements and fixes since 0.1.6.

• Add bounded parallel local scoring, safer input/output staging, task-scoped evaluator environments, and shared scoring logic. Correct score divisors and evaluator failure classification.
• Add task-level Docker runtime services with isolated networks, health checks, and cleanup. Fix timeout classification.
• Preserve model-call lifecycle evidence and report explicit observability gaps rather than inferring empty-response rates.
• Improve Responses protocol conversion and signed reasoning replay, reject lossy or malformed requests, and preserve judge temperature at compatible endpoints.
• Update scientific MCP configurations and expand repository smoke tests and example tasks. These scripts and example tasks remain repository resources; the PyPI artifacts contain the framework package and metadata.

Upgrade:

pip install --upgrade 'asibench==0.2.0'
# Optional model and scientific dependencies:
pip install --upgrade 'asibench[full]==0.2.0'

Python 3.11 or later is required. Both asibench and the compatibility command ai4sci-bench remain available.

Validation: the locked offline test suite, strict Twine checks for wheel and sdist, artifact public-boundary inspection, and clean wheel installation with both CLI entry points and package/runtime version checks.

v0.1.6

Release asibench 0.1.6.\n\n- Fixes construction of the public MPSC evaluator solution object.\n- Distinguishes submission failures from scorer/runtime/reference failures.\n- Marks invalid local evaluations with final_score: null, excludes them from aggregates, and increments scorer_error_count.\n- Synchronizes package and runtime version metadata at 0.1.6.\n\nValidation: 2350 tests passed, 2 skipped, and 24 deselected; wheel and sdist passed strict Twine checks; all six public scorers from Issue #2 loaded against the built asibench[full] wheel.\n\nReproducible route: check out v0.1.6 and install asibench[full]==0.1.6.

v0.1.5

Release asibench 0.1.5.

• Includes current BenchFlow integration and documentation updates.
• Published via the repository PyPI workflow.

asibench 0.1.4

asibench 0.1.4

This patch release restores the advertised public local-scoring route for six seed31415 tasks.

Fixed

• Published ai4sci_bench.runner.submission_sandbox for TSP and Max-3-SAT.
• Added evaluator-only runtimes for:
  • Levin context grid search
  • MPSC safety filter
  • UCB Q-learning regret
  • CMOS op-amp design
• Removed public scorer dependencies on generate_gt.py and unavailable private task-core modules.
• Preserved the existing scorer weights and deterministic scoring semantics.
• Added policy tests for public-only scorer imports and the GT-generation boundary.

Validation

• 2128 passed, 2 skipped, 22 deselected
• Wheel and source distribution passed twine check --strict

Full tagged revision: f96ac81d39f127f2280f736fe1a9b266816947d7

asibench 0.1.3

Publishes the current public ASI-Bench framework, including read-only pulled instance handling and correct N/A reporting for unscored submissions.