Skip to content

v2.0.0 — shipd agent

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 08 Nov 07:24
v2.0.0
83f9826

shipd-agent v2.0.0

A major update to the review agent: evidence-based reviews on Deep Agents, support for any LLM provider, the current Shipd assessment criteria and Olympus agent-run bar, and a new stage that predicts Shipd's Auto Review.

Upgrading from v0.1.0

  • Python 3.11+ is now required. Python 3.10 is no longer supported.
  • New dependencies (Deep Agents 0.7.34, LangChain 1.x, OpenAI and Gemini clients, keyring). Run ./run.sh --setup once after pulling.
  • The review rules have changed. The v0.1.0 agent-run rule and the Olympus solution-LOC floor are replaced by the current Olympus bar (see below). Re-reviewing a submission may give a different verdict than v0.1.0 did.
  • An existing Anthropic-only .env (ANTHROPIC_API_KEY, REVIEW_MODEL, REVIEW_EXPLORE_MODEL) works unchanged.

Highlights

Evidence-based reviews with Deep Agents (#9)

  • The reviewer and focused workers read numbered source pages and pass exact cited excerpts to a separate scorer. Invented or out-of-range citations are rejected.
  • Model-call and token budgets are shared across the reviewer, workers, coverage recheck and scoring. Evidence and usage are saved in the review bundle.

Any LLM provider (#5, #6)

  • Claude (default), GPT, Gemini, DeepSeek, Kimi, Grok, or any OpenAI-compatible endpoint, with separate explore and finalize models.
  • API keys go to the OS keychain (or a 0600 file when there is no keychain), never to .env or the logs.
  • A first run with no key asks for the provider, models and key, then starts. ./run.sh --configure-llm changes them later, and --forget-llm-key removes the stored key.

Current Shipd assessment criteria (#4, #10)

  • Deterministic checks the LLM can't overrule: the 1,000-word prompt limit, too many requirements, and the Olympus agent-run bar.
  • Agent-run bar: Nova, Vega and Orion each need at least 4 finished unhinted runs, and exactly one model solves the task at a rate above 0% and at most 25%. Olympus long horizon is Nova's average sequence length above 400K tokens.
  • Results the page can't confirm now return SKIP instead of PASS.

Auto Review prediction and new checks (#10)

  • A new final stage predicts Shipd's Auto Review. A concrete, cited contradiction is MAJOR and blocks approval. Findings are labeled as predictions, not Shipd's verdict. Turn it off with --no-auto-review.
  • New deterministic checks: false-pass review of passing patches, Dockerfile / test.sh / test.patch harness checks, repo eligibility, prompt style, and repeated Docker runs to catch flaky tests.

Safer browser actions (#7)

  • Scraping no longer clicks page controls, and form selection is limited to the verified review form.
  • Completed submissions are detected before setup, review and submission, and a batch stops when it reaches one.

Project

Known limitations

  • Shipd doesn't publish the models behind Nova, Vega and Orion or its Auto Review criteria, so the defaults and the Auto Review stage are best guesses. Override them with REVIEW_{VEGA,ORION,NOVA}_MODEL.
  • Providers other than Claude were tested against mocked APIs; a first real run per provider will confirm them.

Install

git clone https://github.com/vaishcodescape/shipd-agent.git
cd shipd-agent
./run.sh --setup
cp .env.example .env
./run.sh --configure-llm
./run.sh --check

Python wheel

pip install https://github.com/vaishcodescape/shipd-agent/releases/download/v2.0.0/shipd_agent-2.0.0-py3-none-any.whl

Full changelog: v0.1.0...v2.0.0
Full docs: README

Assets 4
Loading