Skip to content

Releases: Rynaro/prisma

v0.16.0 — Claude 5.x compatibility, default model claude-sonnet-5-5

Choose a tag to compare

@Rynaro Rynaro released this 17 Oct 21:35
04cf227

Since v0.15.0, the Anthropic provider supports the Claude 5.x model family — Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1. Deployments that do not set a Claude model change behavior: the previous default, claude-3-5-sonnet-latest, points to a retired model and was already failing every review with a 404. They now run claude-sonnet-5-5. Set the new ANTHROPIC_MODEL env var to choose another model deployment-wide.

Claude 5.x support (#45)

Model Tool choice Default output cap Input window Token estimate
claude-haiku-5-5, claude-sonnet-5-5, claude-opus-5-5, claude-fable-5-1 (and other 5.x) auto + strict schema + one retry 16,384 1M anthropic-approx-v2 (×1.55)
Claude 4.6 – 4.8 (e.g. claude-sonnet-4-6, claude-opus-4-8) forced (unchanged) 4096 1M Opus 4.7/4.8: anthropic-approx-v2; others: anthropic-approx (×1.15)
Older Claude (e.g. claude-haiku-4-5) forced (unchanged) 4096 200,000 anthropic-approx (×1.15)
  • Opus 5.5, Sonnet 5.5 and Fable 5.1 reject a forced tool call, and Haiku 5.5 skips its thinking step when forced. For 5.x models the reviewer now uses tool_choice: auto with a strict, Anthropic-only copy of the tool schema, and retries once if no tool call comes back. Pre-5 models send exactly the same request as before. ANTHROPIC_TOOL_CHOICE=auto|forced|unforced overrides the rule.
  • Thinking is on by default on 5.x and counts toward max_tokens, so the default cap is 16,384 there. An explicit ANTHROPIC_MAX_OUTPUT_TOKENS always wins.
  • No thinking config and no default effort are sent, so each model uses its own default. A repo can set provider_options.anthropic.effort.

Model selection works

  • New ANTHROPIC_MODEL env var.
  • A per-repo model: anthropic/<id> in .github/review-bot.yml is now honored; the adapter used to drop it silently. So are generation.max_output_tokens and provider_options.anthropic, the latter behind a denylist.
  • temperature, top_p and seed are never sent to Anthropic, because 5.x models reject them.

Failures you can act on

  • Refusals: a safety-classifier refusal (stop_reason: refusal) now gets its own notice and is never retried. Only the declined batch is dropped, and its files get their own "not reviewed" note. Server-side refusal fallbacks are deferred (see ADR-009).
  • Retired models: a 404 now maps to a capability error with a remedy naming current models. It used to be retried as a transport error and never reported.
  • Rejected requests: every 400 invalid_request_error maps to capability. Anthropic-specific remedies cover forced tool choice and thinking/effort rejections, in both the check-run summary and the comment reply.
  • Empty reviews: Claude adaptive-thinking models get their own "no findings" notice.

Observability

  • provider.usage is emitted for Anthropic (endpoint: messages, including cache reads and writes).
  • worker.provider.selected carries model.
  • Refusals carry code on provider.error / provider.batch.error.

Live check

scripts/smoke/anthropic-models.ts runs one real review per model with your own key. It costs a few cents, is never run in CI, and never prints prompts, responses or secrets:

read -rs ANTHROPIC_API_KEY && export ANTHROPIC_API_KEY
./node_modules/.bin/tsx scripts/smoke/anthropic-models.ts

Tests: 1312 across 58 files, all passing; 26/26 deterministic eval scenarios. Details: docs/model-compatibility.md § Anthropic and ADR-009.

v0.15.0 — GPT-6 / GPT-6.1 compatibility, default model gpt-6-luna

Choose a tag to compare

@Rynaro Rynaro released this 16 Oct 23:43
b441099

Since v0.14.0, the OpenAI provider supports the GPT-6 / GPT-6.1 model families and chunking follows the provider's real limits. This release is not byte-identical for deployments that do not set OPENAI_MODEL: they now run gpt-6-luna instead of gpt-4o, with the reasoning-model defaults below. To keep the previous behavior, set OPENAI_MODEL=gpt-4o (that snapshot shuts down on 2027-01-14). Deployments that set OPENAI_MODEL explicitly are unaffected by the default change.

GPT-6 / GPT-6.1 support (#43)

Model Endpoint Input ceiling Default output cap
gpt-6-astra, gpt-6-sol, gpt-6-luna, gpt-6.1-sol /responses (reasoning family) 922,000 32,768
chat-latest /responses (reasoning family) 272,000 32,768
classic (gpt-4o, gpt-4.1, …) /chat/completions 128,000 4096

The output cap is family-aware and resolved per request from the model the request uses; an explicit OPENAI_MAX_OUTPUT_TOKENS always wins. The token estimator now uses the o200k vocabulary for every reasoning-family model (it previously fell back to cl100k for gpt-5.6* and gpt-6*).

Default model is now gpt-6-luna

gpt-4o is no longer the default. Opt out with OPENAI_MODEL=gpt-4o until the 2027-01-14 shutdown.

Safer config on reasoning models

  • Sampling parameters are dropped for reasoning models unless reasoning effort is explicitly none.
  • Reasoning effort is forwarded verbatim. none / minimal are rejected by gpt-6-astra and gpt-6.1-sol; the rejection is surfaced with a remedy.
  • Error mapping carries param and code. unsupported_value / unsupported_parameter map to a capability error with a targeted remedy. 429 insufficient_quota is no longer retried.

Chunking follows the provider

Chunking is coupled to MAX_TOKENS_PER_PR and the provider output cap. When the output cap exceeds the window, the pipeline falls back and emits pipeline.budget.output_cap_exceeds_window. Two side effects:

  • Anthropic batches are now planned at or below MAX_TOKENS_PER_PR.
  • A classic model with MAX_TOKENS_PER_PR above 128K rejects over-window batches up front (cap 123,904 by default).

Observability

  • cache_write_tokens telemetry.
  • Errors carry param / code; 5xx errors carry retry_after_ms.
  • One shared "no findings on a reasoning model" notice replaces the three stale copies that recommended gpt-4.1 / OPENAI_TOOL_CHOICE=required; it now points at provider.usage.reasoning_tokens and the output cap.

Tests: 1134 across 58 files, all passing; 24/24 deterministic eval scenarios. Details: docs/model-compatibility.md and ADR-008.

v0.14.0 — positive feedback, clean-review approval & reviewer interaction

Choose a tag to compare

@Rynaro Rynaro released this 31 Jul 13:41
78a079b

Two feature sets since v0.13.1, both fully opt-in (default-off) via .github/review-bot.yml — with no config changes, behavior is byte-identical to v0.13.1.

Positive feedback & clean-review approval (#37)

positive_feedback:
  enabled: true        # up to max_items "Highlights" in the review summary,
  max_items: 3         # each praising a concrete good decision + why it was good

approval:
  celebrate_clean: true        # clean review → "✅ Prisma reviewed this PR and found no issues — nice work."
  clean_conclusion: success    # clean review check conclusion: neutral (default) | success

The reviewer can now celebrate what's good, not just flag what's wrong — including an opt-in real GitHub PR approval on clean reviews.

Reviewer interaction (#38)

interactions:
  enabled: true        # default false
  max_per_review: 3    # hard cap on provider-backed replies per review round (1–25)

Developers can discuss the feedback with the reviewer directly on the PR:

@bot ask why is finding 2 a security risk? we sanitize upstream
  • Replies are grounded in review context: the latest round's findings and check-run summary.
  • When more than one message is allowed, prior exchanges thread into context for later messages.
  • max_per_review hard-caps interactions per review round; a new review round resets the budget. Template replies (disabled / no review yet / cap reached) never call the provider.
  • The budget ledger is stateless (hidden comment markers — GitHub remains the only durable state) and hardened against forged-marker cap bypass.
  • Vendor-neutral: the provider contract gains respond(), implemented by all adapters (Anthropic, OpenAI, Copilot, deterministic fake for evals).

Verification

1018 tests across 56 files · 23/23 deterministic eval scenarios · vendor-isolation gate green.

Full changelog: v0.13.1...v0.14.0

v0.13.1 — lossless finding titles

Choose a tag to compare

@Rynaro Rynaro released this 30 Jul 22:42
29e06cb

What's fixed

Inline review-comment titles are no longer cut mid-word at 120 characters (#35) — the recurring truncated-title reports:

  • Titles end at a sentence or word boundary, with a visible … when truncated; whitespace-normalized to a single line; UTF-16 surrogate-pair-safe at every cut (no � on GitHub).
  • Zero information loss: when a title is truncated, the full provider message is preserved as the first paragraph of the comment body.
  • The shared prompt now instructs all providers (Anthropic / OpenAI / Copilot) that message is a one-line headline (≤ ~100 chars) with detail belonging in rationale — so truncation rarely triggers at all.
  • Whitespace-only provider messages are rejected with reason_code: blank_message instead of producing an empty title.
  • FINDING_TITLE_MAX_LENGTH = 120 is now exported from @prisma-bot/shared and machine-enforced on NormalizedFinding.title.

819 tests across 52 files, all passing · 15/15 deterministic eval scenarios PASS (report byte-identical — no behavior change for in-cap titles).

Full Changelog: v0.13.0...v0.13.1

🤖 Generated with Claude Code

v0.13.0 — resilient config + Kamal deploy

Choose a tag to compare

@Rynaro Rynaro released this 02 Jul 22:16
080bf4d

Highlights

Resilient repo config — guidance-only degradation (#33)

Previously, any schema violation in .github/review-bot.yml made the worker discard the entire custom config and fall back to DEFAULT_REPO_CONFIG — so a single over-cap review_guidance field (e.g. instructions past the 2,048-byte cap, or more than 5 context_files) silently reverted model, max_files, comment_cap, path_rules.exclude, nickname, command_marker, and every floor to defaults.

Now, when every validation error is confined to review_guidance.*, only the guidance block is dropped (reset to its empty default; has_guidance becomes false) while the rest of the config is honored. Any error outside review_guidance keeps the prior whole-file fallback. Degradation is never silent: worker.config.parse_error is logged with salvaged: true and a check-run summary note names the dropped field.

Coexisting Kamal deployment path (#32)

Adds a single-host Kamal + kamal-proxy deployment path alongside the existing Compose + Traefik flow (mutually exclusive on one host). See docs/deployment-kamal.md.

Verification

typecheck ✓ · 801 tests across 52 files ✓ · biome clean · 15/15 deterministic eval scenarios PASS.

Upgrade

Pull the published image ghcr.io/rynaro/prisma-bot:v0.13.0 (or :latest) and redeploy — e.g. bin/prisma --update --tag v0.13.0 --yes.

Full changelog: v0.12.0...v0.13.0

v0.12.0 — Diff-chunking stability

Choose a tag to compare

@Rynaro Rynaro released this 15 Jun 19:19
e7ccd97

Diff-chunking stability core

Rebuilds the diff-chunking subsystem so reviews stay stable on large PRs — no more dropped batches, token-budget overruns, or all-or-nothing skips. Grounded in how Qodo/PR-Agent, CodeRabbit, Cursor Bugbot, and Ellipsis handle the limited-context constraint.

What changed

  • Output truncation no longer drops findings. Output max_tokens is configurable per provider; on truncation the batch splits-and-retries instead of dropping its review. The PR is never aborted on truncation.
  • One honest, derived token budget. A single real tokenizer (js-tiktoken) feeds both the batcher and the per-call guard over the exact serialized prompt. The per-call budget is derived from the provider's context window (hard_cap_in = window − reserved_output − prompt_overhead − safety), replacing the hand-tuned 60k/120k literals.
  • Genuinely small chunks. A file larger than the budget is split at hunk boundaries (line numbers and dedupe preserved) instead of being skipped wholesale.
  • Graceful degradation. A sizing trip splits/skips the batch instead of aborting the whole PR (genuine auth errors still abort). An oversized PR now gets a prioritized partial review with a visible "not reviewed" list.

New config (all optional, safe defaults)

chunking.reserved_output_tokens (4096), chunking.max_truncation_retries (2), chunking.prompt_overhead_tokens (9000), chunking.safety_fraction (0.07); env ANTHROPIC_/COPILOT_/OPENAI_MAX_OUTPUT_TOKENS. call_token_budget and MAX_TOKENS_PER_PR are kept as optional ceilings. See docs/config-spec.md and ADR-006.

Backward compatibility

Fully backward-compatible: existing configs/envs still apply as ceilings, and happy-path review output is unchanged.

Full diff: #31

v0.11.2 — Explain empty reviews

Choose a tag to compare

@Rynaro Rynaro released this 15 Jun 16:45
d487bdb

Patch: a zero-finding review now explains itself instead of rendering a bare "No findings.".

When the model returns no findings (a clean PR, an under-producing model, or findings that all fell outside the diff), the check-run summary now states why — distinguishing "no issues above your configured floors" from "N findings dropped as outside the diff", and pointing at confidence_floor / severity_floor when you expected more. Applied to both the single-call and chunked paths (the chunked path previously had no empty-review diagnostic, and the single-call path only spoke up for reasoning models).

This is legibility, not recall: it makes an empty result self-explanatory; it does not change how many findings the model produces.

738 tests · 15/15 eval scenarios · typecheck + lint clean.

v0.11.1 — Fix per-call token guard

Choose a tag to compare

@Rynaro Rynaro released this 15 Jun 13:26
51f0a46

Patch fix for a v0.11.0 regression.

After v0.11.0 began sending real diff content to the provider, non-trivial PRs failed with "capability: request exceeds maxTokensPerCall" before any provider call. The adapter pre-flight guard (maxTokensPerCall = MAX_TOKENS_PER_PR / 2 = 30k) sat below the chunker's per-call budget (chunking.call_token_budget = 60k), so the chunker built batches the adapter then rejected. Empty content hid the mismatch pre-v0.11.0.

Fix: the guard is a backstop, not the primary control — it must sit above the chunker budget plus serialization overhead. Removed the / 2 (maxTokensPerCall = MAX_TOKENS_PER_PR) and raised the MAX_TOKENS_PER_PR default to 120000. Regression test added; runbook updated.

No config change required — operators on the default pick this up automatically. If you raised chunking.call_token_budget above ~90k, also raise MAX_TOKENS_PER_PR to keep the guard above it.

736 tests · 15/15 eval scenarios · typecheck + lint clean.

v0.11.0 — Grounded reviews

Choose a tag to compare

@Rynaro Rynaro released this 15 Jun 12:35
c9818c8

The reviewer now reads your actual code.

Grounded reviews

The provider previously received empty hunk content — it reviewed on file paths + repo heuristics alone, anchoring every finding on new files to line 1 and inferring issues it couldn't verify. Now the real diff body is threaded end-to-end and rendered with per-line numbers, so findings:

  • are verified against actual code instead of guessed from paths, and
  • anchor to the exact line rather than defaulting to line 1.

Structural de-duplication

Duplicate comments about the same issue (reworded by the model) are now collapsed via a (path, category) dedupe key that is independent of wording and line, while preserving the cross-hunk dedupe contract. Collapsed siblings still appear in the Checks summary.

Operator note

With real content populated, the batcher's per-file token estimate (content.length / 4) is now exercised instead of the empty-content fallback, raising estimates ~10×. Large PRs will chunk into more batches / hit the oversized cutoff sooner — consider revisiting chunking.callTokenBudget / maxCalls.

735 tests · 15/15 eval scenarios · typecheck + lint clean.

v0.10.0 — Reasoning-model support

Choose a tag to compare

@Rynaro Rynaro released this 14 Jun 22:46
0b0a40c

What's new

The OpenAI adapter now works with reasoning models (the gpt-5 family and o-series), not just gpt-4-class models — and if a model still under-produces, the bot tells you instead of failing silently.

  • Reasoning-model tool calls. gpt-5 / o-series models do interleaved thinking; forcing a specific function short-circuited that into an empty {"findings": []} call (the "succeeds, zero findings, ~1s" symptom). The adapter now sends tool_choice: "required" for reasoning families so the model reasons, then calls the single review tool. Classic models (gpt-4o, gpt-4.1) are byte-identical — no change. Override with OPENAI_TOOL_CHOICE (auto | forced | required).

  • Empty-review safety net. If a reasoning model returns zero findings after being called on a real diff, the check run and comment reply now explain it instead of a silent "no issues found":

    ℹ️ Review produced no findings. The configured model (gpt-5.4-nano) is a reasoning model and may be under-producing with this review flow. If you expected findings, try openai/gpt-4.1, or set OPENAI_TOOL_CHOICE=required. See docs/model-compatibility.md.

    A classic model returning zero findings is still treated as a legitimately clean PR.

  • New docs/model-compatibility.md — which model families work with the adapter, the empty-review symptom, and the remedies (OPENAI_TOOL_CHOICE, generation.max_output_tokens, or gpt-4.1).

A shared isReasoningModel() detector now drives both the token-param and tool-choice selection. 733 tests across 49 files, 15/15 deterministic eval scenarios, typecheck/lint green.

Upgrade notes

No registration or config changes required. Update a deployed stack with:

bin/prisma --update --sync --tag v0.10.0 --yes

To run a gpt-5 / o-series model, set model: openai/gpt-5.4-nano (or similar) and watch findings_count in the worker logs on a substantive PR to confirm it emits findings. If a reasoning model still returns empty, the check-run notice points you at the remedies; openai/gpt-4.1 remains the proven choice for the forced-function-call flow.

Known minor follow-up: the empty-review hint fires on the single-call path; chunked large-PR + reasoning-model + empty result is not yet hinted.

Full changelog: v0.9.0...v0.10.0 (PRs #26, #27)