Skip to content

Releases: seldonframe/reelier

v1.0.0: Dependabot/Renovate bump-safety check

Choose a tag to compare

@fixlyai fixlyai released this 24 Jul 15:39
bc6ea0f

Adds a Dependabot/Renovate bump-safety check: replay your recorded agent tool-call skills live against a bumped dependency at 0 LLM tokens, and fail the PR check on the exact step that drifted.

  • New wrap input to front your own MCP server / tool process (the case that exercises a bumped SDK your tool code imports).
  • Copy-paste recipe: .github/workflows/reelier-bump-check.yml.
  • Honest scope: at max-level 0 the replay never calls an LLM, so it verifies dependency / MCP-tool-call behavior, not model upgrades.

Also works unscoped from bump PRs, as a scheduled drift check or a plain "replay this skill on every PR" gate. See the README.

Reelier replay v1

Choose a tag to compare

@fixlyai fixlyai released this 21 Jul 19:40
3afb781

First Marketplace release of the Reelier replay action: replay a recorded Reelier skill in CI — deterministic, 0 LLM tokens at Level 0, exit 1 on drift — and post the run receipt as a job summary.

- uses: seldonframe/reelier@v1
  with:
    skill: skills/nightly-check.skill.md

The v1 tag floats to the latest validated action. Full docs, the assertion grammar, and the published N=1000 benchmark: https://github.com/seldonframe/reelier

0.21.0 — reelier ci: drift-CI + PR receipts in one command

Choose a tag to compare

@fixlyai fixlyai released this 23 Jul 14:20

reelier ci

One command writes your whole workflow: reelier ci discovers the repo's *.skill.md files and generates .github/workflows/reelier-replay.yml — replay on every PR + daily schedule, manifest preflight failing closed on tool-schema drift, permissions preconfigured (pull-requests: write for the receipt comment, id-token: write for CI attestation).

Sticky PR receipt comment (Action)

The Action now upserts one sticky comment per PR carrying each skill's receipt — pass/fail, steps, duration, tokens, receipt permalink when pushed. A failed replay still comments (a red receipt is a real receipt); comment failures warn, never fail the job. Deliberately inactive on pull_request_target.

Full details: CHANGELOG.md

v0.13.0 — annotation trust ladder + the self-measuring scan

Choose a tag to compare

@fixlyai fixlyai released this 21 Jul 19:11

0.13.0 — Annotation trust ladder + the self-measuring scan

Added

  • MCP annotation consumption. The recording proxy captures each wrapped
    tool's tools/list annotation hints (readOnlyHint / destructiveHint /
    idempotentHint) into the trace meta record (toolAnnotations, keyed by
    exposed tool name; omitted when nothing is annotated — see SPEC §2.2).
    classifyEffect consumes them via a strict trust ladder:
    destructiveHint always wins → destructive verb match → idempotent-write
    verb match → read verb match (idempotentHint may tighten it) →
    readOnlyHint/idempotentHint refine unrecognized verbs → unknown stays
    destructive + flagged. An annotation NEVER downgrades a verb-list match — a
    server's readOnlyHint: true on create_note cannot exempt it from
    --allow-writes. Hints, not security: replay write-gating
    (--allow-writes) still applies to everything idempotent-write or worse.
    The runner's MCP tool adapter now shares this exact classifier, so the
    compiler and the adapter can never disagree.
  • Wrap onboarding in reelier init. Init now closes by offering
    reelier install as the recommended next step: "Wrap captures lossless
    traces (tool annotations included) — scan-from-history is a
    reconstruction; wrap is the recording." Interactive TTY: an explicit y/N
    (default N — the config is never modified without an explicit yes);
    non-TTY (or --yes): the exact reelier install one-liner is printed
    instead of a prompt.
  • Backup-or-abort guard. reelier install (and init's inline offer)
    now refuses to rewrite a config when the pre-write backup itself cannot
    be written — the install aborts with an honest error and the config is
    left byte-identical.
  • Self-measuring scan KPI. reelier scan (and the reelier_scan MCP
    tool, as replayableRate) now reports
    Replayable rate: X/Y sessions fully read-only (Z%) plus
    N session(s) blocked ONLY by unknown-verb tools (top blockers: ...)
    the blocker list names exactly which verbs to consider classifying next.
  • Empirical verb audit (run against a real 2,334-session history):
    read gains count retrieve tail preview ping health browse glob grep stat stats head exists info summarize screenshot logs; idempotent-write gains
    mark upload embed patch append sync; destructive gains spawn exec eval evaluate start stop clear push rotate finalize. Deliberately left out
    (write sense exists): resolve, watch, snapshot, meta, context,
    navigate. On that history the audit collapsed "blocked only by
    unknown-verb tools" from 494 sessions to 6 — 488 of them contained real
    writes now classified confidently instead of flagged as unknown.
  • Compiler variable-extraction polish (flag-only throughout — no new
    auto-substitution; exact-match dataflow binds are unchanged):
    • An array-element bind (json.items.2.id) now asks the concrete
      stability question — "is element [2] positionally stable across runs,
      or should this select it by a field match (e.g. the element whose
      id/name matches)?" — with the candidate fields read from the recorded
      element's own scalar keys (identifying names like id/name first).
    • Date-heuristic hardening: impossible calendar dates (2026-02-30, a
      non-leap 2026-02-29) are flagged "not a real calendar date" instead of
      receiving offset math fabricated from the Date.UTC roll-over; a
      datetime literal's suggestion keeps its time suffix verbatim
      ("{{today-7d}}T09:30:00Z"{{today±Nd}} resolves date-only); a
      non-UTC offset that lands on a different UTC calendar day gets an
      explicit which-day note; "1 day" is singular.
    • The same date/UUID/timestamp literal appearing in 3+ steps now flags
      ONCE with the full step list ("appears in steps 2, 4, 7 — one
      variable?") instead of per-step duplicates (SPEC §6.5).

v0.5.0 — the five atoms, the recorder, the compiler, the ladder

Choose a tag to compare

@fixlyai fixlyai released this 18 Jul 23:22

Reelier's first published npm release (0.1.0 -> 0.5.0) — a deterministic replay engine for AI agent workflows, built on the five-atom model: intent, action, assert, bind, effect.

What's in 0.5.0

  • Five-atom SKILL.md format + L0 runner — record any agent workflow as a SKILL.md recipe and replay it deterministically, zero LLM calls at the default level. A malformed skill (bad frontmatter, unrecognized assert/bind, out-of-order step) is rejected with the exact step and line, never silently skipped.
  • MCP proxy recorderreelier mcp --wrap "..." re-exposes any downstream MCP server's tools 1:1 (pure passthrough) plus three control tools (reelier_start_recording, reelier_note, reelier_stop_recording) so an agent can capture a lossless trace of a live session, with conservative built-in secret redaction at trace-write time.
  • Deterministic compiler with dataflow recovery (reelier compile) — turns a recorded trace into a runner-ready SKILL.md, zero LLM calls. Derives intent from narration, recovers dataflow binds by matching argument values against prior results, assigns effect classes from a verb heuristic, and — the honest-gaps principle — emits an explicit Open questions list for everything it can't confidently derive, rather than a fabricated check.
  • L1/L2 escalation ladder with write-back — strictly opt-in (--max-level 1|2), BYOK against almost any Anthropic- or OpenAI-compatible endpoint. L1 re-evaluates a step's already-captured observation with patched asserts/binds (zero side effects, never re-executes). L2 may patch args and re-executes the step exactly once, and only for read/idempotent-write steps — a diverged destructive step is never handed to an LLM. Every successful heal writes back to the skill file atomically, with a ## Changelog entry, so drift only costs an LLM call once.
  • reelier push — opt-in, fully inert without REELIER_CLOUD_URL/REELIER_CLOUD_KEY — syncs a skill's run records (and, on first push, the skill file) to a hosted Reelier Cloud instance, with honest cursor semantics (permanent-reject vs. transient-error handling) so a bad historical record never blocks everything pushed since.
  • reelier init — the 60-second first receipt: guided detect-config -> record (real MCP session, or a zero-setup live demo) -> compile -> replay -> receipt loop, closing with measured replay time and LLM token count (asserted 0, never assumed) and a comparison against our own published agent-vs-Reelier benchmark.

The benchmark results (measured, not claimed)

Full raw data and methodology: docs/strategy/reelier-launch/benchmark-results.md, reproducible via examples/benchmark.

  • 1,000/1,000 replays byte-identical at N=1000 (tail-variance test against a live npm registry endpoint)
  • 0 tokens per replay, verified from the run record on every single replay — never assumed
  • ~50x cheaper than a comparable agent run ($0.000000/replay vs. $0.019068/run average)
  • ~59x faster (44ms vs. 2,842ms average latency)
  • Honestly reported alongside a real grammar gap the benchmark surfaced: the current bind grammar can't express count/filter/enumerate aggregations over a JSON array — tracked as a good-first-issue, not hidden.

What's still missing

No Level 3 (full agentic recovery when a trace no longer applies at all — today that's a human editing the skill by hand). See the README's "Status" section for the complete list.

Get started

npm i -g @seldonframe/reelier && reelier init

AGPL-3.0 — the engine can never be taken closed. Your skills, traces, and run records are your data; the license doesn't touch them.