Skip to content

fp-cloud-cli 0.0.2

Choose a tag to compare

@github-actions github-actions released this 25 Oct 04:26
a9e4257

fp-cloud-cli 0.0.2 is on PyPI: https://pypi.org/project/fp-cloud-cli/0.0.2/


Stable release. The 0.0.2b0 line the bump job opened after 0.0.1 is cut stable
here rather than published, because the server endpoints these commands drive are now
deployed to FailproofAI Cloud and the CLI should not trail them as a pre-release.

Nothing about the command surface changes at the cut: the entries below are the entire
diff from 0.0.1, and the bump job opens 0.0.3b0 after the upload.

Added

  • Hosted evaluator authoring for code, LLM judge and Jev: create/deploy, AI draft specs, dry-run, labels, toggles, instances, schema, jobs and backfill.

  • Dashboards and tiles; session overview/export/history/re-evaluate; latency and sentiment; query composition/export; alert recipients/issues and agent contracts.

  • Policy author/backtest/candidacy, assistant artifact tables, fail-closed interactive write approvals, and agent-curated home briefs.

  • Organization creation, permission-set roles, member deletion/defaults, usage windows, offers/trial, model context windows, OTEL and Slack controls.

  • fp policies backtest covers every policy kind: a published jev/both policy, or --semantic checks.json (with or without a source file), runs the dashboard's every-kind backtest — a free estimate by default, --run [--sample N] to spend Jev allowance. New --kind --semantic --hours --agent-id --env --run --sample; a regex file or id still uses the module backtest.

  • fp policies author --kind regex|jev|both asks the assistant for Jev checks or a hybrid policy.

  • fp issues list|count take the inbox filters (--agent-id --env --machine --source --audit-id --assignee --archived --search --since, and count --alert-id); new fp issues facets lists the values they can take.

  • fp sessions import FILE... imports Claude Code transcripts, Codex rollouts or event JSONL (--format --env --dry-run), as the dashboard's import dialog does.

  • fp list trigger_events, fp list eval_envs and fp list metric_keys.

  • fp fleet deploy --target POLICY=INTEGRATION[/agt_ID] scopes a machine's assignment; repeat the flag to include multiple alternatives. --all-agents POLICY explicitly clears its scope. Deploy plans, show, history, rollback and JSON readback retain assignment targets.

  • fp policies publish <id> --kind regex|jev|both [--source f.mjs] [--semantic f.json]: a
    FailproofAI Cloud policy can carry Jev checks — jev (declarations only, no JavaScript) or
    both (JavaScript reviewable by exactly its own checks; the server derives the authority).
    The kind follows from what is given when --kind is omitted, and a kind that contradicts its
    inputs is exit 2 before anything is sent. The positional source still works.

  • No Jev mode is set by hand: FailproofAI Cloud derives a machine's Jev mode from what is
    deployed to it (on while a jev/both policy is, off otherwise), so fp fleet deploy has
    no --jev-mode and there is no fp fleet jev-mode. An observe effect applies to jev
    and both policies like any other: their checks are recorded and decide nothing.

  • fp fleet rollback says when the Jev mode differs (jev mode <now> → <then>, and
    jevModeBefore in --json); fp fleet history shows each generation's Jev mode.

  • The fleet deploy and policies publish help says Jev checks run on
    FailproofAI Cloud and nothing is installed on the machine, and fleet show
    names the reasons a machine with a mode cannot ask (jev_unconfigured, transcripts_disabled)
    and the jev_budget report.

  • fp policies list/show print each version's kind and Jev check names (show also the
    declarations); fp fleet list/show print the machine's Jev mode and the policy errors it
    reported, and --json carries kind/semantic/semanticSha256/authority/reviewedBy,
    jevMode and policyErrors/policyErrorsAt when the server sends them.

  • jev:evaluate is a known permission — Jev through FailproofAI Cloud, charged to the
    org's plan — so fp keys and fp users accept it instead of refusing it as unknown,
    and the admin preset carries it, as the server's built-in admin set now does. The
    server refuses a KEY carrying it without both events:add and policies:pull (422,
    naming what to add in missing_permissions); the admin preset has both. (#833)

  • fp keys create/update --permission-set machine: the dashboard's key-only preset
    (events:add + policies:pull + jev:evaluate), for an enrolled machine using Jev through
    FailproofAI Cloud. fp keys also refuses jev:evaluate without both prerequisites itself
    (exit 2, naming what is missing) instead of sending the key for the server's 422. (#833)

  • fp policies list and fp policies show print what a jev/both version's checks take of a
    machine's Jev question budget (jevChars), and a deploy refused with jev_budget_exceeded
    lists each Jev policy's share, largest first (the hint under --json), so the refusal says
    which policy to leave off.

Changed

  • New Cloud API families work in key mode; unsupported session-only commands deny before HTTP and unknown-route 404s exit 6 with an availability message.
  • Shared screens use the mint/pink wordmark and grey labels, without boxes/rules or shortened ids; existing JSON contracts are unchanged.
  • The agent skill, command reference, README and Cloud CLI docs include end-to-end evaluator workflows. No version bump or release is performed.
  • fp policies publish's refusal of Jev fields in the JavaScript now says where Jev checks go:
    the policy's own declarations, --kind both --semantic checks.json.
  • fp guardrails summary shows how many machines run Jev in each mode, reading the summary's
    jev.machines.observe and jev.machines.enforce.
  • fp policies publish (and policies compose --publish) refuses a source carrying Jev
    fields a cloud policy never reads — a semanticPolicies.add(...) call, or a registration
    declaring authority: "reviewable" — before anything is sent (exit 1, --no-verify does
    not skip it). Both were published as versions that read Jev-aware and are not: semantic
    checks load only from a pack, and a cloud policy's authority comes from its deployment, so
    it stays hard. Ship Jev checks in a pack with failproofai publish. Same rule, and the
    same comment-aware detection, as the dashboard's publish form; the server does not read
    the source, so nothing else refuses it for fp. (#833)

Fixed

  • Remove the unused colour-keyed permission rank, document full-id rendering, emit UTC Z timestamps for deploy backfills, and report approval-answer failures without a traceback while restoring the terminal (#885).
  • Reject natural-language binary judge steps, distinguish unknown routes from missing records on 404, normalize standalone backfill ranges to UTC Z, expose all subcommands in help, and render artifact headers before titles with consistent session-only wording (#885).
  • New Cloud commands use deliberate human summaries/tables and actionable empty states at 80/140 columns without raw response blobs. Evaluator previews display before writes and require explicit --force after failures; plan/claim deny key mode, deploy specs can omit the positional key, and older settings-route 404s degrade clearly. Existing JSON payloads stay unchanged (#885).
  • fp evaluators deploy keeps the previous version's condition and labels unless --condition/--label change them, so a new version no longer silently widens to every session (#885).
  • Evaluation backfill covers code, LLM judge and Jev evaluators, as FailproofAI Cloud now does: fp evaluators deploy --backfill no longer refuses judge/Jev, and both commands warn that judge/Jev backfills spend model budget instead of saying they are skipped.
  • fp usage windows says it needs an API key (the dashboard has no session route for it) instead of reporting the route as unavailable.
  • fp issues close --help no longer claims exit 9 for an issue that already ended: it is exit 1, as it always was, and it names fp audits dismiss rather than a command that does not exist.
  • fp fleet history keeps gen, when and jev visible at 80 columns (rollback needs gen): the policy list is shortened, then dropped, and change wraps first.
  • fp fleet show and fp fleet history name an exact-profile target by the profile label the machine reports (claude/.claude-work, with the agt_… id kept in show), falling back to the bare id.