Skip to content

v0.7.0

Latest

Choose a tag to compare

@github-actions github-actions released this 07 Sep 19:24
· 4 commits to main since this release
61208fc

A survey of the other SageMath MCP servers (a feature comparison and a
code-level read of each peer's source, 2026-08-24) drove this window, and four
rounds of external review hardened it. Nothing here is a breaking change: the
40-tool surface, the deny-by-default sandbox and the stdio/HTTP transports are
unchanged; everything added is additive (the verify_claim tool, MCP prompts,
the pre-warmed worker pool, portable workspace handles, image plot responses,
mutation and property testing, an outcome benchmark, and the opt-in passagemath
runtime). Highlights: verify_claim for checking the model's own algebra;
pip install "sagemath-mcp[passagemath]" as a ~1 GB alternative to the 3 GB
Sage image; and plots that render as images instead of a base64 wall.

Added

  • passagemath as a selectable runtime (pip install "sagemath-mcp[passagemath]").
    A pip-installable, modularized fork of SageMath as an alternative to the ~3 GB
    sagemath/sagemath Docker image — ~1 GB download, no Docker, no local Sage
    build; from sage.all import * and the worker run unmodified. Both runtimes
    work: the server detects which is installed at import
    (importlib.metadata.version("passagemath-standard")) and dispatches to the
    matching generated security-artifact set (_artifacts.py
    allowlist_passagemath.py / star_exports_passagemath.py), so the
    deny-by-default sandbox is identical on both. The extra is pinned exactly
    (passagemath-standard==10.8.9) because passagemath's own release QA has
    shipped broken backends (docs/passagemath_evaluation.md §4). Landing this
    required making the denylist derivation layout-aware (the modularized
    sage.interfaces.all re-exports ordinary maths, which the old derivation
    swallowed — REVIEW_ACTIONS 69) and denylisting three passagemath-only
    interface names (Maxima, Mathics3, mathics3) plus its commence_startup
    helper. Verified byte-for-byte no-op on monolithic (artifacts regenerate
    identically, all agreement tests pass) and correct on passagemath (15/15
    star-export modules screen clean, ordinary maths offered, interfaces refused,
    vetted star-imports allowed). make allowlist-passagemath /
    star-exports-passagemath regenerate its artifacts with no Docker. Remaining
    before it is a recommended path: a passagemath CI lane (integration suite +
    doctest corpus sweep against the pin) and filing the upstream maxima_lib
    regression.
  • Outcome benchmark. benchmarks/ — a fixed, seeded case set (cases.json,
    24 problems in five difficulty tiers, every gold answer verified in the Sage
    10.9 container) and a Workflow (outcome_benchmark.workflow.js) that runs it
    through the model twice, reasoning-only vs. with Sage compute, scoring every
    answer for mathematical equivalence in Sage (not string-matched) by an
    independent step. It measures the claim the doctest corpus cannot: do models
    get more mathematics right with these tools. First run (subject haiku,
    judge sonnet, scripts/write_benchmark_stats.pybenchmark-stats.md):
    17/24 → 24/24, the entire +7 in the compute-heavy and infeasible tiers —
    a factoring, an 8×8 determinant, a partition count — where reasoning alone
    refused six and answered one confidently wrong, while Sage got all ten. On
    the arithmetic/competition/advanced tiers both arms score 100%: the tool does
    not help where the model is already right, and does not hurt. A stronger
    subject model closes the gap on its own — this is as much a measurement of the
    model as of the server, so it is never CI-gated.
  • MCP prompts (3). prove_and_verify, solve_and_check and
    explore_object — reusable instructions a client surfaces in its prompt
    picker, each steering the model toward what this server is good at: verifying
    its own algebra with verify_claim, checking a result before presenting it,
    and building an object once in a session and exploring it in evaluate_sage
    rather than the fresh-namespace helper tools. Almost no MCP server ships
    prompts; they are the cheapest way to shape usage.
  • Pre-warmed worker pool. The cost of a session's first call was ~1s of
    Sage lazy initialisation (the import is cheap; the first evaluation is not).
    The server now keeps a small pool of spare workers, each already past that
    init, and a new session adopts one — measured ~980ms → ~2ms on the first
    call — while the pool refills in the background. Sized by
    SAGEMATH_MCP_WARM_POOL_SIZE (default 1, 0 disables), never exceeding
    SAGEMATH_MCP_MAX_SESSIONS.
  • Portable workspace handles. start_sage_session now returns a
    workspace_token alongside the workspace: a server-issued, unguessable bearer
    handle that addresses that one workspace independently of the transport-level
    MCP session id. Passed as any tool's session argument it reaches the same
    state across a reconnect — or a transport that rotates the session id per call
    (which is what the MCP spec's retirement of protocol-level sessions, and
    fastmcp 4, make the norm). Names keep their current transport-scoped behavior,
    so two clients each using default stay isolated. The handle is a bearer
    credential, not authentication: possession grants access, so it is unguessable
    and kept out of monitoring, listings, error messages, logs and journal
    filenames; an unknown or revoked handle is refused, never silently turned into
    a fresh workspace; and stopping or culling a workspace invalidates its handles.
    Every stateful tool and lifecycle operation (evaluate, verify, reset,
    interrupt, cancel, stop) resolves a handle through one central path. The
    fastmcp>=3.4.7,<4 cap stays in place — this is additive, not a lift of it.
  • verify_claim (39 → 40). The checking primitive from the field survey,
    for the dominant failure mode of models doing mathematics: confident wrong
    algebra. A stated claim — integral(x^2/(e^x-1), x, 0, oo) == 2*zeta(3) — is
    re-checked independently through a ladder: Sage's symbolic prover, the exact
    difference ((lhs-rhs).simplify_full().is_zero()), exact arithmetic over
    QQbar/AA for constant claims, then certified interval arithmetic and
    numeric sampling over the free variables. Verdicts are proved, refuted,
    supported or undecided, and two rules keep them honest: the prover
    returning False means not proved, never falserefuted requires an
    exact decision or an exhibited counterexample — and supported always
    carries its evidence (sample count, precision), never a bare confidence
    number. Several more honesty rules landed after external review. Decimal
    literals are read as the exact rationals they denote (0.1 means 1/10, so
    0.1 + 0.2 == 0.3 is proved and 1.0 + 1e-20 == 1.0 is refuted — deciding
    over 53-bit doubles answered both wrongly while claiming exactness), and a
    second review round closed the deeper case: a comparison whose operands are
    genuine machine floats (RR(1), an .n() result, a session value in RR) is
    now reported as supported over inexact numbers via a new float_comparison
    method, never as an exact proof — RR(1) + RR(1)/10^20 == RR(1) is true only
    by rounding. The session's active assumptions are honored, now including
    non-substitutable domain declarations: under assume(x, 'integer') a sampled
    1/2 is inadmissible and never offered as a counterexample to x != 1/2 (the
    first pass silently ignored such declarations and refuted falsely), and any
    verdict that relied on an assumption names it in the evidence. No new security
    surface: the claim passes the same fragment gate as every other tool parameter
    before touching generated code.
  • Two diagnostics tools (37 → 39). check_sage_health is an MCP-level
    readiness probe for stdio clients that cannot reach the HTTP /health route:
    it spins up (or reuses) the workspace worker, evaluates 1+1, and reports
    ok/backend/latency, reporting failure in its result rather than erroring.
    lookup_sage_doc returns upstream documentation links for a Sage name and —
    the part the manual cannot answer — whether this server offers that name to
    evaluate_sage caller code.
  • MCP annotations on every tool. Each tool now declares
    readOnlyHint/destructiveHint/idempotentHint/openWorldHint, so a client
    can tell which calls discard state (cancel/reset/stop) from those that
    keep it (interrupt). An inventory test pins the memberships.
  • Mutation testing of the security policy. make mutation
    (scripts/run_mutation_tests.py) drives cosmic-ray over
    src/sagemath_mcp/security.py: it applies each deliberate weakening — a
    flipped comparison, a dropped not, a relaxed and — and runs the security
    suite, counting how many the tests catch. That is a claim line coverage
    cannot make: security.py was already at 100% coverage and still let these
    through. The run is parallelised across HTTP workers, each mutating its own
    copy of the tree with PYTHONPATH shadowing the editable install, so a
    ~35-minute serial sweep finishes in ~2 minutes (--workers, default 8; 1
    is the serial reference); a weekly, non-gating CI job publishes
    mutation-stats.md. New Hypothesis property tests
    (tests/test_security_property.py) assert the policy's invariants over
    generated inputs — every forbidden name in every referencing position, any
    attribute on any forbidden module, any import at all — which is what kills the
    behavioural mutants. Nine of these tests were written directly against survivors
    the first run surfaced, closing real gaps: _is_dunder's length boundary and
    its and (the shortest path out of the sandbox), the resource limits accepted
    at the limit rather than only rejected past it, forbid_global/forbid_nonlocal
    firing on the right node, and the attribute-chain exemption not shielding a
    forbidden third segment (operator.abs.os). Score: 426/696 killed (61.2%);
    excluding the 209 equivalent type-annotation mutants (an X | None hint is a
    never-evaluated string under from __future__ import annotations, so no test
    can kill it), the effective score is 87.5%. The remaining survivors are
    equivalent or near-equivalent (interned-string ==/is, keyword-only *
    markers, index == last where index <= last), tracked in TODO.

Changed

  • The evaluation-timeout error now coaches the retry. Instead of a bare
    "timed out after Ns", it says the worker was restarted and variables were
    discarded, and names the fix: a larger per-call timeout,
    evaluate_sage_streaming to watch a long computation, or
    interrupt_sage_session to stop one while keeping its variables.

Fixed

  • Non-finite results stay valid JSON and keep their shape (2026-09-06
    return-shape audit). A float('inf')/nan serialises to the bare tokens
    Infinity/NaN, which are not valid JSON — a strict client rejects the whole
    response. And a result containing one (calculate_expression's
    {string, numeric} for log(0)) collapsed entirely to a single
    double-encoded string, because the -inf token in its repr defeated result
    reconstruction, dropping the documented numeric field. Both are fixed
    centrally, where every helper tool's result passes: reconstruction now accepts
    inf/nan at any depth via a bounded literal evaluator (no code execution),
    and non-finite floats are sent as the strings Infinity/-Infinity/NaN.
    The rest of the audit was already sound — large integers travel as decimal
    strings, plots as image content, and ordinary string results are fine.
  • Plots now render as images (2026-09-06 external evaluation).
    plot_expression, plot3d_expression and plot_multi_expression returned
    {"image_base64": ...} — a JSON dict a client serialised as text, so a plot
    arrived as a ~200 KB wall of base64 that displayed nothing and ate the context
    window. They now return proper MCP image content (a fastmcp Image
    ImageContent) the client renders inline, at a bounded canvas/DPI (a PNG
    dropped to ~25 KB), with a new image_format argument to choose SVG (vector,
    smaller for line plots) instead of PNG.
  • Slimmer, cache-friendly Docker image (2026-09-06 external evaluation). The
    Dockerfile did COPY . /workspace, pulling the whole repo — tests,
    external_docs, the 100 KB+ review file — into the image and busting the
    install layer's cache on every edit to any of them. It now copies only the
    wheel-build inputs (pyproject.toml, README.md, LICENSE, src/), so the
    image excludes the working tree and the layer survives doc/test edits.
    Verified the built image still runs the stateful smoke and serves all 40 tools.
  • Tool count reconciled and pinned to the inventory. The count disagreed
    across files (README 40, GitHub description 40, server.json "34");
    server.json now says 40, and a new test
    (test_hardcoded_tool_counts_match_the_inventory) fails if any stated count
    drifts from tests/fixtures/tool_inventory.json, the source of truth — so the
    next added tool points at every place to bump.
  • Honest scope language (2026-09-06 external review). "full access to
    SageMath", "run any SageMath code" and "arbitrary SageMath code" are replaced
    across the README, USAGE and the evaluate_sage tool description with the
    deny-by-default subset the sandbox actually offers. The usability point the
    same review raised is fixed alongside it: that the specialized tools evaluate
    in a fresh namespace and cannot see evaluate_sage variables — so stateful
    multi-step work belongs in evaluate_sage despite its "LAST RESORT" framing —
    is now stated in the tool description the model reads and at the top of the
    evaluate_sage reference, not only in a note far below.
  • Session/worker robustness (2026-09-06 external review). Four fixes:
    worker startup is now serialized by a per-session lock, closing a race where
    two simultaneous first requests to one session launched two workers and
    leaked one; a configurable ceiling (SAGEMATH_MCP_MAX_SESSIONS, default 128)
    bounds concurrently live workers so a client opening a workspace per call
    cannot exhaust the host, while existing sessions stay reachable; the ~30
    helper tools now record the same monitoring counters evaluate_sage does,
    where before they evaluated invisibly to the metrics; and readiness moved to
    a new HTTP /ready endpoint that evaluates 1+1 on the backend (503 when it
    cannot), with the Helm readiness probe pointed at it, so a pod whose Sage is
    unusable stops receiving traffic. /health stays a shallow liveness check on
    purpose — a wedged computation should not restart the pod.
  • The release now validates the artifact it publishes (2026-09-06 external
    review). The Docker release job builds the image, runs a stateful smoke test
    inside it (assign, read back in the same session — the exact workflow fastmcp
    4.0.3 broke while every signature stayed valid), and only then pushes and
    signs; a manual dry_run dispatch used to push and sign a GHCR image anyway
    and now publishes nothing. A second review round found the same class of gap
    on the other destinations: PyPI, the MCP registry and the GitHub release each
    gated on startsWith(github.ref, 'refs/tags/v') alone, and a
    workflow_dispatch can target a tag ref — so a dry-run dispatch against a tag
    still satisfied them. Every publish now requires the tag push event under
    one shared policy, and a static test asserts no ref-only gate returns. The
    Docker job now publishes the exact image the smoke test ran against — it
    retags and pushes the tested candidate and signs it by its registry digest,
    rather than a second build that could differ from the one just verified. The
    CI compose smoke test asserted nothing about its stateful call — it printed
    the result and reported success even when the second call failed — and now
    fails unless the read-back returns 42.
  • The onboarding paths now match the security model (2026-09-06 external
    review). The README's docker run example published the unauthenticated
    evaluator on every host interface while overriding the image's CMD without
    its --host 0.0.0.0 — unsafe and non-functional at once; it now carries the
    same hardening flags as Compose and publishes on the loopback interface, and
    a lint test holds every README port mapping to that. The dev/test container
    scripts defaulted to the moving sagemath/sagemath:latest tag with no
    resource ceilings; they now pin the Dockerfile's Sage release (a test keeps
    the three in step), apply pids/memory limits and no-new-privileges, and
    say plainly that they are a development fixture, not a hardened runtime.
  • fastmcp capped below 4. The requirement was >=3.4.7 with no upper
    bound, so a fresh install resolved fastmcp 4.0.3 — under which the
    cache-isolation suite fails: a second client's identical tool call is not
    executed in its own session, the cross-client leak
    tests/test_cache_isolation.py exists to catch. Now >=3.4.7,<4; raising it
    is deliberate work gated on that suite (REVIEW_ACTIONS item 68).
  • Orphaned worker grandchildren. The worker now leads its own process group
    and every hard kill goes through os.killpg, so helper processes Sage forks
    (the pexpect interfaces fork GAP among others) are reaped on cancel or timeout
    instead of being left to run. interrupt still signals only the worker, the
    way the Sage REPL forwards Ctrl-C.
  • Protocol-framing corruption from inherited descriptors. The JSON worker
    protocol moved off descriptor 1: the pipe is duplicated to a private stream and
    descriptor 1 is pointed at stderr, so a child a Sage internal forks — or a C
    library writing to the descriptor directly — surfaces as logged noise rather
    than a corrupted response line.
  • The allowlist generator no longer bakes in the caller shims. Regenerating
    allowlist.py on any Sage version had begun emitting attrcall and
    set_verbose (installed into the worker namespace after the scrub); both are
    meant to be absent, and three tests enforce it. The generator now subtracts the
    shims.
  • Workspace tokens no longer leak into logs (2026-09-07 external review,
    REVIEW_ACTIONS 70). reset/interrupt/cancel/stop interpolated the
    caller's session argument — now a bearer workspace_token — into MCP
    notifications and responses, contrary to the secrecy the handle promises. A
    token is shown as the generic label the workspace; only names appear.
  • verify_claim enforces exactness on every proof path (2026-09-07
    external review, REVIEW_ACTIONS 71, 73, 75). A rounded result wrapped in a
    list, a symbolic expression, a dict key, a bare predicate
    ((RR(1)+RR(1)/10^20-RR(1)).is_zero()), or that predicate wrapped in == True
    or a lambda, was still reported proved/exact. Exactness is now judged from a
    claim's inputs, not its collapsed value: every value-bearing sub-expression is
    evaluated from its original source (so Python's bit-xor precedence for ^, and
    UTF-8-byte-vs-character offsets for non-ASCII like α, can neither corrupt nor
    crash a claim) and any machine number among them blocks an exact rung. The
    session's active assumptions are attached centrally — a structured
    assumptions field and the evidence — so no branch can conceal them.
  • Warm pool holds its ceiling and reclaims cancelled workers (2026-09-07
    external review, REVIEW_ACTIONS 72, 74, 76). Total-worker accounting (live
    sessions + pool + in-flight refills) is shared, so a refill can no longer
    overshoot SAGEMATH_MCP_MAX_SESSIONS; and a cancelled refill's worker is
    reclaimed to completion by a manager-owned cleanup that survives cancellation
    of the request that triggered it — rather than being dropped from tracking
    while still alive — before its slot is reused.

Removed

  • smithery.yaml. Smithery's post-Arcade.dev publish flow accepts only a
    public HTTPS endpoint, so the GitHub/smithery.yaml connect the file existed
    for no longer exists; listing there would require hosting a public,
    authenticated code-execution endpoint against the local-only posture in
    SECURITY.md. Distribution is covered by the official MCP registry
    (io.github.XBP-Europe/sagemath-mcp) and Glama.

Tests

  • Covered sympy-mcp's entire self-demonstration (calculus, linear algebra, the
    damped oscillator, a coupled two-tank ODE system checked against its algebraic
    steady state, general relativity via SageManifolds, units) and the peer field's
    lattice-reduction and GAP-structure workloads as end-to-end use cases — all
    through evaluate_sage in one carried-over session.