Skip to content

v1.23.0

Choose a tag to compare

@a-akimov a-akimov released this 17 Oct 06:20
2cc2f6d

Five new transport rules check the Streamable HTTP envelope, and
authorization discovery now follows the spec. Scores change for most HTTP
servers and for servers behind a login.

Added

  • Five rules check the Streamable HTTP envelope on every remote server.
    They count in the Protocol category. Scores change for most HTTP servers,
    mostly upward, because each new rule adds the points it awards to the
    maximum. Each rule is skipped as not applicable when its probe cannot
    observe the behaviour: the server refuses initialize, redirects to another
    origin, or issues no session ID.
    • transport_notification_accepted (LOW): a notification POST must get
      HTTP 202 with no body, or an HTTP error status if the server refuses it
      (Transports §Sending Messages to the Server,
      and the same text in 2026-07-28). A 200, a 204, or a 202 with a body
      fails.
    • transport_invalid_request_rejected (MEDIUM): a request whose method
      is a number must be rejected with a JSON-RPC error or an HTTP 4xx.
      Serving it, crashing with a 5xx, or never answering fails.
    • transport_invalid_request_error_code (LOW): the refusal must carry
      -32600 (Invalid Request), the code JSON-RPC 2.0 assigns. Several MCP
      SDKs send -32700 or -32602 instead, so servers built on them fail this
      rule until the SDK is fixed. On the stateless 2026-07-28 path, -32020
      from header validation also passes. It is skipped when the request was
      not refused, which the rule above already fails.
    • transport_get_stream_contract (LOW, 2025-03-26 to 2025-11-25): a GET
      must open a text/event-stream or return HTTP 405
      (Transports §Listening for Messages from the Server).
    • transport_session_delete (LOW, 2025-03-26 to 2025-11-25): DELETE with
      Mcp-Session-Id must be refused with HTTP 405, or end the session, after
      which a request in it must get HTTP 404
      (Transports §Session Management).
      A server that accepts the DELETE but keeps serving the session is not
      judged.
    • A 404 to the GET, the DELETE or the invalid request is not judged when a
      ping in the same session also gets 404: the session itself has expired.
    • The probes open short sessions of their own with initialize and close
      them again; none of them calls a tool. They follow a same-origin redirect
      the way the audit's own session does.
    • In one URL per host from 4,869 public registry hosts, the failure rates
      are 12.7% (notification), 4.7% (invalid request), 74.3% of the servers
      that refuse it (error code), 19.9% (GET) and 20.3% of the 516 servers that
      issue session IDs (DELETE). Servers on hosting
      platforms fail at the same rates as servers on their own domains.
  • Python 3.14 and 3.15 support. Both are tested in CI on Linux, macOS and
    Windows and listed in the package classifiers. No dependency changes were
    needed.

Fixed

  • Authorization discovery follows the spec, so three auth rules stop failing
    correctly configured servers.
    Scores change for servers behind a login.
    • The probe now reads the challenge's resource_metadata URL first,
      wherever it is hosted, then the RFC 9728 well-known locations, the order
      an MCP client uses. auth_challenge_references_metadata passes when that
      URL serves the metadata. It no longer fails metadata hosted on another
      origin, which RFC 9728 §5.1 allows.
    • From 2025-11-25 a server may publish its metadata at a well-known
      location instead of naming it in the 401 challenge. A challenge without
      resource_metadata now skips auth_challenge_references_metadata as
      not applicable, unless the server negotiated 2025-06-18, which requires
      the parameter. A 403 response is never held to the 401 requirement.
    • Authorization server metadata is looked up in the MCP discovery order,
      including path insertion for an issuer with a path, such as
      https://github.com/login/oauth. GitHub's MCP server used to fail
      auth_server_metadata_pkce, and Stripe's failed
      auth_server_metadata_present; both now pass.
    • A metadata document counts only when its issuer is identical to the
      listed issuer (RFC 8414 §3.3). Clients must reject any other, so
      auth_server_metadata_present now fails one and names both values. An
      issuer that differs only by a trailing slash is accepted, and the result
      names the declared value.
    • Discovery URLs that point a public server at a loopback, link-local or
      private address are reported, not fetched. Host names are resolved
      first, and one non-public address is enough to block the URL. A
      malformed challenge URL now fails auth_challenge_references_metadata
      instead of being skipped as unreachable, and URL schemes are matched
      case-insensitively, so HTTPS:// is accepted.
    • In a sample of 161 auth-gated public servers, this change raised 14
      scores and lowered 5. The drops are 3 issuers on a different host than
      the metadata declares; one issuer published
      only at a location outside the MCP discovery order; and one challenge
      URL whose metadata names a different resource.

Changed

  • -- after --stdio now goes to the server command on every Python
    version, so mcpscore --stdio ./server -- --flag runs ./server -- --flag.
    Before, Python 3.11–3.14 ended the command at -- and treated what followed
    as the audit target, which always failed with "give either a target or
    --stdio"; Python 3.15's argparse already passed it through. mcpscore now
    splits the command line itself, so the result no longer depends on the
    Python version. The same holds for --stdio=CMD and abbreviations such as
    --std: mcpscore --stdio=./server --flag runs ./server --flag, where it
    used to stop with a usage error.
  • Dependencies refreshed. The lockfile picks up starlette 1.8.0 and patch
    or minor releases of distlib, filelock, platformdirs, pluggy,
    python-discovery and virtualenv, and the development linter moves to ruff
    0.17.1. The mcp==2.3.0 pin is unchanged; it is still the latest SDK.