Add pdscheck — pure-client-side PDS conformance verifier#174
Merged
Conversation
New app at apps/check/ (deployed to check.cirrus.earth). Solid + Vite + Tailwind v4, served as static assets via Wrangler. Tests AT Protocol PDS conformance across three modes: - Read-only: identity resolution, server endpoints, repo reads, sync 1.1 firehose, blobs, OAuth discovery - Write tests: lifecycle of createRecord / applyWrites / uploadBlob / putRecord / deleteRecord (OAuth-authenticated, ephemeral session) - OAuth conformance: full PAR + DPoP + PKCE + iss + revocation flow, with isolated probes for unregistered redirect_uri rejection, permission-set resolution (bogus, advertised, and published site.standard.authFull), and post-token scope/boundary enforcement Firehose sampling uses cursor=0 historical replay with cap (200) / timeout (8s) / inactivity (1.5s) / diversity-aware exit (creates + updates/deletes seen). Reports termination reason so users can tell "hit the cap" from "PDS went idle". Validates every response against atcute lexicon schemas and links every check to its spec section. Designed to be a one-stop conformance check for anyone running a PDS implementation.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
atproto-pds | d3dea3b | May 24 2026, 07:50 PM |
commit: |
Permission sets get resolved by the AS into either an echoed include: literal (the grant preserves the reference; AS recomputes at refresh) or expanded resource scopes (a snapshot of the computed permissions). Both are spec-valid — atproto.com/specs/permission doesn't mandate either representation in the token's scope claim. Previously the check flagged the expanded form as a "narrowing" because the include: token was dropped and replaced with repo?collection=X&... tokens. Now it recognizes pure expansion (only include: dropped, scopes added) as a pass with the expansion surfaced as evidence. Also teach scopeGrantsWriteTo about the multi-collection token form (repo?collection=X&collection=Y) that expansion produces, alongside the existing single-collection form (repo:X). Without this, boundary write checks falsely reported no write coverage after a permission set expanded.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
pdscheck | d3dea3b | May 24 2026, 07:50 PM |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
cirrusdocs | d3dea3b | Commit Preview URL Branch Preview URL |
May 24 2026, 07:50 PM |
Two of the three granular-scope advertisement checks were checking for syntax that pre-dated the final atproto.com/specs/permission spec and flagged spec-conformant ASes as warnings: - scope-phase2-granular looked for repo:read, repo:write:<nsid>, account.*, pds.* — none of these are in the actual spec. The real grammar uses bare resource-type tokens (repo, rpc, blob, account, identity) with parameters appended at request time by the client. Removed entirely; it was redundant with scope-resource-buckets. - scope-permission-sets looked for include:<nsid> strings in scopes_supported, but specific NSIDs are dynamically resolved at PAR time via lexicon resolution — the AS only advertises bare `include` as a resource type. Updated to check for that. - scope-resource-buckets reworked to expect all five resource tokens (repo, rpc, blob, identity, account) and warn on partial coverage. Spec URLs now point at atproto.com/specs/permission instead of the obsolete proposal discussion thread.
The OAuth conformance result only offered "verify a different account", forcing users back to the landing page when they wanted to also run read-only or write tests against the same target. Add the same pivot buttons RunView already shows. The handlers exit the flow first (clearing flow state and signing out), then start the requested run mode against the OAuth target.
The anonymous firehose sample is historical (cursor=0), so on long-lived PDSes every #commit frame can predate the Sync 1.1 upgrade — producing false fails on `commit-has-prevdata` and `commit-ops-have-prev`. Two changes: - Anonymous: downgrade the strict pass/fail to a tri-state. Any sampled frame with prevData → pass. None at all → warn (ambiguous: PDS may not support Sync 1.1, or the sample may predate the upgrade). Same shape for ops[].prev on update/delete. - Authenticated write probe: subscribe to the firehose with no cursor before the create/applyWrites/uploadBlob/delete run, then close + validate after. Fresh frames go through the same validators in strict mode, giving a definitive Sync 1.1 verdict.
# Conflicts: # pnpm-lock.yaml # pnpm-workspace.yaml
The package has no test files; vitest run was failing CI with "No test files found" plus a jsdom env-detection complaint. Removing the script lets pnpm's filtered test runner skip the package cleanly.
apps/check was pulling six runtime deps and vitest with no consumers: @atcute/bluesky, @atcute/tid, @kobalte/core, @solid-primitives/storage, jose, multiformats. Drop them from package.json. Also un-export two oauth-flow helpers that are only used within the same module. Ignore .claude/** in knip — local subagent worktrees pollute results without affecting CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
apps/check/(deployed at check.cirrus.earth) — a fully client-side AT Protocol PDS conformance verifierWhat it tests
Read-only: identity resolution (handle / DID / combined), server endpoints, repo reads (describe / get / list), sync 1.1 firehose (#commit prevData, blocks-as-CAR, ops-have-prev, #sync / #account / #identity event presence), blobs, OAuth discovery (protected-resource + auth-server + JWKS).
Write tests: createRecord / applyWrites / uploadBlob / putRecord / deleteRecord against an
earth.cirrus.check.testrecordlexicon (published in this PR). Uses ephemeral OAuth — auto signs out after each run.OAuth conformance: full PAR + DPoP + PKCE + iss + revocation flow, with isolated PAR probes (unregistered redirect_uri rejection, bogus permission set rejection, advertised include acceptance, and resolution of the published
site.standard.authFullpermission set lexicon). Post-token: scope-echo, DPoP-bound API call, in-scope and out-of-scope boundary writes, refresh, revocation.Test plan
site.standard.authFullinclude:— real flow fails at send-par with a clear scope error, isolated probe pinpoints the gap