feat(feeds): publish sharded catalog snapshots

[giodl73-repo]

Summary

Completes the ClawHub producer side of hosted catalog distribution with signed shard roots and immutable digest-addressed plugin/skill shards.

This is stacked after #3160. Review the final commit (7479e2af) for this slice; earlier commits are the existing feed query/change stack.

What changes

• defines strict shard-root, descriptor, and shard schemas with exact-byte SHA-256 verification and bounded aggregate limits
• publishes complete plugin and skill snapshots as immutable shards before atomically making a signed root ready
• removes silent 1,000-entry truncation from skills; the legacy skills route redirects to the complete root only when the atomic form cannot represent the latest sequence
• stores lightweight descriptors separately so root reads do not load shard payload bytes
• retains immutable shards for 30 days and serves them through that retention window
• serializes publication actions with a recoverable lease and queues overlapping workflow runs
• exposes public /v1/feeds/** URLs through the existing Nitro-to-Convex /api/v1/feeds/** proxy
• documents limits, caching, publication ordering, and consumer verification requirements

Bounds

• wire shard: at most 1 MiB and 10,000 entries
• ClawHub stored shard target: at most 900 KiB to reserve Convex document overhead
• root: at most 1 MiB, 1,024 shards, and 1,000,000 entries
• aggregate described shard bytes: at most 256 MiB

Compatibility

Small feeds continue publishing the legacy atomic response and a same-sequence sharded form. Oversized skills no longer publish a partial atomic snapshot; /v1/feeds/skills returns a no-store 308 to /v1/feeds/skills/root. Activating sharded roots as the default still requires the matching OpenClaw consumer.

Validation

• bun run --cwd packages/schema build
• bun x tsc --noEmit
• type-aware oxlint on changed source/test files
• 89 focused feed, schema, signing, query/change, cron, and proxy tests
• codex review --commit HEAD: no actionable correctness defects

[vercel]

Someone is attempting to deploy a commit to the OpenClaw Foundation Team on Vercel.

A member of the Team first needs to authorize it.

[clawsweeper]

Codex review: needs real behavior proof before merge. Reviewed July 17, 2026, 11:32 PM ET / July 18, 2026, 03:32 UTC.

Summary
The branch adds signed, digest-addressed sharded plugin and skill catalog snapshots, signed query/change endpoints, publication retention, and feed proxy/documentation updates.

Reproducibility: not applicable. as a feature proposal; the relevant merge evidence is a real producer publication plus compatible consumer retrieval, which has not been supplied.

Review metrics: 2 noteworthy metrics.

• Patch surface: 48 files; 6,926 additions, 152 deletions. The branch changes a public feed protocol, persistence schema, HTTP delivery, workflow behavior, and generated schema artifacts in one stacked review.
• Stack depth: 14 commits. Only the final commit is presented as this slice, while the branch also includes the preceding query, change, and signing implementation.

Merge readiness
Overall: 🦪 silver shellfish
Proof: 🦪 silver shellfish
Patch quality: 🦐 gold shrimp
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• [P1] Add redacted live proof of a signed root, immutable shard retrieval, and exact SHA-256 verification.
• Rebase the complete stacked branch onto current main and make the intended consumer compatibility path explicit.
• Obtain maintainer approval for signing-key custody, rotation, trust-root deployment, and oversized-skills redirect behavior.

Proof guidance:

• [P1] Needs real behavior proof before merge: The PR reports automated tests and static checks only; before merge, add redacted evidence from a real signed publication and retrieval flow (for example, live endpoint output verifying a root signature, shard digest, and consumer-compatible response). Update the PR body after adding proof; if no fresh review appears, ask a maintainer to comment @clawsweeper re-review.

Risk before merge

• [P2] Merging the new root/shard routes and the oversized-skills redirect before the corresponding OpenClaw consumer and fallback behavior are agreed can break or strand existing feed clients.
• [P1] The new production signing-key configuration creates a security-sensitive operational boundary that needs an explicit key custody, rotation, and trust-root decision.
• [P1] The branch is draft, stacked, and reported dirty against its base; a rebase and whole-stack review are needed before judging the final merge result.

Maintainer options:

1. Land as a coordinated protocol rollout (recommended)
   Rebase this stack after its query dependency and require consumer compatibility plus a redacted live signed-publication proof before merging.
2. Accept producer-first activation risk
   Approve the redirect and signing rollout before consumer adoption only with explicit ownership of the compatibility fallback and production key lifecycle.
3. Pause the protocol expansion
   Keep the draft open until a maintainer sponsors the public feed contract and signing-key deployment plan.

Next step before merge

• [P1] A maintainer must choose the public protocol activation and signing-key rollout policy; missing real behavior proof is contributor follow-up, not a safe automated code-repair task.

Maintainer decision needed

• Question: Should ClawHub adopt this signed sharded-feed producer contract now, including the production signing-key/trust-root rollout and the oversized-skills redirect, or wait until the matching OpenClaw consumer is ready to land with it?
• Rationale: The patch establishes a new public cross-repository distribution protocol and an operational signing boundary; tests cannot decide rollout timing, trust-root ownership, or compatibility policy for existing consumers.
• Likely owner: unknown — No reliable individual owner is identifiable from the supplied current-main history context; the decision belongs with the owners of the hosted-feed and OpenClaw consumer contract.
• Options:
  • Stage with the consumer (recommended): Keep this producer stack draft until the matching OpenClaw consumer and production signing-key rollout can be reviewed and proven together.
  • Approve producer-first rollout: Accept the compatibility and operational rollout risk, document the fallback contract, and land only after live producer evidence confirms the new endpoints.

Security
Cleared: The reviewed diff adds a signing boundary but shows no discrete code-level credential exposure, unpinned dependency, permission broadening, or supply-chain regression; production key handling still needs maintainer approval.

Review details

Best possible solution:

Agree on the producer/consumer activation contract and production signing-key ownership, rebase the complete stack onto current main, then prove a real signed publication and consumer retrieval path with redacted live output before landing the producer changes.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a feature proposal; the relevant merge evidence is a real producer publication plus compatible consumer retrieval, which has not been supplied.

Is this the best way to solve the issue?

Unclear: sharding is a plausible scalable design, but the best rollout depends on maintainer agreement about the consumer contract, the legacy redirect, and production signing-key handling.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against aaa73625ed41.

Label changes

Label changes:

• add P2: This is a substantial but not currently user-breaking proposed protocol expansion that needs normal-priority maintainer review.
• add merge-risk: 🚨 compatibility: New public feed routes and the oversized-skills 308 redirect require coordinated consumer and upgrade behavior.
• add merge-risk: 🚨 security-boundary: The branch introduces production feed-signing key configuration and a new signed trust boundary that needs explicit operational ownership.
• add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
• add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR reports automated tests and static checks only; before merge, add redacted evidence from a real signed publication and retrieval flow (for example, live endpoint output verifying a root signature, shard digest, and consumer-compatible response). Update the PR body after adding proof; if no fresh review appears, ask a maintainer to comment @clawsweeper re-review.

Label justifications:

• P2: This is a substantial but not currently user-breaking proposed protocol expansion that needs normal-priority maintainer review.
• merge-risk: 🚨 compatibility: New public feed routes and the oversized-skills 308 redirect require coordinated consumer and upgrade behavior.
• merge-risk: 🚨 security-boundary: The branch introduces production feed-signing key configuration and a new signed trust boundary that needs explicit operational ownership.
• rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
• status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR reports automated tests and static checks only; before merge, add redacted evidence from a real signed publication and retrieval flow (for example, live endpoint output verifying a root signature, shard digest, and consumer-compatible response). Update the PR body after adding proof; if no fresh review appears, ask a maintainer to comment @clawsweeper re-review.

Evidence reviewed

What I checked:

• Stacked branch scope: The PR body says this slice is stacked after the still-open query PR and asks reviewers to focus on its final commit; the supplied commit list confirms it also carries the preceding feed query/change/signing stack. (convex/catalogFeed.ts:1, 7479e2af4fbd)
• Current contract requires coordinated consumer rollout: The current hosted-feed specification identifies schema/versioned feed delivery as a cross-repository contract and says signed envelopes require a separate production key-management decision before they are advertised to OpenClaw clients. (specs/hosted-catalog-feed.md:267, aaa73625ed41)
• Compatibility-sensitive public route expansion: The diff adds root, shard, query, and changes routes and changes the oversized skills legacy endpoint to a 308 redirect, so existing client fallback and upgrade behavior need end-to-end confirmation with the corresponding consumer. (packages/schema/src/routes.ts:25, 7479e2af4fbd)
• No real after-fix behavior evidence: The PR body reports focused automated tests and static checks, but the supplied discussion has no live publication, signed-root retrieval, shard verification, or redacted runtime output proving the changed path in a real deployment. (convex/httpApiV1/catalogFeedShards.ts:1, 7479e2af4fbd)
• Related implementation remains open: The referenced upstream ClawHub query implementation remains an open draft PR, so this branch should be reviewed as a coordinated stack rather than as an independently mergeable final slice. (convex/httpApiV1/catalogFeedQuery.ts:1, e66b8ce4039c)

Likely related people:

• unknown: The available review context establishes a cross-repository producer/consumer contract but does not provide reliable current-main blame or merged-history attribution for an individual owner. (role: feed-protocol decision owner; confidence: low; files: specs/hosted-catalog-feed.md, convex/catalogFeed.ts, convex/httpApiV1/catalogFeedShards.ts)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[clawsweeper]

ClawSweeper status: review started.

I am starting a fresh review of this pull request: feat(feeds): publish sharded catalog snapshots This is item 1/1 in the current shard. Shard 7/18.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.