chore: add gap-analysis maintainer skill#571
Merged
Merged
Conversation
Adds a Claude Code skill at .claude/skills/gap-analysis/ that audits each provider adapter against upstream docs and produces a dated markdown report under .agent/gap-analysis/. Triggered via /gap-analysis with scopes <provider>, feature <name>, models, or --all. The skill is read-only: it never edits feature-support.ts or any model-meta.ts directly. Also gitignores .agent/gap-analysis/ and .agent/triage/ since those are local agent scratch output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughAdds a maintainer-only Claude gap-analysis skill, curated provider docs, an audit checklist expanded with telemetry parity, a report template, and .gitignore entries to ignore generated agent artifacts. The skill documents invocation, workflow, citation rules, known providers (19 features), and a verification checklist. ChangesGap-Analysis Skill Documentation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version PreviewNo changeset entries found. Merging this PR will not cause a version bump for any packages. |
|
View your CI Pipeline Execution ↗ for commit 3fb5adf
☁️ Nx Cloud last updated this comment at |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/gap-analysis/references/provider-doc-urls.md:
- Around line 90-92: Update the maintenance instruction in provider-doc-urls.md
so it no longer directs agents to edit the file “in the same report turn”;
instead instruct agents to keep audit runs read-only (only write the report
under .agent/gap-analysis/) and, if a URL needs to be changed, create a separate
PR or open an issue referencing the report so maintainers can apply the
update—modify the sentence that begins "If you find a URL has permanently
moved..." to reflect this workflow and preserve the skill's clean-working-tree
rule.
In @.claude/skills/gap-analysis/SKILL.md:
- Around line 78-84: The documented "Known features (22)" count is out of sync
with the enumerated list (19); reconcile by using the canonical source
ALL_FEATURES: update the numeric header or the enumerated items so both match
ALL_FEATURES' length, and ensure any references/templates that say "all 22" are
changed to the real count or dynamically derived from ALL_FEATURES; locate the
features block (the list including `chat`, `one-shot-text`, ... `video-gen`) and
the ALL_FEATURES constant/definition and make them consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e61cb1fd-b082-4fff-865a-cb721bdab468
📒 Files selected for processing (5)
.claude/skills/gap-analysis/SKILL.md.claude/skills/gap-analysis/references/audit-checklist.md.claude/skills/gap-analysis/references/provider-doc-urls.md.claude/skills/gap-analysis/references/report-template.md.gitignore
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/gap-analysis/references/audit-checklist.md:
- Around line 100-101: The description for the media adapters should use the
same feature keys as the mapping table: replace the shorthand adapter names
`image` and `video` with `image-gen` and `video-gen` (and confirm
`tts`/`transcription` match the table) so the union expression using
ALL_FEATURES + media adapters is consistent with the feature table; update the
text around ALL_FEATURES and the four media adapters to reference `image-gen`,
`video-gen`, and the exact feature names used in the table.
- Around line 153-154: The sentence "4. Pricing drift is **out of scope** for an
audit (changes too often) — note in the report only if upstream announcement is
< 30 days old." is ambiguous; update this item to a single clear rule: state
that pricing drift is generally out of scope but must be included in the audit
report when there is an upstream pricing announcement within the prior 30 days
(or similar explicit time window), and rewrite the line (the "Pricing drift"
checklist item) to reflect that conditional as a definitive exception rather
than a parenthetical note so readers know when to include pricing information.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d392a152-188b-4918-a838-0c14de53071b
📒 Files selected for processing (3)
.claude/skills/gap-analysis/SKILL.md.claude/skills/gap-analysis/references/audit-checklist.md.claude/skills/gap-analysis/references/report-template.md
✅ Files skipped from review due to trivial changes (1)
- .claude/skills/gap-analysis/SKILL.md
🚧 Files skipped from review as they are similar to previous changes (1)
- .claude/skills/gap-analysis/references/report-template.md
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
- Fix feature count drift (22 -> 19) and point at ALL_FEATURES as canonical. - Align media adapter names with the feature table (image-gen, video-gen). - Rewrite pricing-drift rule as an unambiguous conditional. - Defer URL-file updates to a follow-up PR to preserve the clean-tree rule. - Add audit dimension 5: telemetry / observability parity (usage tokens, cache/reasoning counts, request ids, logging asymmetry across adapters). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.claude/skills/gap-analysis/references/audit-checklist.md (1)
229-231: ⚡ Quick winConsider clarifying "per-provider format" for telemetry rows.
The instruction asks subagents to "emit dimension-5 telemetry rows in the per-provider format" but doesn't define what that format is. The example table at lines 187-200 shows the final cross-adapter format with providers as columns. It may help to briefly specify what each subagent should return (e.g., "return a list of which telemetry fields your provider's adapter emits, with presence/absence marked" or similar) so subagents know the expected structure before the orchestrator stitches them together.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.claude/skills/gap-analysis/references/audit-checklist.md around lines 229 - 231, Clarify the "per-provider format" for telemetry rows by adding a one-paragraph spec that each subagent must return dimension-5 rows as a uniform list of per-provider records (one record per telemetry point) containing these fields: provider (string), metric_key/name (string), metric_value (typed/nullable), unit (string|null), timestamp (ISO string), and a presence flag or nulls for any fields the adapter does not emit; state that the orchestrator will map metric_key to cross-adapter columns and stitch provider records into the final cross-adapter table. Reference the terms "per-provider format", "dimension-5 telemetry rows", "orchestrator", and "cross-adapter table" so subagents know exactly which structure to emit.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.claude/skills/gap-analysis/references/audit-checklist.md:
- Around line 229-231: Clarify the "per-provider format" for telemetry rows by
adding a one-paragraph spec that each subagent must return dimension-5 rows as a
uniform list of per-provider records (one record per telemetry point) containing
these fields: provider (string), metric_key/name (string), metric_value
(typed/nullable), unit (string|null), timestamp (ISO string), and a presence
flag or nulls for any fields the adapter does not emit; state that the
orchestrator will map metric_key to cross-adapter columns and stitch provider
records into the final cross-adapter table. Reference the terms "per-provider
format", "dimension-5 telemetry rows", "orchestrator", and "cross-adapter table"
so subagents know exactly which structure to emit.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 5b999549-12d4-4db4-80a2-ef08d570ebc7
📒 Files selected for processing (4)
.claude/skills/gap-analysis/SKILL.md.claude/skills/gap-analysis/references/audit-checklist.md.claude/skills/gap-analysis/references/provider-doc-urls.md.claude/skills/gap-analysis/references/report-template.md
✅ Files skipped from review due to trivial changes (2)
- .claude/skills/gap-analysis/SKILL.md
- .claude/skills/gap-analysis/references/report-template.md
AlemTuzlak
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.claude/skills/gap-analysis/that audits each provider adapter (OpenAI, Anthropic, Gemini, Ollama, Grok, Groq, OpenRouter, fal, ElevenLabs) against upstream provider docs andtesting/e2e/src/lib/feature-support.ts. Triggered via/gap-analysis <provider | feature <name> | models | --all>. Produces a dated markdown report under.agent/gap-analysis/.feature-support.tsor anymodel-meta.ts..gitignores.agent/gap-analysis/and.agent/triage/(local agent scratch output).What's in the skill
SKILL.md— entry point with workflow and critical rules.references/provider-doc-urls.md— curated WebFetch targets per provider (+ context7 lib-id hints for SDK API surface).references/audit-checklist.md— per-dimension steps with a TanStack-feature → upstream-capability mapping table.references/report-template.md— markdown template the skill fills in (Summary / High / Medium / Low / Out-of-scope / Suggested follow-ups).Test plan
/gap-analysisand confirm the skill is discoverable./gap-analysis groq(smallest adapter — text only). Expect a report under.agent/gap-analysis/flagging any missing models and listing groq's matrix exclusions./gap-analysis feature tool-approval. Expect a per-provider table; Gemini row should cite the exclusion comment atfeature-support.ts:56.git statusshould show only new files under.agent/gap-analysis/(which is now gitignored, so effectively nothing changes in tracked files). No mutation ofpackages/ortesting/.🤖 Generated with Claude Code
Summary by CodeRabbit