feat: add /dogfood skill for automated release validation#69
Conversation
10-phase dogfooding workflow: install from npm, cold-start testing, full command sweep with flag coverage, rebuild/staleness detection, engine comparison, changelog-targeted tests, thinking space for coverage gaps, auto-file bugs as GitHub issues with fix PRs, and structured report generation. Self-improving — the skill encourages the agent to update its own instructions. Usage: /dogfood <version>
|
Claude finished @carlos-alm's task —— View job PR Review Complete ✅Root Cause AnalysisProblem Being Solved: This PR addresses the need for comprehensive, automated end-to-end validation of published codegraph releases. Currently, there's no systematic way to validate that a published npm package works correctly across all CLI commands, engines (native vs WASM), and edge cases. Why This Approach: The Risk Assessment: LOW RISK - This is purely additive documentation/tooling that doesn't modify any production code. Phase-by-Phase AssessmentPhase 1: Root Cause Analysis ✅
Phase 2: Configuration Change Detection ✅
Phase 3: Test Integrity ✅
Phase 4: Code Quality ✅
Strengths
Technical AssessmentThe skill methodology is technically sound:
The testing scope covers areas that are known pain points:
Minor Concerns
Final AssessmentThis is a well-designed quality assurance tool that follows software engineering best practices:
The skill addresses a real need for release validation and provides comprehensive coverage without introducing any technical debt or configuration relaxation. |
Re-apply generated/ output paths for gen-deps.cjs and publish.yml, add mkdir -p guards for fresh clones, and gitignore both generated/DEPENDENCIES.md and generated/DEPENDENCIES.json.
Greptile SummaryAdds a comprehensive Key issue: Phase 7d instructs direct push to main branch (lines 290-296), which conflicts with the parallel sessions worktree workflow documented in Confidence Score: 4/5
Important Files Changed
Last reviewed commit: c713ce6 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| 3. Commit and push directly to main: | ||
| ```bash | ||
| git checkout main | ||
| git add package.json package-lock.json | ||
| git commit -m "chore: pin native binaries to v$ARGUMENTS after clean dogfood" | ||
| git push origin main | ||
| ``` |
There was a problem hiding this comment.
Direct push to main conflicts with the parallel sessions workflow in CLAUDE.md. The repo enforces /worktree isolation, but these instructions bypass that by directly committing to main. The guard-git hook may block this.
Consider creating a PR instead:
| 3. Commit and push directly to main: | |
| ```bash | |
| git checkout main | |
| git add package.json package-lock.json | |
| git commit -m "chore: pin native binaries to v$ARGUMENTS after clean dogfood" | |
| git push origin main | |
| ``` | |
| 3. Create a PR to update the native binary pins: | |
| ```bash | |
| git checkout -b chore/pin-native-binaries-v$ARGUMENTS main | |
| git add package.json package-lock.json | |
| git commit -m "chore: pin native binaries to v$ARGUMENTS after clean dogfood" | |
| gh pr create --base main \ | |
| --title "chore: pin native binaries to v$ARGUMENTS" \ | |
| --body "Validated in dogfooding session — zero bugs found" |
There was a problem hiding this comment.
Fixed in beddf94 — now creates a PR (chore/pin-native-binaries-v<version>) instead of pushing directly to main.
|
Claude finished @carlos-alm's task —— View job PR Review Complete ✅Conducting comprehensive 4-phase review of the dogfood skill implementation: Review Tasks:
Phase 1: Root Cause Analysis ✅Problem Being Solved: This PR addresses the need for systematic, automated end-to-end validation of published codegraph npm releases. Currently there's no structured way to verify that a published package works correctly across all CLI commands, engines (native vs WASM), incremental rebuilds, embeddings consistency, and edge cases. Why This Approach: The
Risk Assessment: VERY LOW - This is purely additive documentation that creates a new skill file. No production code changes, no configuration changes, no test modifications. Phase 2: Configuration Change Detection ✅HIGH-RISK Changes: None detected. Supporting Changes Analysis:
Configuration Assessment: No linting rules disabled, no test timeouts increased, no coverage thresholds lowered, no build configuration modified, no TypeScript strict mode flags disabled. All changes are organizational/path corrections only. Phase 3: Test Integrity ✅RED FLAGS: None detected. Test Analysis:
Test Coverage Assessment: The skill actually enhances testing by providing systematic end-to-end validation of published releases that would otherwise be manual or missing entirely. Phase 4: Code Quality ✅Implementation Quality: The dogfood skill is exceptionally well-structured: Strengths:
Security Assessment:
Greptile's Workflow Conflict Concern ✅Issue Identified: Lines 290-296 instruct direct push to main branch, which conflicts with CLAUDE.md:133 requiring Analysis: This is a valid concern. The repository enforces strict worktree isolation:
However, Context Matters: The direct push instruction only applies in the "green path" scenario (Phase 7d) where zero bugs are found during comprehensive testing. This indicates the release is thoroughly validated and ready for production version pin updates. Recommendation: The greptile suggestion to use a PR workflow instead of direct push is technically correct and aligns with repository practices. The direct push should be replaced with a PR creation to maintain consistency with the worktree workflow. Minor Technical Notes
Final AssessmentCondition for Approval: Address the workflow conflict in lines 290-296 by replacing direct push with PR creation as suggested by greptile. |
|
Claude finished @carlos-alm's task —— View job Comprehensive PR Review Complete ✅Root Cause AnalysisProblem Being Solved: This PR addresses the need for comprehensive, automated end-to-end validation of published codegraph releases. Currently, there's no systematic way to validate that a published npm package works correctly across all CLI commands, engines (native vs WASM), and edge cases. Why This Approach: The Risk Assessment: LOW RISK - This is purely additive documentation/tooling that doesn't modify any production code. Phase-by-Phase AssessmentPhase 1: Root Cause Analysis ✅
Phase 2: Configuration Change Detection ✅
Phase 3: Test Integrity ✅
Phase 4: Code Quality ✅
Strengths
Technical AssessmentThe skill methodology is technically sound:
The testing scope covers areas that are known pain points:
Critical Workflow Conflict Issue🚨 BLOCKING ISSUE: Greptile correctly identified a workflow conflict in Phase 7d (lines 290-296). The skill instructs direct push to main branch, which conflicts with the parallel sessions worktree workflow documented in The Problem: git checkout -b chore/pin-native-binaries-v$ARGUMENTS main
# ... commits directly to main branchRepository Rules Violated:
Required Fix: Phase 7d must use PR workflow instead of direct main branch push, consistent with other phases that properly create PRs. Minor Concerns
Final AssessmentThis is a well-designed quality assurance tool that follows software engineering best practices, but has one critical workflow violation that must be fixed before merge. The skill addresses a real need for release validation and provides comprehensive coverage without introducing any technical debt or configuration relaxation. However, the direct main branch push instruction conflicts with established repository safety practices. |
1 participant
Summary
/dogfood <version>skill that runs a full 10-phase validation session against a published npm releaseTest plan
/dogfood 2.2.1and verify the skill executes end-to-endgenerated/DOGFOOD_REPORT_v2.2.1.md