Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/claude.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,16 @@ jobs:
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
contents: write # Allow commits
pull-requests: write # Allow PR creation
issues: write # Allow issue comments
id-token: write
actions: read # Required for Claude to read CI results on PRs
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
fetch-depth: 0 # Full history for better context

- name: Run Claude Code
id: claude
Expand Down
120 changes: 120 additions & 0 deletions .github/workflows/quality-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: Quality Check

on:
workflow_run:
workflows: ["Test and Generate Previews"]
types: [completed]

jobs:
quality-check:
name: AI Quality Evaluation
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
permissions:
contents: read
pull-requests: write
issues: write
id-token: write

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Download preview metadata
uses: actions/download-artifact@v4
with:
name: preview-metadata
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Load metadata
id: metadata
run: |
if [ ! -f preview_metadata.json ]; then
echo "No preview metadata found, skipping quality check"
exit 0
fi

PR_NUMBER=$(jq -r '.pr_number' preview_metadata.json)
BUCKET=$(jq -r '.bucket' preview_metadata.json)
BASE_PATH=$(jq -r '.base_path' preview_metadata.json)
CHANGED_FILES=$(jq -r '.changed_files[]' preview_metadata.json | tr '\n' ' ')

echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
echo "bucket=$BUCKET" >> $GITHUB_OUTPUT
echo "base_path=$BASE_PATH" >> $GITHUB_OUTPUT
echo "changed_files=$CHANGED_FILES" >> $GITHUB_OUTPUT

- name: Setup Google Cloud authentication
uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.GCS_CREDENTIALS }}

- name: Download preview images
run: |
mkdir -p preview_images
gsutil -m cp -r "gs://${{ steps.metadata.outputs.bucket }}/${{ steps.metadata.outputs.base_path }}/*" preview_images/ || echo "No images to download"

- name: Trigger quality check with @claude
uses: actions/github-script@v7
with:
script: |
const prNumber = ${{ steps.metadata.outputs.pr_number }};
const bucket = '${{ steps.metadata.outputs.bucket }}';
const basePath = '${{ steps.metadata.outputs.base_path }}';

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body: `@claude

## 🎯 Task: Quality Evaluation

Preview images have been generated and uploaded to GCS. Please evaluate the implementations.

### Instructions

1. **Preview images location**
- Already downloaded to \`preview_images/\` directory in this repository
- GCS Bucket: \`${bucket}\`
- GCS Path: \`${basePath}/\`

2. **For each preview image:**
- Parse filename to extract: spec_id, library, variant (format: \`{spec_id}_{library}_{variant}.png\`)
- Read corresponding spec file: \`specs/{spec_id}.md\`
- View the preview image carefully using your vision capabilities
- Evaluate against quality criteria listed in the spec

3. **Check each quality criterion:**
- Does it meet ALL quality criteria from the spec?
- Are visual elements clear and readable?
- Are colors appropriate and accessible?
- Is the layout well-structured?
- Score: 0-100 (≥85 to pass)

4. **Generate quality report with:**
- Overall verdict (PASS if all implementations score ≥85, FAIL otherwise)
- Score for each implementation (matplotlib and seaborn separately)
- Specific feedback for each quality criterion (met/failed)
- Strengths and improvements needed
- Concrete, actionable suggestions if FAIL

5. **Update this PR:**
- Post quality report as comment on this PR
- Use \`gh pr edit\` to add appropriate label:
- "quality-approved" if PASS
- "quality-check-failed" if FAIL

### Scoring Guidelines
- 90-100: Excellent - All criteria met, production ready
- 85-89: Good - Minor issues, acceptable
- 75-84: Needs improvement - regeneration recommended
- <75: Rejected - Major issues, regeneration required

Be objective and constructive in your feedback. Focus on measurable criteria.

Preview images are ready - please proceed with the evaluation!`
});
104 changes: 104 additions & 0 deletions .github/workflows/spec-to-code.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
name: Generate Code from Spec

on:
issues:
types: [labeled]

jobs:
trigger-claude-code:
# Only run when "approved" label is added
if: github.event.label.name == 'approved'
runs-on: ubuntu-latest
permissions:
issues: write

steps:
- name: Extract spec ID from issue
id: extract_spec
env:
ISSUE_BODY: ${{ github.event.issue.body }}
ISSUE_TITLE: ${{ github.event.issue.title }}
run: |
# Try to extract spec ID from title (format: "scatter-basic-001: Title" or just "scatter-basic-001")
# Case-insensitive, allows 3-4 digits, converts to lowercase
SPEC_ID=$(echo "$ISSUE_TITLE" | grep -oiP '^[a-z]+-[a-z]+-\d{3,4}' | tr '[:upper:]' '[:lower:]' || echo "")

if [ -z "$SPEC_ID" ]; then
# Try to find spec ID in body (look for markdown heading like "# scatter-basic-001:")
SPEC_ID=$(echo "$ISSUE_BODY" | grep -oiP '^#\s*\K[a-z]+-[a-z]+-\d{3,4}' | tr '[:upper:]' '[:lower:]' || echo "")
fi

if [ -z "$SPEC_ID" ]; then
echo "❌ Could not extract spec ID from issue"
echo "Expected format: {type}-{variant}-{001-9999}"
exit 1
fi

# Validate spec file exists
if [ ! -f "specs/${SPEC_ID}.md" ]; then
echo "⚠️ Warning: Spec file specs/${SPEC_ID}.md does not exist"
echo "Please ensure the spec file is created before code generation"
fi

echo "spec_id=$SPEC_ID" >> $GITHUB_OUTPUT
echo "✅ Extracted spec ID: $SPEC_ID"

- name: Trigger Claude Code with @claude comment
uses: actions/github-script@v7
with:
script: |
const specId = '${{ steps.extract_spec.outputs.spec_id }}';

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `@claude

## 🎯 Task: Generate Plot Implementations

This issue has been approved! Please generate implementations for **${specId}**.

### Instructions

1. **Read the specification**
- File: \`specs/${specId}.md\`
- If file doesn't exist, create it from the issue body

2. **Read generation rules**
- \`rules/generation/v1.0.0-draft/code-generation-rules.md\`
- \`rules/generation/v1.0.0-draft/quality-criteria.md\`
- \`rules/generation/v1.0.0-draft/self-review-checklist.md\`

3. **Generate matplotlib implementation**
- Target: \`plots/matplotlib/scatter/${specId}/default.py\`
- Follow generation rules exactly
- Implement all spec requirements
- Use type hints, docstrings, validation
- Add standalone execution block (\`if __name__ == '__main__':\`)
- Self-review: Max 3 attempts to pass quality criteria

4. **Generate seaborn implementation**
- Target: \`plots/seaborn/scatterplot/${specId}/default.py\`
- Same requirements as matplotlib

5. **Create Pull Request**
- Branch: \`auto/${specId}\`
- Title: \`feat: implement ${specId}\`
- Include summary of generation results
- Link to this issue

### Requirements
- Each implementation must pass self-review before moving to next
- Use deterministic sample data (fixed seeds or hardcoded data)
- Code must work standalone (no external files)
- Follow Python 3.10+ syntax
- Commit with clear messages

### Expected Outcome
- PR created with matplotlib and seaborn implementations
- Both pass self-review
- Tests will run automatically on PR

Please proceed with the generation!`
});
Loading
Loading