Skip to content
Merged
  •  
  •  
  •  
44 changes: 39 additions & 5 deletions .github/workflows/impl-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,15 +144,17 @@ jobs:

3. **Read library rules**: `prompts/library/${{ steps.pr.outputs.library }}.md`

4. **MANDATORY: View the plot image**
4. **Read impl-tags guide**: `prompts/impl-tags-generator.md` (for step 8)

5. **MANDATORY: View the plot image**
- You MUST use the Read tool to open `plot_images/plot.png`
- Visually analyze the image - this is critical for the review
- DO NOT skip this step - a review without seeing the image is invalid
- If the image cannot be read, STOP and report the error

5. **Evaluate against quality criteria** from `prompts/quality-criteria.md`
6. **Evaluate against quality criteria** from `prompts/quality-criteria.md`

6. **Post verdict as PR comment** on PR #${{ steps.pr.outputs.pr_number }}:
7. **Post verdict as PR comment** on PR #${{ steps.pr.outputs.pr_number }}:

```markdown
## AI Review - Attempt ${{ steps.attempts.outputs.display }}/3
Expand Down Expand Up @@ -199,7 +201,7 @@ jobs:
### Verdict: APPROVED / REJECTED
```

7. **Save review data to files** (for the workflow to parse):
8. **Save review data to files** (for the workflow to parse):
```bash
echo "XX" > quality_score.txt

Expand Down Expand Up @@ -235,7 +237,27 @@ jobs:
EOF
```

8. **DO NOT add ai-approved or ai-rejected labels** - the workflow will add them after updating metadata.
9. **Generate impl_tags** (based on prompts/impl-tags-generator.md):
Analyze the implementation code and create impl_tags with 5 dimensions:
- `dependencies`: External packages beyond numpy/pandas/plotting library
- `techniques`: Visualization techniques (twin-axes, colorbar, etc.)
- `patterns`: Code patterns (data-generation, iteration-over-groups, etc.)
- `dataprep`: Data transformations (kde, binning, correlation-matrix, etc.)
- `styling`: Visual style (publication-ready, alpha-blending, etc.)

```bash
cat > review_impl_tags.json << 'EOF'
{
"dependencies": [],
"techniques": ["colorbar", "annotations"],
"patterns": ["data-generation"],
"dataprep": [],
"styling": ["publication-ready"]
}
EOF
```

10. **DO NOT add ai-approved or ai-rejected labels** - the workflow will add them after updating metadata.

**IMPORTANT**: Your review MUST include the "Image Description" section. A review without an image description will be considered invalid.
**IMPORTANT**: All review data (strengths, weaknesses, image_description, criteria_checklist) is saved to metadata for future regeneration. Be specific!
Expand Down Expand Up @@ -314,6 +336,7 @@ jobs:
image_description = None
criteria_checklist = None
verdict = None
impl_tags = None

if Path('review_strengths.json').exists():
try:
Expand Down Expand Up @@ -350,6 +373,13 @@ jobs:
except:
pass

if Path('review_impl_tags.json').exists():
try:
with open('review_impl_tags.json') as f:
impl_tags = json.load(f)
except:
pass

# Load existing metadata
with open(metadata_file, 'r') as f:
data = yaml.safe_load(f)
Expand All @@ -372,6 +402,10 @@ jobs:
if verdict:
data['review']['verdict'] = verdict

# Add impl_tags (issue #2434)
if impl_tags:
data['impl_tags'] = impl_tags

def str_representer(dumper, data):
if isinstance(data, str) and data.endswith('Z') and 'T' in data:
return dumper.represent_scalar('tag:yaml.org,2002:str', data, style="'")
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/spec-create.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ jobs:
6. **Create specification files:**
- Read template: `prompts/templates/specification.md`
- Read metadata template: `prompts/templates/specification.yaml`
- Read tagging guide: `docs/reference/tagging-system.md`
- Read tagging guide: `prompts/spec-tags-generator.md`
- Create directory: `plots/{specification-id}/`
- Create: `plots/{specification-id}/specification.md` (follow template structure)
- Create: `plots/{specification-id}/specification.yaml` with:
Expand Down Expand Up @@ -213,7 +213,7 @@ jobs:
6. **Create specification files:**
- Read template: `prompts/templates/specification.md`
- Read metadata template: `prompts/templates/specification.yaml`
- Read tagging guide: `docs/reference/tagging-system.md`
- Read tagging guide: `prompts/spec-tags-generator.md`
- Create directory: `plots/{specification-id}/`
- Create: `plots/{specification-id}/specification.md` (follow template structure)
- Create: `plots/{specification-id}/specification.yaml` with:
Expand Down
16 changes: 13 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,14 @@ preview_html: null
# Quality
quality_score: 92

# Implementation-level tags (describes HOW the code implements the plot)
impl_tags:
dependencies: [] # External packages (scipy, sklearn, etc.)
techniques: [colorbar, annotations] # Visualization techniques
patterns: [data-generation] # Code patterns
dataprep: [] # Data transformations
styling: [publication-ready] # Visual style

# Review feedback (used for regeneration)
review:
# AI's visual description of the generated plot
Expand Down Expand Up @@ -364,8 +372,8 @@ Quality: 92/100 | Created: 2025-01-10
- **Review feedback** stored in metadata for regeneration (AI reads previous feedback to improve)
- **Extended review data**: `image_description`, `criteria_checklist`, and `verdict` for targeted fixes
- Contributors credited via `suggested` field
- Tags are at spec level (same for all libraries)
- Per-library metadata updated automatically by `impl-review.yml` (quality score, review feedback)
- **Two-level tagging**: Spec-level `tags` describe WHAT is visualized (same for all libraries), impl-level `impl_tags` describe HOW code implements it (per-library)
- Per-library metadata updated automatically by `impl-review.yml` (quality score, review feedback, impl_tags)
- `sync-postgres.yml` workflow syncs to database on push to main
- Database stores full spec content (markdown) and implementation code (Python source)

Expand Down Expand Up @@ -473,7 +481,7 @@ uv run python -c "from core.database import is_db_configured; print(is_db_config
- Spec content (full markdown from specification.md)
- Spec metadata (title, description, tags, structured_tags from specification.yaml)
- Implementation code (full Python source)
- Implementation metadata (library, variant, quality score, generation info from metadata/*.yaml)
- Implementation metadata (library, variant, quality score, impl_tags, generation info from metadata/*.yaml)
- GCS URLs for preview images

**What's in Repository** (source of truth):
Expand Down Expand Up @@ -512,6 +520,8 @@ The `prompts/` directory contains AI agent prompts for code generation, quality
| `quality-evaluator.md` | AI quality evaluation prompt |
| `spec-validator.md` | Validates plot request issues |
| `spec-id-generator.md` | Assigns unique spec IDs |
| `spec-tags-generator.md` | AI rules for spec-level tag assignment |
| `impl-tags-generator.md` | AI rules for impl-level tag assignment |

### Using Prompts

Expand Down
42 changes: 42 additions & 0 deletions alembic/versions/a2f4b8c91d23_add_impl_tags.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
"""add_impl_tags

Add impl_tags JSONB column to impls table for issue #2434:
Implementation-level tags describing HOW code is implemented.
5 dimensions: dependencies, techniques, patterns, dataprep, styling

Revision ID: a2f4b8c91d23
Revises: 6345896e2e90
Create Date: 2026-01-07

"""

from typing import Sequence, Union

import sqlalchemy as sa
from sqlalchemy.dialects import postgresql

from alembic import op


# revision identifiers, used by Alembic.
revision: str = "a2f4b8c91d23"
down_revision: Union[str, None] = "6345896e2e90"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
"""Add impl_tags JSONB column and GIN index to impls table."""
# Add impl_tags column (JSONB for 5 tag dimensions)
op.add_column("impls", sa.Column("impl_tags", postgresql.JSONB(), nullable=True))

# GIN index for fast JSONB containment/existence queries
# Enables efficient filtering by any impl_tags dimension:
# - dependencies, techniques, patterns, dataprep, styling
op.execute("CREATE INDEX ix_impls_impl_tags ON impls USING GIN (impl_tags)")


def downgrade() -> None:
"""Remove impl_tags column and GIN index from impls table."""
op.execute("DROP INDEX ix_impls_impl_tags")
op.drop_column("impls", "impl_tags")
Loading
Loading