feat: responsive image delivery with multi-size PNG + WebP#5192
feat: responsive image delivery with multi-size PNG + WebP#5192MarkusNeusinger merged 4 commits intomainfrom
Conversation
Backend: add create_responsive_variants() to generate 400/800/1200px PNGs and WebPs plus full-size WebP from source plot.png. Frontend: replace <img> with <picture> + srcSet/sizes in ImageCard, SpecDetailView, and SpecOverview. Browser auto-selects optimal size based on viewport and DPR. Falls back to existing thumb/url if responsive variants are not yet available. https://claude.ai/code/session_01WUQ7f2vcScXKtqLKgfdRtw
…iants - Removed preview_thumb references from metadata and code - Added functionality to generate responsive image variants (PNG and WebP) - Updated upload process to handle multiple image formats
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
Implements responsive plot image delivery by generating and serving multi-size PNG/WebP variants (400/800/1200) derived from preview_url, while removing the legacy preview_thumb field across backend, frontend, workflows, tests, and documentation.
Changes:
- Backend adds responsive variant generation and updates scripts/workflows to produce and upload all variants.
- Frontend switches plot rendering to
<picture>withsrcSet/sizesand migration fallback toplot.png. - API/contracts/docs/tests are updated to remove
preview_thumb/thumb.
Reviewed changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/automation/scripts/test_sync_to_postgres.py | Updates expected synced implementation payloads to remove preview_thumb. |
| tests/unit/api/test_routers.py | Removes thumb-related fixtures/attributes from router unit tests. |
| tests/unit/api/test_main.py | Removes thumb-related fixtures/attributes from app startup/unit tests. |
| tests/unit/api/mcp/test_tools.py | Removes preview_thumb usage in MCP tool tests. |
| tests/e2e/conftest.py | Removes thumb URL constant and preview_thumb from seeded e2e data. |
| tests/conftest.py | Removes thumb URL constant and preview_thumb from seeded test data. |
| scripts/remove_preview_thumb_from_yaml.py | Adds one-time script to strip preview_thumb from metadata YAML files. |
| scripts/backfill_responsive_images.py | Adds backfill script to generate/upload responsive variants for historical GCS images. |
| prompts/templates/metadata.yaml | Updates metadata template to only include preview_url (variants derived by convention). |
| prompts/templates/library-metadata.yaml | Updates per-library metadata template to remove preview_thumb URL. |
| docs/reference/repository.md | Updates repository reference docs to remove preview_thumb and legacy thumbnail naming. |
| docs/reference/database.md | Updates DB reference docs to remove preview_thumb from examples/schema description. |
| docs/reference/api.md | Updates API reference docs to remove preview_thumb/thumb from payload examples. |
| core/images.py | Adds create_responsive_variants() and CLI command for generating variants. |
| core/database/repositories.py | Removes preview_thumb from repository field sets used for DB updates. |
| automation/scripts/sync_to_postgres.py | Stops reading/updating preview_thumb from metadata during DB sync. |
| app/src/utils/responsiveImage.ts | Adds utilities to derive variant URLs and responsive sizes/srcSet from plot.png. |
| app/src/types/index.ts | Removes thumb/preview_thumb from frontend types. |
| app/src/pages/CatalogPage.tsx | Switches catalog hero/preview image rendering to <picture> with responsive variants + fallback. |
| app/src/components/SpecOverview.tsx | Updates implementation cards to render responsive images via <picture>. |
| app/src/components/SpecDetailView.tsx | Updates detail view image to responsive <picture> with fallback handling. |
| app/src/components/ImageCard.tsx | Replaces CardMedia with <picture> and uses responsive utilities + fallback logic. |
| app/src/components/ImageCard.test.tsx | Updates tests to assert new fallback src behavior (plot_800.png). |
| api/schemas.py | Removes preview_thumb/thumb fields from Pydantic response models. |
| api/routers/specs.py | Removes preview_thumb from spec response and spec-images payload. |
| api/routers/plots.py | Removes thumb from collected image dicts for filtered plot responses. |
| api/routers/og_images.py | Updates OG collage generation to fetch preview_url (no longer prefers thumbnail). |
| api/routers/libraries.py | Removes thumb from library images endpoint payload. |
| api/mcp/server.py | Removes preview_thumb from MCP spec detail responses (but not fully from file). |
| agentic/docs/project-guide.md | Updates project guide metadata example to remove preview_thumb. |
| agentic/commands/update.md | Updates update command docs to remove thumbnail generation/upload steps. |
| .github/workflows/impl-repair.yml | Generates responsive variants in repair workflow and uploads all plot image variants. |
| .github/workflows/impl-generate.yml | Generates responsive variants in generate workflow and uploads all plot image variants. |
| for width in target_sizes: | ||
| # Skip sizes larger than the original | ||
| if width >= img.width: | ||
| resized = img | ||
| actual_width, actual_height = img.width, img.height | ||
| else: | ||
| ratio = width / img.width | ||
| actual_width = width | ||
| actual_height = int(img.height * ratio) | ||
| resized = img.resize((actual_width, actual_height), Image.Resampling.LANCZOS) | ||
|
|
||
| for ext, fmt, opts in RESPONSIVE_FORMATS: | ||
| out_path = output_dir / f"plot_{width}.{ext}" | ||
| resized.save(out_path, fmt, optimize=True, **opts) |
There was a problem hiding this comment.
In create_responsive_variants(), when the source image is narrower than a target width, the code reuses the original image but still writes it to a filename like plot_1200.png and the frontend will advertise it as 1200w in srcset. That breaks srcset semantics (declared width != actual) and can cause the browser to pick an overly-large candidate and render a blurry/upscaled image. Consider either (a) always resizing/upscaling to the requested width so the file and descriptor match, or (b) explicitly enforce a minimum source width (raise/log+skip generation) so plot.png is guaranteed to be >= max responsive size.
| img = Image.open(input_path) | ||
| if img.mode in ("RGBA", "P"): | ||
| img = img.convert("RGB") |
There was a problem hiding this comment.
create_responsive_variants() opens the source image via Image.open() without closing it. This can leak file descriptors and also interfere with deleting the input file on some platforms (notably Windows) in batch/backfill scripts. Prefer using a context manager (with Image.open(...) as img:) and copying the image object before converting/resizing as needed.
| img = Image.open(input_path) | |
| if img.mode in ("RGBA", "P"): | |
| img = img.convert("RGB") | |
| with Image.open(input_path) as src: | |
| if src.mode in ("RGBA", "P"): | |
| img = src.convert("RGB") | |
| else: | |
| img = src.copy() |
| def create_responsive_variants( | ||
| input_path: str | Path, output_dir: str | Path, sizes: list[int] | None = None, optimize: bool = True | ||
| ) -> list[dict[str, str | int]]: | ||
| """Generate multi-size, multi-format image variants for responsive delivery. |
There was a problem hiding this comment.
New responsive variant generation logic (create_responsive_variants) isn’t covered by unit tests, while this module already has tests for create_thumbnail/process_plot_image (tests/unit/core/test_images.py). Adding tests that assert expected files are created, dimensions match requested widths, and WebP/PNG variants are generated (including behavior when the source is smaller than 1200px) would help prevent regressions.
| try: | ||
| # Fetch all images in parallel — prefer thumbnails (smaller, faster) | ||
| images = list( | ||
| await asyncio.gather(*[_fetch_image(impl.preview_thumb or impl.preview_url) for impl in selected_impls]) | ||
| ) | ||
| # Fetch all images in parallel | ||
| images = list(await asyncio.gather(*[_fetch_image(impl.preview_url) for impl in selected_impls])) | ||
| labels = [f"{spec_id} · {impl.library_id}" for impl in selected_impls] |
There was a problem hiding this comment.
get_spec_collage_image now fetches impl.preview_url (full plot.png) for up to 6 images. That can significantly increase bytes transferred and image processing time for OG image generation. Since responsive variants are now available by convention, consider fetching a smaller variant (e.g., plot_400.png/plot_800.png) and falling back to plot.png if the variant 404s during the migration window.
| impl_response = ImplementationResponse( | ||
| library_id=impl.library.id, | ||
| library_name=impl.library.name, | ||
| preview_url=impl.preview_url, | ||
| preview_thumb=impl.preview_thumb, | ||
| preview_html=impl.preview_html, | ||
| quality_score=impl.quality_score, | ||
| code=impl.code, |
There was a problem hiding this comment.
preview_thumb was removed from API schemas, but this file still passes preview_thumb in get_implementation (later in the file). With Pydantic v2 this will be treated as an extra field and ignored, which can mask drift between the MCP server output and the declared schema. Remove the remaining preview_thumb usage so MCP responses consistently match ImplementationResponse.
- Fix missed preview_thumb in MCP server get_implementation - Use context manager for Image.open() to prevent fd leaks - Skip responsive variants larger than source (correct srcset semantics) - OG images: try plot_800.png first for efficiency, fallback to plot.png - Add 7 unit tests for create_responsive_variants() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
plot_thumb.pngwith responsive multi-size PNG + WebP variants (400/800/1200px) for all plot images<picture>elements withsrcSet+sizeslet the browser pick optimal size based on viewport and DPRpreview_thumbfield from API, database sync, metadata templates, and all documentationplot.pngduring migration (before backfill runs)Image savings (scatter-basic/matplotlib example)
Changes (33 files)
create_responsive_variants()incore/images.pygenerates all 7 variants<picture>+srcSetin ImageCard, SpecDetailView, SpecOverview, CatalogPageimpl-generate.ymlandimpl-repair.ymlgenerate + upload all variantspreview_thumb/thumbfrom schemas, routers, MCP serverbackfill_responsive_images.pyfor historical plots,remove_preview_thumb_from_yaml.pyfor YAML cleanupPost-merge steps
python scripts/backfill_responsive_images.pyto generate variants for ~2,668 existing images (~2h)python scripts/remove_preview_thumb_from_yaml.py→ commit YAML cleanup separatelypreview_thumbDB columnCloses #5191
Test plan
🤖 Generated with Claude Code