feat: add sprite sheet support for 3D model thumbnails

[devonjones]

Implements sprite sheet generation and interactive viewing for OpenForge 3D models, reducing storage costs and improving user experience with 360° model previews.

Backend Changes

• Add sprite_metadata JSONB column to images table (schema v15-16)
• Update image SQL queries to handle sprite_metadata
• Add /api/sprites endpoint for sprite sheet generation
• Fix image comparison to detect sprite_metadata changes
• Update insert_image() to use ON CONFLICT DO UPDATE for sprite metadata

Scanner Improvements

• Add --sprites flag to generate sprite sheets during scan
• Add bin/generate_sprite tool for sprite sheet creation
• Add bin/resort_fixtures tool for stable fixture sorting
• Fix fixture sorting to use full_name for top-level array
• Use /tmp for temp sprite files to avoid path length issues
• Preserve sprite_metadata.angles sort order by index

Frontend Components

• Add SpriteViewer component with interactive 3D rotation
• Implement unwrapped cube layout for angle selection (10 angles)
• Add keyboard navigation (arrow keys for rotation)
• Add mouse drag support for horizontal rotation
• Auto-focus viewer for immediate keyboard interaction
• Reset angle when switching between blueprints

Type Definitions

• Add SpriteThumbnailData, SpriteAngle types
• Update ThumbnailData union type
• Add comprehensive test coverage (27 tests for SpriteViewer)

Testing

• Add sprite_metadata change detection test
• Update image-gallery tests for sprite support
• Add full SpriteViewer test suite (rendering, controls, keyboard, mouse, a11y)

Next Steps

After this PR is merged, a follow-up PR will update fixture files with sprite_metadata for existing blueprints.

🤖 Generated with Claude Code

[gemini-code-assist]

Summary of Changes

Hello @devonjones, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly upgrades the 3D model thumbnail system by introducing sprite sheet functionality. This change allows for interactive, multi-angle previews of models directly within the application, enhancing user experience and potentially optimizing storage. The update spans across the backend, scanner, and frontend, ensuring a seamless integration of the new interactive viewing capabilities while maintaining backward compatibility with existing single thumbnails.

Highlights

• Sprite Sheet Support: Introduced comprehensive support for generating and displaying sprite sheets for 3D model thumbnails, enabling interactive 360° previews.
• Interactive Frontend Viewer: Added a new SpriteViewer React component that allows users to interactively rotate 3D model thumbnails using keyboard navigation (arrow keys) and mouse drag, with a visual 'unwrapped cube' control layout.
• Backend API & Database Changes: Implemented new API endpoints to fetch thumbnail variants (sprite or legacy) and to set the default viewing angle for sprite sheets. The database schema was updated with a sprite_metadata JSONB column in the images table to store sprite sheet details.
• Automated Sprite Generation: Integrated sprite sheet generation into the file scanning pipeline with a new --sprites flag. A standalone bin/generate_sprite CLI tool was added for on-demand sprite sheet creation.
• Improved Fixture Handling: Enhanced fixture sorting logic to ensure stable ordering, particularly for top-level blueprint arrays and sprite angle metadata, and added a bin/resort_fixtures tool.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces comprehensive support for sprite sheet thumbnails, including backend API changes, database schema updates, new CLI tools, and a new interactive frontend component. The implementation is well-structured, with good test coverage and thoughtful features like keyboard and mouse controls in the SpriteViewer. The refactoring to centralize thumbnail creation logic is a solid improvement. My feedback focuses on enhancing maintainability, portability, and efficiency in a few areas, such as removing magic numbers, avoiding hardcoded paths, and improving file processing.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent pull request that introduces a significant and valuable feature: sprite sheet thumbnails for 3D models. The changes are comprehensive, touching the backend, scanner, and frontend, and are implemented with high quality.

Key highlights include:

• Robust Backend: The database schema change is clean, and the use of ON CONFLICT DO UPDATE in images.py is a great improvement for atomicity and performance. The new API endpoints are well-designed and handle both new sprite sheets and legacy thumbnails gracefully.
• Improved Scanner Logic: The scanner and incremental processing scripts are updated to be resilient to thumbnail generation failures. The fix to the fixture sorting logic to ensure stable, deterministic output is a crucial improvement for repository maintenance.
• Well-crafted Tooling: The new bin/generate_sprite script is a great addition for testing and manual generation.
• Rich Frontend Experience: The new SpriteViewer component provides an interactive and user-friendly way to view models from multiple angles, complete with keyboard and mouse controls.
• Excellent Documentation & Testing: The inclusion of a design document, progress report, and a comprehensive test suite for the new components demonstrates a thorough and professional approach.

I have one minor suggestion in openforge/data/io.py to improve portability, but overall this is a fantastic contribution to the project.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an impressive and comprehensive pull request that successfully implements sprite sheet support for 3D model thumbnails. The changes span the entire stack, from database schema and backend APIs to CLI tools and frontend components, and are executed to a high standard. The new SpriteViewer component is particularly well-crafted, with excellent interactive controls and accessibility considerations. The backend changes, including the use of a feature flag and robust fallback mechanisms, are well-designed for a smooth rollout.

I have identified one potentially critical issue where a new API endpoint might be calling a database function that hasn't been updated to support the new data structure, which could cause the feature to fail. I've also noted a TODO in a new CLI tool where functionality is incomplete.

It's also worth noting that the database migration for user authentication (version_15.py) is a significant and unrelated feature. While it doesn't seem to cause any issues, it would be better to submit such large, unrelated changes in a separate pull request to keep the review process focused.

Overall, this is a fantastic contribution. Addressing the points I've raised will ensure the new functionality is fully robust.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent and comprehensive pull request that successfully implements sprite sheet support for 3D model thumbnails. The changes are well-structured across the backend, frontend, and CLI tools. The new SpriteViewer component is particularly impressive with its interactive controls. The backend changes are robust, including graceful fallbacks for legacy thumbnails and efficient database updates. One suggestion for improvement would be to separate the user authentication database migration (version_15.py) into its own pull request to keep this one focused solely on the sprite sheet feature. Overall, this is a high-quality contribution.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces comprehensive support for sprite sheet thumbnails, which is a significant feature enhancement. The changes span the backend, frontend, and CLI tooling, and are well-executed. Key improvements include a new sprite_metadata column in the database, corresponding API endpoints, a robust SpriteViewer React component with interactive controls, and new CLI tools to support sprite sheet generation. The code is well-structured, and the feature is implemented with backward compatibility in mind, gracefully handling legacy thumbnails. I have one piece of feedback regarding an incomplete feature in the new generate_sprite tool. Overall, this is an excellent contribution.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent and comprehensive pull request that adds significant new functionality for 3D model thumbnails. The implementation of sprite sheet generation on the backend is robust, with good use of temporary files and error handling. The new CLI tools (generate_sprite, resort_fixtures) are valuable additions for development and maintenance. The database schema changes and SQL updates are well-designed, especially the use of ON CONFLICT DO UPDATE for idempotency. The frontend SpriteViewer component is impressive, with interactive controls for keyboard and mouse, and a thoughtful UI design. The test coverage for the new frontend components is thorough, and the updates to existing tests are correct. The inclusion of a detailed design document (sprite-sheet-storage-design.md) is highly appreciated and sets a great standard. I have one minor point of feedback regarding an incomplete feature in a new CLI tool.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces comprehensive support for sprite sheet thumbnails, which is a great enhancement for performance and user experience. The changes are well-implemented across the backend, scanner, and frontend. The new SpriteViewer component is interactive and feature-rich. The database and API changes are solid, especially the move to ON CONFLICT DO UPDATE for more atomic upserts. The code is generally clean and includes good test coverage for the new features.

I have a couple of suggestions. One is to complete the --db-update functionality in the new generate_sprite tool. Another is a suggestion to refactor the SpriteViewer component by extracting event handling logic into custom hooks for better maintainability. Overall, this is an excellent contribution.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new multi-angle thumbnail sprite sheet system for STL files, enhancing the OpenForge catalog's visual capabilities. Key changes include adding a new sprite_metadata JSONB column to the images table in the database (version 16 schema migration) to store sprite sheet configuration, camera angles, and default view. A new bin/generate_sprite CLI tool is added to create these sprite sheets using stl-thumb and Pillow, supporting R2 upload and local file cleanup. The bin/dropbox_scanner and openforge/data/incremental.py scripts are updated to optionally generate and upload these sprite sheets, gracefully handling failures. New API endpoints (/api/blueprints/<blueprint_id>/thumbnail-variants and /api/blueprints/<blueprint_id>/thumbnail-variants/default-angle) are introduced to retrieve sprite data and allow authenticated users to set default angles. The frontend (src/components/blueprint/image-gallery.tsx and src/components/blueprint/sprite-viewer.tsx) is updated to fetch this new thumbnail data and display an interactive SpriteViewer component, allowing users to rotate views via mouse drag or keyboard arrows, with a fallback to legacy single thumbnails. Additionally, new beads configuration and documentation files (.beads/.gitignore, .beads/README.md, .beads/config.yaml, AGENTS.md) are added, along with minor markdown formatting fixes across various documentation files.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is a fantastic and comprehensive feature addition. The implementation of sprite sheet support is well-thought-out, covering backend changes, new CLI tooling, and a polished interactive frontend component. The new SpriteViewer is particularly impressive with its support for keyboard and mouse controls. The accompanying design document and extensive test suite demonstrate great attention to detail and quality. I have a couple of minor suggestions for improvement, but overall this is excellent work.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new multi-angle sprite sheet thumbnail system for STL models, replacing or complementing existing single thumbnails. Key changes include adding a .beads directory with configuration and issue tracking for the new feature, updating .gitignore and .gitattributes for Beads integration, and adding new documentation for AI agents and code organization philosophy. A new Python script bin/generate_sprite is added to create sprite sheets using stl-thumb and Pillow, with options for R2 upload. The bin/dropbox_scanner is updated to support generating these sprite sheets via a new --sprites flag. The database schema is extended with a sprite_metadata JSONB column in the images table to store sprite sheet details. New API endpoints are introduced to retrieve thumbnail variants and set the default angle for sprite sheets. The frontend is updated to use a new SpriteViewer component that dynamically displays sprite sheets with keyboard and mouse drag controls, falling back to legacy single images if sprite data is unavailable. Review comments highlighted the need for robust error handling and cleanup of temporary files in the sprite generation process, ensuring currentAngle state updates correctly in the SpriteViewer when thumbnailData changes, making the default_angle validation more dynamic, and simplifying a PostgreSQL INSERT statement.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces significant new functionality for displaying 3D model thumbnails using sprite sheets. The changes are comprehensive, touching the backend, database, CLI tools, and frontend. The backend work includes a database migration for sprite_metadata, new API endpoints for fetching thumbnail variants and setting default angles, and improved SQL queries. The new CLI tools for generating sprites and resorting fixtures are valuable additions. The frontend implementation of the SpriteViewer is impressive, with interactive controls for rotation via mouse drag and keyboard, and a well-thought-out UI. The test coverage for the new components is excellent.

My review includes a couple of suggestions for the frontend to improve maintainability and robustness, particularly around how the SpriteViewer handles its state and maps angles for its controls. Overall, this is a high-quality contribution that greatly enhances the user experience for model previews.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent pull request that introduces significant new functionality for sprite sheet thumbnails. The changes are comprehensive, covering the backend, database, frontend, and tooling. The code is well-structured, robust, and includes thorough documentation and testing. The implementation of backward compatibility for legacy thumbnails is also handled very well. I have one suggestion to improve the error reporting in the new CLI tool for better debuggability.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent and comprehensive pull request that adds significant new functionality with sprite sheet support for 3D model thumbnails. The changes span the entire stack, from backend database and API updates to new CLI tools and a sophisticated interactive frontend component. The implementation is robust, with thoughtful details like graceful fallbacks for legacy thumbnails, stable sorting for fixture files, and thorough test coverage. The new SpriteViewer component is particularly well-crafted with support for both keyboard and mouse controls.

I have a couple of suggestions for improving the error handling in the new generate_sprite CLI tool to make it more robust and reusable.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive multi-angle sprite sheet thumbnail system for STL models. Key changes include adding a new .beads/README.md and config.yaml for AI-native issue tracking, along with new issue data in .beads/issues.jsonl related to the sprite sheet project. A new bin/generate_sprite CLI tool is added for generating these sprite sheets, and the bin/dropbox_scanner is updated to optionally enable sprite sheet generation. A detailed docs/sprite-sheet-storage-design.md outlines the R2 storage structure, grid layout, camera angles, and integration. Backend API routes (/api/blueprints/<blueprint_id>/thumbnail-variants and /api/blueprints/<blueprint_id>/thumbnail-variants/default-angle) are added to serve and manage sprite sheet metadata. The database schema is updated with a sprite_metadata JSONB column in the images table (version 16) to store this information. Python utility functions in openforge/data/io.py are refactored to support both single and sprite sheet thumbnail creation and upload, and openforge/data/scanner.py and openforge/db/fixtures/incremental.py are updated to use this new functionality, including a more robust image comparison for incremental updates. The frontend (src/components/blueprint/image-gallery.tsx and src/components/blueprint/sprite-viewer.tsx) is enhanced to display these interactive sprite sheets, with keyboard and mouse drag controls. Minor formatting fixes and documentation updates are also included across various files. A review comment suggests improving security by explicitly listing required environment variables instead of copying the entire os.environ dictionary when configuring the S3 client.

[devonjones]

/gemini review

[gemini-code-assist]

Code Review

This is an excellent and comprehensive pull request that successfully implements sprite sheet support for 3D model thumbnails. The changes are well-structured across the backend, frontend, and tooling. The new /api/sprites endpoint, database schema changes, and scanner improvements are solid. The SpriteViewer component is a great addition to the frontend, providing an interactive 360° preview. The test coverage, including a full suite for the new component, is impressive and ensures high quality. I have one suggestion to improve the robustness of the SpriteViewer component by removing a dependency on hardcoded angle indices, which will make it more resilient to future changes in the backend data.

[gemini-code-assist]

The getAngleIndices function has hardcoded fallbacks for topIndex and bottomIndex, and the useKeyboardRotation hook assumes a fixed ordering of angles (e.g., horizontal angles are always indexed less than vertical ones). This creates a fragile coupling with the backend's SPRITE_ANGLES data structure. If the angle definitions change on the backend, keyboard navigation could break.

This refactoring makes the logic more robust by:

1. Removing hardcoded fallback indices in getAngleIndices.
2. Explicitly checking if an angle is horizontal in useKeyboardRotation instead of relying on its index.
3. Gracefully handling cases where 'top' or 'bottom' angles might not be found.

function getAngleIndices(angles: SpriteThumbnailData['angles']) {
  const topIndex = angles.findIndex(a => a.name === 'top');
  const bottomIndex = angles.findIndex(a => a.name === 'bottom');

  // Horizontal angles are those that are not top or bottom.
  const horizontalIndices = new Set(
    angles.filter(a => a.name !== 'top' && a.name !== 'bottom').map(a => a.index)
  );

  return {
    horizontalCount: horizontalIndices.size,
    horizontalIndices,
    topIndex, // Will be -1 if not found
    bottomIndex, // Will be -1 if not found
  };
}

/**
 * Custom hook for keyboard-based sprite rotation
 */
function useKeyboardRotation(
  currentAngle: number,
  setCurrentAngle: (angle: number | ((prev: number) => number)) => void,
  angleIndices: ReturnType<typeof getAngleIndices>
) {
  const { horizontalCount, horizontalIndices, topIndex, bottomIndex } = angleIndices;

  const handleKeyDown = useCallback((e: React.KeyboardEvent) => {
    const isHorizontal = horizontalIndices.has(currentAngle);

    if (e.key === 'ArrowLeft') {
      e.preventDefault();
      setCurrentAngle(prev => {
        if (!isHorizontal) return 0;
        // Assumes horizontal indices are 0 to horizontalCount-1
        return prev <= 0 ? horizontalCount - 1 : prev - 1;
      });
    } else if (e.key === 'ArrowRight') {
      e.preventDefault();
      setCurrentAngle(prev => {
        if (!isHorizontal) return 0;
        // Assumes horizontal indices are 0 to horizontalCount-1
        return prev >= horizontalCount - 1 ? 0 : prev + 1;
      });
    } else if (e.key === 'ArrowUp' && topIndex !== -1) {
      e.preventDefault();
      setCurrentAngle(topIndex);
    } else if (e.key === 'ArrowDown' && bottomIndex !== -1) {
      e.preventDefault();
      setCurrentAngle(bottomIndex);
    }
  }, [setCurrentAngle, topIndex, bottomIndex, horizontalCount, horizontalIndices, currentAngle]);

  return { handleKeyDown };
}