Skip to content

feat: Add Stability AI Inpainting v2 block with local and cloud execution modes #1366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

feat: Add Stability AI Inpainting v2 block with local and cloud execution modes #1366

wants to merge 13 commits into from

Conversation

yeldarby
Copy link
Contributor

  • Create v2 of the Stability AI Inpainting block that supports both cloud API and local execution
  • Add dropdown to switch between 'Run Locally' and 'Run via Stability AI Cloud' (default: cloud)
  • Cloud mode functions identically to v1 with required API key
  • Local mode uses Stable Diffusion Inpainting v1.5 via HuggingFace diffusers
  • Add optional parameters for local mode: num_inference_steps, guidance_scale, seed
  • Maintain backward compatibility with v1 input/output formats
  • Add diffusers to transformers requirements
  • Add comprehensive documentation and unit tests
  • Register block in loader while keeping v1 untouched

@yeldarby
feat: Add Stability AI Inpainting v2 block with local and cloud execu…
be7934b 
…tion modes

- Create v2 of the Stability AI Inpainting block that supports both cloud API and local execution
- Add dropdown to switch between 'Run Locally' and 'Run via Stability AI Cloud' (default: cloud)
- Cloud mode functions identically to v1 with required API key
- Local mode uses Stable Diffusion Inpainting v1.5 via HuggingFace diffusers
- Add optional parameters for local mode: num_inference_steps, guidance_scale, seed
- Maintain backward compatibility with v1 input/output formats
- Add diffusers to transformers requirements
- Add comprehensive documentation and unit tests
- Register block in loader while keeping v1 untouched
@yeldarby yeldarby marked this pull request as draft June 17, 2025 22:46
yeldarby and others added 9 commits June 17, 2025 15:52
@yeldarby
fix: Fix mask generation and validation in Stability AI Inpainting v2
75d7718 
- Replace MaskAnnotator with direct mask manipulation to avoid class_id requirement
- Move API key validation from manifest to run method
- Update tests to reflect the validation changes
- Add name field to test manifests as required by WorkflowBlockManifest
@yeldarby
fix: Fix WorkflowImageData initialization in unit test
9246de6 
- Add required parent_metadata argument to WorkflowImageData in test_cloud_inference
@yeldarby
feat: Add caching and better error handling for local SD model
457b325 
- Add pipeline caching to avoid reloading model on each inference
- Add init parameters for proper workflow block initialization
- Improve error messages for missing model files
- Add use_safetensors=True for better compatibility
- Add progress logging when loading the model
@yeldarby
refactor: Simplify local model loading with proper caching
d39e068 
- Use module-level caching for Stable Diffusion pipeline to match other models
- Store models in MODEL_CACHE_DIR for consistency
- Add proper logging with inference logger
- Use torch.inference_mode() for better performance
- Remove complex model manager integration (to be added when loading from Roboflow backend)
- Keep implementation simple and focused on functionality
@yeldarby
fix: Use correct Stable Diffusion inpainting model repository
cade200 
- Switch from discontinued runwayml/stable-diffusion-inpainting to stable-diffusion-v1-5/stable-diffusion-inpainting
- Add variant='fp16' parameter to load model.fp16.safetensors correctly
- Update documentation to reflect the correct model repository
- The new repository contains the actual model weights needed for inference
@yeldarby
fix: Always use fp16 variant for stable-diffusion-inpainting model
15159af 
- The stable-diffusion-v1-5/stable-diffusion-inpainting repo only contains .fp16.safetensors files
- Remove conditional variant selection and always use variant='fp16'
- This fixes the 'diffusion_pytorch_model.safetensors not found' error
- Simplify the loading logic and improve error messages
@yeldarby @github-advanced-security
Potential fix for code scanning alert no. 142: Incomplete URL substri…
8e49ed8 
…ng sanitization

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@grzegorz-roboflow
Merge branch 'main' into feat/stability-ai-inpainting-v2
b473cff
@grzegorz-roboflow
Port preset and invert_segmentation_mask from v1
e7b1b7f
@grzegorz-roboflow
Copy link
Contributor

Ported recent changes made to v1 - preset and invert_segmentation_mask parameters

@yeldarby
Copy link
Contributor Author

@grzegorz-roboflow please do not merge this draft; using it to work w/ a hardware partner on testing some things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grzegorz-roboflow grzegorz-roboflow grzegorz-roboflow approved these changes

@capjamesg capjamesg Awaiting requested review from capjamesg capjamesg is a code owner

@PawelPeczek-Roboflow PawelPeczek-Roboflow Awaiting requested review from PawelPeczek-Roboflow PawelPeczek-Roboflow is a code owner

@hansent hansent Awaiting requested review from hansent hansent is a code owner

@probicheaux probicheaux Awaiting requested review from probicheaux probicheaux is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
DO-NOT-MERGE
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yeldarby @grzegorz-roboflow