Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic Seamless Detection #410

Merged
merged 36 commits into from
Jan 1, 2023
Merged

Automatic Seamless Detection #410

merged 36 commits into from
Jan 1, 2023

Conversation

NullSenseStudio
Copy link
Collaborator

Adds automatic seamless axis detection to simplify source image and upscaling (when supported) workflows. Currently only shows the detected result, but once finished it'll have options to automatically apply the result, prompt the user to change, or have detection disabled.

blender_oORSDXZFIt

Detection is done through a custom trained model on 2000 generated images with 2x and 4x Real-ESRGAN upscaled versions of each. Accuracy is 99.75% per edge or 99.5% per image and may vary somewhat depending on the type of textures that're run through it.

@carson-katri
Copy link
Owner

Looks great! Would you want to include the training script (possibly with a top level scripts folder)?

My suggestion for the interface would be to change the "Seamless" checkbox to a dropdown with the options:

  • Automatic
  • X
  • Y
  • Both
  • Off

where "Automatic" is the default (or maybe "Auto-detect"). What do you think?

@carson-katri carson-katri added the enhancement New feature or request label Dec 6, 2022
@NullSenseStudio
Copy link
Collaborator Author

I plan on adding the training script but it's quite a mess right now and really should have comments explaining certain parts of it.

I think naming the option "Auto-detect" would be suitable.

Base automatically changed from diffusers to main December 6, 2022 17:53
@NullSenseStudio NullSenseStudio marked this pull request as ready for review December 26, 2022 23:34
@NullSenseStudio
Copy link
Collaborator Author

Seamless axes are now handled as a single enum property with the detected result appearing below, and will only run if source image is checked and has modify or inpaint selected. It can also now look through prompt history for how an image was generated and fallback to the trained model if nothing was found.
blender_tTPsTDPqnv
It's also possible to use with texture projection and the render pass, but the user won't be notified of what seamless mode it's using. It might be better to change the default to off instead of auto-detect as I don't see it being likely to be used outside of modifying or inpainting.

Copy link
Owner

@carson-katri carson-katri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just a couple suggestions.

property_groups/seamless_result.py Outdated Show resolved Hide resolved
ui/panels/upscaling.py Outdated Show resolved Hide resolved
ui/panels/dream_texture.py Outdated Show resolved Hide resolved
property_groups/seamless_result.py Outdated Show resolved Hide resolved
generator_process/actions/detect_seamless/__init__.py Outdated Show resolved Hide resolved
NullSenseStudio and others added 3 commits January 1, 2023 12:54
Co-authored-by: Carson Katri <Carson.katri@gmail.com>
Co-Authored-By: Carson Katri <Carson.katri@gmail.com>
Co-authored-by: Carson Katri <Carson.katri@gmail.com>
@NullSenseStudio
Copy link
Collaborator Author

Do you normally squash and merge when complete?

@carson-katri
Copy link
Owner

Yep

@carson-katri carson-katri merged commit f8f5c37 into main Jan 1, 2023
@NullSenseStudio NullSenseStudio deleted the detect-seamless branch January 1, 2023 20:44
This was referenced Jan 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants