Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Real-ESRGAN integration #233

Merged
merged 6 commits into from
Oct 4, 2022
Merged

Add Real-ESRGAN integration #233

merged 6 commits into from
Oct 4, 2022

Conversation

carson-katri
Copy link
Owner

A new "AI Upscaling" panel has been added. I also refactored the generator process a bit to allow for multiple "Intents" (the two current ones being PROMPT_TO_IMAGE and UPSCALE)
Screen Shot 2022-10-02 at 9 48 38 PM

@NullSenseStudio
Copy link
Collaborator

Due to the stable diffusion branch change I've been getting ImportError: DLL load failed while importing cv2: The specified module could not be found. on line from stable_diffusion.ldm.generate import Generate while debugging, and oddly no exception is caught when starting Blender normally and just get Process closed unexpectedly.

For some reason Blender puts its python3.dll in Blender's root directory rather than in blender/3.3/python/bin. I've had no luck with changing the subprocess's cwd or modifying the PATH environment variable to help the subprocess find it.

So far only workarounds I've got are copying/symlink python3.dll into blender/3.3/python/bin, running the subprocess from a standard python runtime, or with some added code before that import:

if sys.platform == 'win32':
    from ctypes import WinDLL
    WinDLL(r"C:\Blender\stable\blender-3.3.0+lts.0759f671ce1f\python3.dll")

We should be able to get a proper path for other systems by modifying bpy.app.binary_path and sending to the subprocess as a command line argument.

@carson-katri
Copy link
Owner Author

@NullSenseStudio Thanks for catching this, I hadn’t gotten to Windows testing yet. Would you be able to implement that suggestion as a PR against this branch? Also I assume the DLL is at the root for blender installations in Program Files as well?

@NullSenseStudio
Copy link
Collaborator

I should have time to do that later today. I'll also double check if the solution works for the msi installer and on steam.

@carson-katri carson-katri added this to the v0.0.7 milestone Oct 3, 2022
@NullSenseStudio
Copy link
Collaborator

PR has been made and can confirm that zip, msi, and steam versions all had the same issue and worked with the fix.

@carson-katri
Copy link
Owner Author

@NullSenseStudio Can you give this a try before I merge it?

Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's some issues I ran into.

operators/dream_texture.py Outdated Show resolved Hide resolved
generator_process.py Outdated Show resolved Hide resolved
generator_process.py Outdated Show resolved Hide resolved
generator_process.py Outdated Show resolved Hide resolved
operators/dream_texture.py Show resolved Hide resolved
@carson-katri
Copy link
Owner Author

carson-katri commented Oct 4, 2022

FYI, I added a "Full Precision" option to the upscaler but default to enabled. The precision options for the Dream Texture panel has been overhauled as well, and use an "Automatic" option now which can autodetect 16 series cards (due to invoke-ai enhancements).

@NullSenseStudio
Copy link
Collaborator

Nice addition. Looks like everything is working well now.

Did there used to be a strength option for upscaling or am I misremembering?

Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All previous issues seem fixed.

@carson-katri
Copy link
Owner Author

Thanks for the reviews! I removed strength (for now) because it required two weight files to be downloaded which could be confusing, and it also is not available in 0.2.5.0 as far as I could tell (the dni_weight argument)?

@carson-katri carson-katri merged commit 821355e into main Oct 4, 2022
@carson-katri carson-katri deleted the real-esrgan branch October 4, 2022 01:42
JasonHoku pushed a commit to JasonHoku/dream-textures that referenced this pull request Dec 20, 2022
* Add Real-ESRGAN integration

* Use SharedMemory to pass image data around and support outscale argument

* Updates for latest submodule changes

* fix cv2 ImportError (carson-katri#237)

* Support Image Texture nodes

* Revisions from review

Co-authored-by: NullSenseStudio <47096043+NullSenseStudio@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants