Note on Python 3.11

[vladmandic]

Note on Python 3.11

Python 3.11 support is experimental at the moment and some functionality will be disabled:

• numba is not compatible so facexlib, gfpgan and realesrgan modules will not be available
• rembg is not compatible so stable-diffusion-webui-rembg extension will not be available

All errors should be reported on console, but still automatically handled, so core operations with Python 3.11 are possible.

Recommended version of Python is 3.10

[shawncaple]

I currently have Python 3.11.3 on my MacOS. I have no idea how to fix the below error. I've looked around and rolling back my python looks dangerous for my computer OS integrity. Any ideas? Sorry if this has been solved, it gets hard to keep track of all the discussions.

Here is the error:

Launching launch.py...
################################################################
Python 3.11.3 on Darwin
Traceback (most recent call last):
File "/Users/xxxx/automatic/launch.py", line 88, in
setup.run_setup(False)
File "/Users/xxxx/automatic/setup.py", line 363, in run_setup
check_python()
File "/Users/xxxx/automatic/setup.py", line 143, in check_python
raise RuntimeError(f"Incompatible Python version: {sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro} required 3.9-3.11")
RuntimeError: Incompatible Python version: 3.11.3 required 3.9-3.11
xxxx@xxxx-iMac automatic %

[vladmandic]

The torch variable should work either way, Conda should not be required. But I don't have Mac, so no way of testing.

[shawncaple]

After a lot of failure, I installed and setup anaconda, and here is where I'm stuck:

(base) xxxx@xxxx-iMac ~ % conda env create -f environment-mac.yaml
EnvironmentFileNotFound: '/Users/xxxx/environment-mac.yaml' file not found
(base) xxxx@xxxx-iMac ~ %

I cannot execute the above code. So I can't "export PYTORCH_ENABLE_MPS_FALLBACK=1"

Biggest tease is getting the GUI to work and then not being able to use it.

[vladmandic]

sorry, i don't know what environment-mac.yaml is, that comes from outside of my code.

[shawncaple]

ok, let me back up:

How can I execute this code so I can generate images? What are the steps?

set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1

[vladmandic]

i don't know - i don't have apple m1, sorry.
the instructions you've posted clearly mention a 3rd party library: git checkout apple-silicon-mps-support - i don't know what that does or how to make it work.
i hope someone else can help.
once there are clear steps, i can integrate as much as possible and make it a one-step solution. but someone needs to provide clear verifiable steps - i cannot integrate that library blindly and "hope it works"

[shawncaple]

I understand. I’m on an intel Mac. I thought this bit of code could be added to something like the webui.sh or something. Shawn

On Fri, Apr 14, 2023 at 9:40 PM Vladimir Mandic ***@***.***> wrote: i don't know - i don't have apple m1, sorry. the instructions you've posted clearly mention a 3rd party library: git checkout apple-silicon-mps-support - i don't know what that does or how to make it work. i hope someone else can help. once there are clear steps, i can integrate as much as possible and make it a one-step solution. but someone needs to provide clear verifiable steps - i cannot integrate that library blindly and "hope it works" — Reply to this email directly, view it on GitHub <#110 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4TRMS5JVWO2JVS42NEI3GTXBH37PANCNFSM6AAAAAAW5L363E> . You are receiving this because you commented.Message ID: ***@***.***>

-- With Regards, Shawn Caple

[vladmandic]

it probably could, but i need someone to verify that package apple-silicon-mps-support, i don't want to do it blindly.

[HubKing]

Just asking, but why do Python apps require that specific version of the runtime? I mean, it is not Python 4.0 or something. Just 3.10 and 3.11.3. I don't know Python well, but that looks like a minor change. And shouldn't 3.11.3 have backward compatibility with 3.9? I mostly use C#, and I don't think I had this kind of situation with .NET's minor version change.

[1r00t]

Just asking, but why do Python apps require that specific version of the runtime? I mean, it is not Python 4.0 or something. Just 3.10 and 3.11.3. I don't know Python well, but that looks like a minor change. And shouldn't 3.11.3 have backward compatibility with 3.9? I mostly use C#, and I don't think I had this kind of situation with .NET's minor version change.

Python 2 to 3 was a really big jump. Like partly different syntax and such. And these "minor" version bumps introduce a lot of changes as well. That's why Ubuntu for example still sticks to version 3.8 for quite some time.

It's the same for the Java runtime for example.

[vladmandic]

And shouldn't 3.11.3 have backward compatibility with 3.9?

it does for pure python code, not for any code that links to c libraries - so in reality it breaks a lot.
as a result package maintainers have to release a python 3.11 specific version of their libraries and until that is done for all required libraries, i cannot cleanly support 3.11. by now that has happened for 99% of libraries, but not 100%. if you feel experimental, you can enable usage of python 3.11 using --experimental flag, but don't complain about any issues then.

[afbagwell]

Has anyone tried using 3.11 yet with the support now available in pytorch 2.0.1?

pytorch/pytorch#86566 (comment)

[vladmandic]

it works for 99% of things, still not supported for some upscalers - they need to update their packages.
you can disable internal checks in app using --experimental flag and try your luck, just don't report issues :)

[Threnos]

All right, but what are the benefits of 3.11?

[HubKing]

For me, the only benefit is that I don't have to install Python 3.10 in AUR which takes tens of minutes to install/update because it must be compiled on my PC. Other yhan that who cares. I don't even like Python.

[afbagwell]

For me it's just keeping security fixes up to date. But yeah since 3.10.12 was only released as source, I don't want to spend time compiling it. It would also be nice if there's some performance improvements, but I'm not tracking anything on that front.

[Tillerz]

Benefits of 3.11:

Summary – Release highlights

Python 3.11 is between 10-60% faster than Python 3.10.
On average, we measured a 1.25x speedup on the standard
benchmark suite. See [Faster CPython]
(https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-faster-cpython) for details.

https://docs.python.org/3/whatsnew/3.11.html

Not sure tho how much python code is actually being executed where speed is relevant, as the rendering is all running in the GPU.

[feffy380]

rembg is available for 3.11 now. The extension just hasn't been updated in several months