Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux Fixes #384

Merged
merged 11 commits into from
Nov 11, 2022
Merged

Linux Fixes #384

merged 11 commits into from
Nov 11, 2022

Conversation

NullSenseStudio
Copy link
Collaborator

Main included fix is for SSLCertVerificationError while importing kornia. Made it so the base ssl module can use certifi's cacert.pem file. The Blender 3.3.1 version I used has a broken ssl module that points to certs which do not exist, and I doubt this is the only version with the same problem.

>>> import ssl
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(
	cafile=None,
	capath=None,
	openssl_cafile_env='SSL_CERT_FILE',
	openssl_cafile='/home/sybren/buildbot-builder/linux_glibc217_x86_64_cmake/build_deps/deps/Release/ssl/cert.pem',
	openssl_capath_env='SSL_CERT_DIR',
	openssl_capath='/home/sybren/buildbot-builder/linux_glibc217_x86_64_cmake/build_deps/deps/Release/ssl/certs'
)
>>> import os
>>> import certifi
>>> os.environ['SSL_CERT_FILE'] = certifi.where()
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(
	cafile='/home/nullsense/blender/blender-3.3.1/3.3/python/lib/python3.10/site-packages/certifi/cacert.pem',
	capath=None,
	openssl_cafile_env='SSL_CERT_FILE',
	openssl_cafile='/home/sybren/buildbot-builder/linux_glibc217_x86_64_cmake/build_deps/deps/Release/ssl/cert.pem',
	openssl_capath_env='SSL_CERT_DIR',
	openssl_capath='/home/sybren/buildbot-builder/linux_glibc217_x86_64_cmake/build_deps/deps/Release/ssl/certs'
)

After the change urllib won't fail on every https request and correctly downloads checkpoint_liberty_with_aug.pth. If there ends up being more issues with SSL verification it's likely fixable by updating the certifi module via pip.

Had an issue where for some reason pip was installed into the user site packages so ensurepip would not install it to Blender's site-packages directory. Made it so that should no longer accidentally happen.

Removed model preloading from the install dependencies operator since that's taken care of in prompt_to_image.py and just errors here.

Made it so the subprocess's stdout isn't discarded and is instead the same as stderr so it's still visible in the terminal. The model loading step will just print its error information so it was getting completely lost and leading to harder to debug issues.

This PR is not ready to merge yet since I'd like to spend more time trying to find more bugs to fix.

@carson-katri carson-katri added the bug Something isn't working label Nov 5, 2022
@carson-katri carson-katri added this to the v0.0.9 milestone Nov 5, 2022
@NullSenseStudio
Copy link
Collaborator Author

Forgot to mention I had also removed the urlopen decorator since it did not seem to help with the SSL issue I encountered, and I'd rather not have a potentially unsafe implementation like that.

I've now changed it so the certifi cert is always used on the backend rather than just during the model preloading phase in case if there are ever other methods using the standard SSL module.

The install dependencies operator will no longer attempt to preload models since that is handled on the backend and will always just error here.

Fixed a regression that made it so the missing dependencies warning wouldn't appear.

Snap installs programs under a read-only filesystem so now ensurepip can install to the user site when bootstrapping normally fails, and skips copying header files into the include directory.

@carson-katri
Copy link
Owner

Do you know if there’s a bug filed for Blender about the OpenSSL paths?

@NullSenseStudio
Copy link
Collaborator Author

I don't know. I'll try to look.

@NullSenseStudio
Copy link
Collaborator Author

Can't find anything open or recent. I'll see about testing a beta build and file a report if it's still present.

@NullSenseStudio
Copy link
Collaborator Author

Made it so Install Dependencies can now install and use pip in .python_dependencies, blender site-packages, and user site-packages (attempts in that order). I didn't like the user site-packages workaround since that takes precedence in sys.path and may interfere with a separate python interpreter running its pip module, but I'm keeping it in case if both alternatives fail.

@NullSenseStudio
Copy link
Collaborator Author

I believe it's ready. I've tested on Windows to make sure it didn't accidentally cause issues there and it worked like normal.

Copy link
Owner

@carson-katri carson-katri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@carson-katri carson-katri merged commit 12e45c5 into carson-katri:main Nov 11, 2022
@NullSenseStudio NullSenseStudio deleted the linux-fixes branch November 11, 2022 21:31
JasonHoku pushed a commit to JasonHoku/dream-textures that referenced this pull request Dec 20, 2022
* isolate install_pip() from user site

* kornia SSLCertVerificationError mitigation

* simplify kornia SSLCertVerificationError mitigation

requests uses certifi.where() and setting SSL_CERT_FILE will allow urllib to use the same certs

* redirect subprocess's stdout to stderr

* remove model preloading from install dependencies operator

* use certifi for whole backend

* fix dependencies missing warning

* fix installing when certain directories are read only

* install pip to .python_dependencies

* add check argument

* fix default argument
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants