Some things don't easily work in virtualenv and venv, so use them from the system python, without --system-site-packages.
Example using pygame
After installing pygame to the system python, install vext.pygame in your virtualenv:
(my-virtualenv) $ pip install vext.pygame
These can be installed using pip as long as corresponding package is installed in the system python
Vext packages get deprecated when the originating project supports installation from pip and virtualenv / venv...
let's try and deprecate them all so vext is no longer needed.
OpenCV now supports installation through pip and virtualenv, so vext.opencv is not needed, good work OpenCV Team !
See the section "Fixing packages is better" below.
Virtualenv / venv are really well supported in the web development community.
Graphics and audio and GUI packages often don't work with virtualenv / venv.
Vext lets you use access specific modules in the system python, using just pip and standard requirements.
Why not use --system-site-packages?
Pulling in everything gets things working quickly, but negates many of the advantages of virtualenv / venv.
For instance it can be easy to forget to add packages to requirements.txt when everything "works for me(TM)".
What about symlinks?
This is quite a clean solution, but doesn't work in Windows.
Vext becomes a gatekeeper to system modules, allowing access to particular libraries.
Vext works on the same OSs as virtualenv and venv.
Fixing packages is better
Vext is a workaround, it is always better to fix packages to work with virtualenv / venv if you can.
Packages that use C extensions often don't work, because their setup.py doesn't know about virtualenv / venv.
Pillow, Numpy and recent versions of OpenCV are notable packages that can be installed in virtualenv / venv with pip.
Using Vext Instead
To enable a vext package, create a .vext file.
As an example, here pygtk.vext
# Pygtk vext # # Vext will allow access to these modules modules: cairo gtk pygtk glib gobject test_import: # Vext will try and import any modules here # To check if the install worked gtk, pygtk install_hint: This message will be displayed if the system dependencies are not installed. # pygtk.pth does most of the initialisation of pygtk pths: pygtk.pth
A setup.py is needed to install the vext, see the repository for vext.pygtk.
Vext is cross platform, so please try and test your vext specs on windows as well as osx and linux !
Next, open a ticket on the Vext github.
Vext Spec File Format
List of modules to allow importing from the system site packages.
These are modules that should be able to be imported if everything is working correctly, you can test this using vext -c, for instance
$ vext -c pygtk pygtk.vext import gtk: [success] import pygtk: [success]
You can also use
vext -c '*' to check every vext files imports,
which can be useful to help diagnose problems.
Specify any pth files that the module uses to initialise itself.
Vext won't stop all imports, (especially when pth files are used) the idea is that an error will be thrown fairly early though if you are running Vext and try and import a module.
Mostly you will use vext just by installing a package like 'vext.pygtk' the commandline options allow you to get status and enable / disable Vext.
optional arguments: -l, --list List external packages "Vext" -e, --enable Disable Vext loader -d, --disable Enable Vext loader -s, --status Show Vext status -c CHECK, --check CHECK [external package] Test imports for external package
If Vext is causing an issue with other tools try disabling it,
also feel free to send a bug report with any information and
the output of
Vext works as a finder and loader of modules, it installs itself on the end of the system path, and then will check for any allowed modules.
If Vext runs any .pth files they will be installed after it in the .pth practically it guards the base level of sitepackages.
So in a virtualenv
import pygtk should definitally fail if vext is
installed but not vext.pygtk
VEXT_DEBUG_LOG=1 enables extra debugging.
VEXT_LOG_BLOCKS=1 store blocked imports in vext.blocked_imports.
VEXT_DISABLED=1 disable vext when set.
Vext acts as gatekeeper to system modules, to aid the process of creating a vext file it can log blocked imports or just save them to a local data structure.
Logging Blocked Imports
To see which modules Vext is blocking set VEXT_LOG_BLOCKS instead of standard ImportErrors, you will get ones like this:
>>> import pyaudio ImportError("Vext blocked import of pyaudio")
Remembering Blocked Imports
Vext can store all blocked imports in vext.blocked_imports by setting vext.remember_blocks to True:
>>> import vext >>> vext.remember_blocks = True >>> vext.blocked_imports set() >>> import pygtk ImportError: >>> vext.blocked_imports set(["pygtk"])
Please test this, so far it has only been tested on Ubuntu and Windows.
The 'test import' part of the .vext file is not implemented yet, this would stop installs unless the system requirements are available (or at least write a warning to the console).
Merging the finder and loader should make it possible to know if we are importing an allowed module, this would mean the submodules wouldn't need to be specified in the .vext file.
ruamel/venvgtk - for showing something like this is possible
pymotw article on modules and imports