Some dependencies are not needed by all users

[mdmintz]

Some dependencies are not needed by all users

SeleniumBase has a lot of dependencies. Not all of these dependencies are needed by all users. Perhaps something can be done about this?

I was recently reading about https://stackoverflow.com/questions/12332975/installing-python-module-within-code, which may help solve this problem. According to the solution, https://stackoverflow.com/a/50255019/7058266, a dependency could be installed as needed at runtime:

import subprocess
import sys

def install(package):
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])

Using this system for some optional dependencies (rather than installing all dependencies every time) will speed up the main install.

An option could also be created to optionally install these dependencies at the start by using extras_require in the setup.py file. Here's an example from the existing extras_require in SeleniumBase:

    extras_require={
        # pip install -e .[flake]
        # Usage: flake8
        "flake": [
            'flake8==3.7.9;python_version<"3.6"',
            'flake8==4.0.1;python_version>="3.6"',
            "mccabe==0.6.1",
            'pyflakes==2.1.1;python_version<"3.6"',
            'pyflakes==2.4.0;python_version>="3.6"',
            'pycodestyle==2.5.0;python_version<"3.6"',
            'pycodestyle==2.8.0;python_version>="3.6"',
        ],
    },

Here are some candidates for the optional-install system:

pdfminer.six
Pillow
boto

Details on each one:

pdfminer.six - This library is used to read from PDF files. SeleniumBase has build-in methods for reading and verifying text from PDF files.

Pillow - This library is used for advanced imaging abilities. SeleniumBase has a method that uses this library to add an image overlay to an existing image.

boto - This library is used for AWS/S3 interaction. SeleniumBase has an old feature that lets you upload logs to an S3 bucket of your choice if you provide connection details in settings.py. There's a newer boto3, but it's not a drop-in replacement, so I left this feature alone as is. These days, I'm doing more work with Azure, so I haven't had a recent reason to revisit old cloud functionality.

There might be more dependencies that won't be needed by all users, but this should provide a good starting point.

[mdmintz]

For the case of runtime installs, will need to use thread-locking because tests could be running multi-threaded with multiple processes using pytest-xdist, in which case, if there are multiple tests that all-of-a-sudden need an optional dependency installed, then they all might try to install the same dependency multiple times at the same time.

One possible drawback to waiting until runtime to install optional dependencies is that people might be running in offline mode (but have previously gone online before running tests to install dependencies), and so they'll have problems if they need to install a dependency from PyPI when offline. It's a rare case, but should be easily handled with minimal impact if the situation comes up for people.