Installer bloat

[quicklizard99]

The migration to Python 3 introduced some installer bloat. Investigate and fix.

[quicklizard99]

On Windows - tests directories are being distributed

./distutils/tests
./gouda/tests
./inselect/tests
./numpy/core/tests
./numpy/f2py/tests
./numpy/lib/tests
./pbr/tests
./pylibdmtx/tests
./pyzbar/tests
./scipy/_lib/tests
./scipy/cluster/tests
./scipy/constants/tests
./scipy/fftpack/tests
./scipy/integrate/tests
./scipy/interpolate/tests
./scipy/io/arff/tests
./scipy/io/harwell_boeing/tests
./scipy/io/matlab/tests
./scipy/io/tests
./scipy/linalg/tests
./scipy/misc/tests
./scipy/ndimage/tests
./scipy/odr/tests
./scipy/optimize/tests
./scipy/signal/tests
./scipy/sparse/csgraph/tests
./scipy/sparse/linalg/dsolve/tests
./scipy/sparse/linalg/eigen/arpack/tests
./scipy/sparse/linalg/eigen/lobpcg/tests
./scipy/sparse/linalg/isolve/tests
./scipy/sparse/linalg/tests
./scipy/sparse/tests
./scipy/spatial/tests
./scipy/special/tests
./scipy/stats/tests
./sklearn/datasets/tests

[quicklizard99]

On Windows - C and Fortran source

./numpy/f2py/
├── src
│   ├── fortranobject.c
│   └── fortranobject.h
└── tests
    └── src
        ├── array_from_pyobj
        │   └── wrapmodule.c
        ├── assumed_shape
        │   ├── foo_free.f90
        │   ├── foo_mod.f90
        │   ├── foo_use.f90
        │   └── precision.f90
        ├── kind
        │   └── foo.f90
        ├── mixed
        │   ├── foo.f
        │   ├── foo_fixed.f90
        │   └── foo_free.f90
        ├── regression
        │   └── inout.f90
        └── size
            └── foo.f90

[quicklizard99]

On Windows distutils installer stubs

distutils/
├── command
│   ├── wininst-10.0-amd64.exe
│   ├── wininst-10.0.exe
│   ├── wininst-14.0-amd64.exe
│   ├── wininst-14.0.exe
│   ├── wininst-6.0.exe
│   ├── wininst-7.1.exe
│   ├── wininst-8.0.exe
│   ├── wininst-9.0-amd64.exe
│   └── wininst-9.0.exe

[quicklizard99]

On Windows - lots of Qt stuff that we aren't using

PyQt5/Qt/bin/Qt5Bluetooth.dll
PyQt5/Qt/bin/Qt5CLucene.dll
PyQt5/Qt/bin/Qt5Core.dll
PyQt5/Qt/bin/Qt5DBus.dll
PyQt5/Qt/bin/Qt5Designer.dll
PyQt5/Qt/bin/Qt5Gui.dll
PyQt5/Qt/bin/Qt5Help.dll
PyQt5/Qt/bin/Qt5Location.dll
PyQt5/Qt/bin/Qt5Multimedia.dll
PyQt5/Qt/bin/Qt5MultimediaWidgets.dll
PyQt5/Qt/bin/Qt5Network.dll
PyQt5/Qt/bin/Qt5Nfc.dll
PyQt5/Qt/bin/Qt5OpenGL.dll
PyQt5/Qt/bin/Qt5Positioning.dll
PyQt5/Qt/bin/Qt5PrintSupport.dll
PyQt5/Qt/bin/Qt5Qml.dll
PyQt5/Qt/bin/Qt5Quick.dll
PyQt5/Qt/bin/Qt5QuickWidgets.dll
PyQt5/Qt/bin/Qt5Sensors.dll
PyQt5/Qt/bin/Qt5SerialPort.dll
PyQt5/Qt/bin/Qt5Sql.dll
PyQt5/Qt/bin/Qt5Svg.dll
PyQt5/Qt/bin/Qt5Test.dll
PyQt5/Qt/bin/Qt5WebChannel.dll
PyQt5/Qt/bin/Qt5WebEngineCore.dll
PyQt5/Qt/bin/Qt5WebEngineWidgets.dll
PyQt5/Qt/bin/Qt5WebSockets.dll
PyQt5/Qt/bin/Qt5Widgets.dll
PyQt5/Qt/bin/Qt5WinExtras.dll
PyQt5/Qt/bin/Qt5Xml.dll
PyQt5/Qt/bin/Qt5XmlPatterns.dll
PyQt5/Qt/bin/QtWebEngineProcess.exe

[quicklizard99]

On Windows - include neither PyQt5\Qt\qml nor PyQt5\Qt\translations. Exclude all of PyQt5\Qt\plugins except (I think)platforms

[quicklizard99]

On Windows - cv2.cp35-win_amd64.pyd depends (idiotically) on every OpenCV DLL, totalling more than 55 MB. It should use delay-loaded DLL mechanism to load dependent DLLs as they are required. This might be a defect with the build of OpenCV that we are using (Anaconda channel menpo::opencv3) or might be a defect OpenCV itself.

[quicklizard99]

On Mac - PyInstaller builds the app and each command-line tool into their own package, leading to duplicated code. See #292

[quicklizard99]

On Windows - reduced by > 130MB by including only the PyQt and Qt binaries that Inselect uses.

[quicklizard99]

Main culprits on Windows are OpenCV (55 MB) and SciPy (34 MB).

[quicklizard99]

Also on Windows, the Intel mkl libraries take up 44 MB. There is not much we can do about this because Continuum build numpy to depend on mkl and we are tied to Continuum's Anaconda in order to get scipy.

https://www.continuum.io/blog/developer-blog/anaconda-25-release-now-mkl-optimizations
https://docs.continuum.io/mkl-optimizations/

[quicklizard99]

Some dependencies that might make sense in a virtualenv but not in a frozen program.

• pydoc is a dependency of sklearn
• unittest is a dependency of numpy.core.multiarray
• Several (all?) of scipy __init__.py files import numpy.testing (and run tests when imported)