Add testing for Nilearn setup & install & Fix the broken dependency i…

…nstallation (nilearn#2201)

* Added job in Azure Pipelines to build & install Nilearn wheel

* Fix inadvertant pip install with the intended pip list

* Wheel install command is Nilearn version agnostic to work with future versions [skip ci]

* Ensure all required packages are listed for `pip install`

* Eanble error mode in shell

* Fixed Could not find a version that satisfies the requirement sklearn>=0.19

(Replaced name sklearn with proper name scikit-learn.)

* Removed deactivated code

* Corrected typo in package name

* Restored sklearn as module name in conf.py to fix import failure

* Restored sklearn as module name in version.py to fix import failure

* scikit-learn is the package name passed on to pip

* Dummy sklearn added to required modules list

* Flake8 fix

* Tinkering with sklearn version for successful Appveyor install

* Added installation testing on Windows as well

 - Changed next version to 0.6.0b0 as setuptools warns & normalizes to that.
 - Tweaked Azure job names for clarity.

* Fixed spaces in Azure Pipelines job names

* Fixed non-alphanumeric characters in Azure Pipelines job names

* Removed code & package name redundancy

* Restored previous code, (sklearn not scikit-learn is essential in versions.py)

* Placate Fake8

* Added installation testing for MacOS

 - Renamed Azure Pipelines jobs for clarity.

* Added test for checking the required packages are installed

* `\` path separator for windows installation test; sorted jobs alphabetically

* Setting Error mode for Windows Powershell

* Corrected incorrect commands and path specs

* Corrected installation of Nilearn wheel in Powershell

* Replaced Powershell commands with cmd commands (the correct shell for AZP)

* Corrected command to execute pip install on discovered file

* Proactively installed sklearn to fix not found error when installing nilearn

* Installation tests are only done for Py3.6, seems unnecessary for more

* Expected failure tests (older python); windows install test during code test

* Added expected to fail tests for older python versions

* Segregated steps for easier human parsing

* Removed py34 (not supported by AZP), Py27 test failure now expected in test

appveyor.yml

@@ -27,7 +27,7 @@ install:
   - "conda install pip numpy scipy scikit-learn joblib nose pytest wheel matplotlib -y -q"
 
   # Install other nilearn dependencies
-  - "pip install nibabel coverage nose-timer pytest-cov"
+  - "pip install nibabel coverage nose-timer pytest-cov sklearn"
   - "python setup.py bdist_wheel"
   - ps: "ls dist"
 

azure-pipelines.yml

@@ -8,7 +8,8 @@ trigger:
 
 jobs:
 
-- job: 'TestWindows'
+
+- job: 'run_tests_on_windows'
   pool:
     vmImage: 'vs2017-win2016'
   strategy:
@@ -44,3 +45,149 @@ jobs:
       testResultsFiles: '**/test-results.xml'
       testRunTitle: 'Python $(python.version)'
     condition: succeededOrFailed()
+
+
+- job: 'test_installing_on_linux'
+  pool:
+    vmImage: 'ubuntu-18.04'
+  strategy:
+    matrix:
+      Python36:
+        python.version: '3.6'
+    maxParallel: 4
+
+  steps:
+  - task: UsePythonVersion@0
+    inputs:
+      versionSpec: '$(python.version)'
+      architecture: 'x64'
+
+  - script: |
+      set -e
+      python -m pip install --upgrade pip
+      pip install --prefer-binary setuptools wheel
+    displayName: 'Install dependencies for building Nilearn'
+
+  - script: |
+      set -e
+      python setup.py sdist bdist_wheel
+    displayName: 'Build Nilearn'
+
+  - script: |
+      set -e
+      pip install dist/nilearn-*-py3-none-any.whl
+      python -c "from nilearn.tests.test_package_installation import test_required_package_installation as test_pkg; test_pkg()"
+    displayName: 'Install Nilearn from wheel & check required package installation'
+
+  - script: |
+      set -e
+      python -c "from nilearn.tests.test_package_installation import test_required_package_installation as test_pkg; test_pkg()"
+    displayName: 'Test required package installation'
+
+
+- job: 'test_installing_on_macos'
+  pool:
+    vmImage: 'macOS-10.13'
+  strategy:
+    matrix:
+      Python36:
+        python.version: '3.6'
+    maxParallel: 4
+
+  steps:
+  - task: UsePythonVersion@0
+    inputs:
+      versionSpec: '$(python.version)'
+      architecture: 'x64'
+
+  - script: |
+      set -e
+      python -m pip install --upgrade pip
+      pip install --prefer-binary setuptools wheel
+    displayName: 'Install dependencies for building Nilearn'
+
+  - script: |
+      set -e
+      python setup.py sdist bdist_wheel
+    displayName: 'Build Nilearn'
+
+  - script: |
+      set -e
+      pip install dist/nilearn-*-py3-none-any.whl
+      python -c "from nilearn.tests.test_package_installation import test_required_package_installation as test_pkg; test_pkg()"
+    displayName: 'Install Nilearn from wheel & check required package installation'
+
+  - script: |
+      set -e
+      python -c "from nilearn.tests.test_package_installation import test_required_package_installation as test_pkg; test_pkg()"
+    displayName: 'Test required package installation'
+
+
+- job: 'test_installing_on_windows'
+  pool:
+    vmImage: 'vs2017-win2016'
+  strategy:
+    matrix:
+      Python36:
+        python.version: '3.6'
+    maxParallel: 4
+
+  steps:
+  - task: UsePythonVersion@0
+    inputs:
+      versionSpec: '$(python.version)'
+      architecture: 'x64'
+
+  - script: |
+      :; set -o errexit
+      python -m pip install --upgrade pip
+      pip install --prefer-binary setuptools wheel
+      python setup.py sdist bdist_wheel
+    displayName: 'Install dependencies for building Nilearn'
+
+  - script: |
+      :; set -o errexit
+      python setup.py sdist bdist_wheel
+    displayName: 'Build Nilearn'
+
+  - script: |
+      :; set -o errexit
+      forfiles /p dist /m nilearn-*-py3-none-any.whl /c "cmd /c pip install @file"
+    displayName: 'Install binary distribution'
+
+  - script: |
+      :; set -o errexit
+      python -c "from nilearn.tests.test_package_installation import test_required_package_installation as test_pkg; test_pkg()"
+    displayName: 'Test that required package are installed'
+
+
+- job: 'test_expeceted_to_fail_installation_on_linux'
+  pool:
+    vmImage: 'ubuntu-18.04'
+  strategy:
+    matrix:
+      Python27:
+        python.version: '2.7'
+    maxParallel: 4
+
+  steps:
+  - task: UsePythonVersion@0
+    inputs:
+      versionSpec: '$(python.version)'
+      architecture: 'x64'
+
+  - script: |
+      set -e
+      python -m pip install --upgrade pip
+      pip install --prefer-binary setuptools wheel
+    displayName: 'Install dependencies for building Nilearn'
+
+  - script: |
+      set -e
+      python setup.py sdist bdist_wheel
+    displayName: 'Build Nilearn'
+
+  - script: |
+      set -e
+      pip install dist/nilearn-*-py3-none-any.whl 2>&1 | grep "^ERROR"
+    displayName: 'Attempt to install Nilearn from wheel (expected to fail)'

continuous_integration/show-python-packages-versions.py

@@ -1,6 +1,7 @@
 import sys
 
-DEPENDENCIES = ['numpy', 'scipy', 'sklearn', 'joblib', 'matplotlib', 'nibabel']
+DEPENDENCIES = ['numpy', 'scipy', 'scikit-learn', 'joblib', 'matplotlib',
+                'nibabel']
 
 
 def print_package_version(package_name, indent='  '):

nilearn/tests/test_package_installation.py

@@ -0,0 +1,17 @@
+from distutils.version import LooseVersion
+
+from nilearn.version import REQUIRED_MODULE_METADATA
+
+
+def test_required_package_installation():
+    for package_specs in REQUIRED_MODULE_METADATA:
+        package = package_specs[0]
+        min_version = package_specs[1]['min_version']
+        imported_package = __import__(package)
+        installed_version = imported_package.__version__
+        assert LooseVersion(installed_version) >= LooseVersion(min_version)
+        print(package, 'min:', min_version, 'installed:', installed_version)
+
+
+if __name__ == '__main__':
+    test_required_package_installation()

nilearn/version.py

@@ -21,7 +21,7 @@
 # Dev branch marker is: 'X.Y.dev' or 'X.Y.devN' where N is an integer.
 # 'X.Y.dev0' is the canonical version of 'X.Y.dev'
 #
-__version__ = '0.6.0b'
+__version__ = '0.6.0b0'
 
 _NILEARN_INSTALL_MSG = 'See %s for installation information.' % (
     'http://nilearn.github.io/introduction.html#installation')

setup.py

@@ -30,6 +30,20 @@ def is_installing():
     return install_commands.intersection(set(sys.argv))
 
 
+def list_required_packages():
+    required_packages = []
+    required_packages_orig = ['%s>=%s' % (mod, meta['min_version'])
+                              for mod, meta
+                              in _VERSION_GLOBALS['REQUIRED_MODULE_METADATA']
+                              ]
+    for package in required_packages_orig:
+        if package.startswith('sklearn'):
+            package = package.replace('sklearn', 'scikit-learn')
+        required_packages.append(package)
+    required_packages.append('sklearn')
+    return required_packages
+
+
 # Make sources available using relative paths from this file's directory.
 os.chdir(os.path.dirname(os.path.abspath(__file__)))
 
@@ -51,11 +65,6 @@ def is_installing():
         module_check_fn = _VERSION_GLOBALS['_check_module_dependencies']
         module_check_fn(is_nilearn_installing=True)
 
-    install_requires = \
-        ['%s>=%s' % (mod, meta['min_version'])
-            for mod, meta in _VERSION_GLOBALS['REQUIRED_MODULE_METADATA']
-            if not meta['required_at_installation']]
-
     setup(name=DISTNAME,
           maintainer=MAINTAINER,
           maintainer_email=MAINTAINER_EMAIL,
@@ -96,6 +105,6 @@ def is_installing():
                         'nilearn.datasets.tests.data': ['*.*'],
                         'nilearn.datasets.description': ['*.rst'],
                         'nilearn.reporting.data.html': ['*.html']},
-          install_requires=install_requires,
+          install_requires=list_required_packages(),
           python_requires='>=3.5',
           )