Increased M2Crypto to 0.30.1; Used M2Crypto on Windows

M2Crypto 0.29.0 introduced support for Windows again, so it is no longer necessary to use the very outdated M2CryptoWin32/64 packages. This change uses the M2Crypto package now also on Windows, and eliminates using M2CryptoWin32/64. To enable that, it increases the minimum version of M2Crypto to 0.30.1 (which has a fix for the Windows support introduced with 0.29.0). It also adds a pywbem_os_setup.bat script which installs OS-level dependencie for installing M2Crypto (e.g. its build requires swig, Win32/64OpenSSL). Addressed review comments * Updated README.rst file to scope shown installation approach to Linux. * Typo in appveyor.yml file. Addressed review comments (2) - Added prerequisites such as initial python packages, etc. to the installation sections of each platform. - The pywbem_os_setup.bat file got extended to perform install as a default when no argument is specified, consistent with the .-sh version. Signed-off-by: Andreas Maier <andreas.r.maier@gmx.de>
pywbem · Jun 7, 2018 · 8dd40fb · 8dd40fb
1 parent 0a78b43
commit 8dd40fb
Show file tree

Hide file tree

Showing 14 changed files with 339 additions and 217 deletions.
diff --git a/README.rst b/README.rst
@@ -100,7 +100,7 @@ activities:
 Installation
 ------------
 
-To install the latest released version of pywbem:
+To install the latest released version of pywbem on Linux:
 
 * Install the prerequisite OS-level packages:
 
@@ -120,7 +120,8 @@ To install the latest released version of pywbem:
 
   This will also install any prerequisite Python packages.
 
-For more details and alternative ways to install, see the
+For more details, installation on other operating systems, and alternative
+ways to install, see the
 `Installation section`_ in the pywbem documentation.
 
 .. _Installation section: http://pywbem.readthedocs.io/en/stable/intro.html#installation

diff --git a/appveyor.yml b/appveyor.yml
@@ -2,29 +2,15 @@
 environment:
   matrix:
 #------------------------
-# Notes on disabled environments:
-# * Python 2.6 is disabled because on Windows, pywbem uses M2Crypto via the
-#   M2CryptoWin32 package. That package is not supported for Python 2.6, and
-#   its __m2crypto.pyd file depends on python27.dll. As a result, importing
-#   M2Crypto on Python 2.6 fails with:
-#     ImportError: Module use of python27.dll conflicts with this version of
-#     Python.
-#   To support Python 2.6, M2Crypto would need to be built upon install from
-#   the M2Crypto package. See https://github.com/dsoprea/M2CryptoWindows for
-#   details on how to build it.
+# Python 2.6 is not supported by M2Crypto on Windows (nor by the earlier used
+# M2CryptoWin32/Win64; its __m2crypto.pyd file depends on python27.dll).
 #------------------------
-#    - PYTHON_VERSION: 2.6
-#      PYTHON_ARCH: 32
-#      PYTHON_HOME: C:\Python26
-#    - PYTHON_VERSION: 2.6
-#      PYTHON_ARCH: 64
-#      PYTHON_HOME: C:\Python26-x64
     - PYTHON_VERSION: 2.7
       PYTHON_ARCH: 32
       PYTHON_HOME: C:\Python27
-#    - PYTHON_VERSION: 2.7
-#      PYTHON_ARCH: 64
-#      PYTHON_HOME: C:\Python27-x64
+    - PYTHON_VERSION: 2.7
+      PYTHON_ARCH: 64
+      PYTHON_HOME: C:\Python27-x64
 #    - PYTHON_VERSION: 3.4
 #      PYTHON_ARCH: 32
 #      PYTHON_HOME: C:\Python34
@@ -51,6 +37,13 @@ configuration:
 
 install:
 
+  # Examine Windows version and current directory
+  - ver
+  - cd
+
+  # Examine the initial environment
+  - set
+
   - git --version
   - if %APPVEYOR_REPO_BRANCH%.==manual-ci-run. set _NEED_REBASE=true
   # This Git version requires user configuration in rebase step
@@ -64,25 +57,9 @@ install:
   - if %APPVEYOR_REPO_BRANCH%.==manual-ci-run. set _MANUAL_CI_RUN=true
   - if %APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH%.==manual-ci-run. set _MANUAL_CI_RUN=true
 
-  # Verify that the commands used in this file are available
-  - where where
-  - where choco
-  - choco --version
-
   # Set PACKAGE_LEVEL for make
   - set PACKAGE_LEVEL=%configuration%
 
-  # Examine the initial environment
-  - ver
-  - 'echo "%PATH%"'
-  - 'echo "%PYTHONPATH%"'
-  - 'echo "%INCLUDE%"'
-  - 'echo "%LIB%"'
-  - choco source list
-  - dir C:\
-  - dir
-  - set
-
   # Remove Python 2.7 from PATH.
   # Note that YAML interprets some characters in a special way (including '!' and '#')
   # so we have to use single quotes to protect some CMD commands from YAML.
@@ -99,20 +76,16 @@ install:
   - 'set PATH=%$newpath%'
 
   # Add CygWin
+  # This must be after removal of Python 2.7 (because CygWin has the Unix find) and
+  # before adding Python (because CygWin also has a python).
   - set PATH=C:\cygwin\bin;%PATH%
 
-  # Verify that the commands provided by CygWin and used in this file are available
-  - where wget
-  - wget --version
-  - where unzip
-  - unzip --help
-  - where 7z
-  - 7z --help
-
   # Add Python
   - reg ADD HKCU\Software\Python\PythonCore\%PYTHON_VERSION%\InstallPath /ve /d "%PYTHON_HOME%" /t REG_SZ /f
   - reg ADD HKLM\Software\Python\PythonCore\%PYTHON_VERSION%\InstallPath /ve /d "%PYTHON_HOME%" /t REG_SZ /f
   - set PATH=%PYTHON_HOME%;%PYTHON_HOME%\Scripts;%PATH%
+  - where python
+  - python --version
 
   ## Install InnoSetup - disabled because it is not needed
   #- choco install -y InnoSetup
@@ -121,29 +94,41 @@ install:
   ## Install pip - disabled because pip is already installed
   #- ps: (new-object System.Net.WebClient).Downloadfile('https://bootstrap.pypa.io/get-pip.py', 'C:\Users\appveyor\get-pip.py')
   #- ps: Start-Process -FilePath "C:\Python27\python.exe" -ArgumentList "C:\Users\appveyor\get-pip.py" -Wait -Passthru
+  - where pip
+  - pip --version
+
+  # Verify that Chocolatey is available
+  - where choco
+  - choco --version
+
+  # Circumvention:
+  # The installation in pywbem_os_setup.bat installs swig but without generating
+  # the swig.exe shim file (using GenShim).
+  # See GenShim issue https://github.com/chocolatey/shimgen/issues/43
+  # Therefore, we install swig here in order to create the swig.exe shim file.
+  # TODO: Remove the circumvention once it works in pywbem_os_setup.bat.
+  - choco install -r -y swig
+  - where swig
 
   # Examine the final environment
   - 'echo "%PATH%"'
   - 'echo "%PYTHONPATH%"'
   - 'echo "%INCLUDE%"'
   - 'echo "%LIB%"'
+  - 'echo "%SWIG_FEATURES%"'
 
   # Install tox
-  - where pip
-  - pip --version
   - pip install tox==2.0.0
-
-  # Verify that the commands used in tox.ini are available
   - where tox
   - tox --version
+
+  # Verify that the commands used in tox.ini are available
+  # Already verified further up: tox, pip, python.
   - where make
   - make --version
-  - where pip
-  - pip --version
-  - where python
-  - python --version
 
   # Verify that the commands used in makefile are available
+  # Already verified further up: pip, python.
   - where sh
   - sh --version
   - where bash
@@ -163,6 +148,10 @@ install:
   - where find
   - find --version
 
+  # Verify that the commands used in pywbem_os_setup.bat are available
+  - where curl
+  - curl --version
+
   # Verify that the commands used in the pywbem testsuite are available
   - where xmllint
   - xmllint --version

diff --git a/docs/appendix.rst b/docs/appendix.rst
@@ -407,6 +407,27 @@ name). On RHEL, the missing package is `python-dev`.
 For more details, see
 :ref:`Prerequisite operating system packages for development`.
 
+Installation fails with "invalid command 'bdist_wheel'"
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The installation of M2Crypto and probably other Python packages requires the
+Python "wheel" package. If that package is not installed in the current Python
+environment, the installation will fail with the following (or similar)
+symptom::
+
+    Creating library build\temp.win-amd64-2.7\Release\SWIG_m2crypto.lib and object build\temp.win- amd64-2.7\Release\SWIG_m2crypto.exp
+    python setup.py bdist_wheel
+    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
+    or: setup.py --help [cmd1 cmd2 ...]
+    or: setup.py --help-commands
+    or: setup.py cmd --help
+    error: invalid command 'bdist_wheel'
+
+To fix this, install the Python "wheel" package::
+
+    pip install wheel
+
+
 .. _'Glossary`:
 
 Glossary

diff --git a/docs/changes.rst b/docs/changes.rst
@@ -135,6 +135,12 @@ Released: not yet
   parameter and a mof file and pythong to testsuite to test the new option.
   (Issue #1268)
 
+* Installation on Windows is now more automated by means of a new
+  `pywbem_os_setup.bat` script. As part of that, the latest `M2Crypto` version
+  0.30.1 is now used on Windows, and no longer the somewhat aged versions in
+  the `M2CryptoWin32/64` packages. For details, see the installation section
+  in the documentation.
+
 **Cleanup**
 
 * Moved class `NocaseDict` into its own module (Issue #848).

diff --git a/docs/intro.rst b/docs/intro.rst
@@ -108,14 +108,25 @@ Pywbem is supported in these environments:
 Limitations:
 
 * On Windows (native), pywbem is not supported on Python 2.6, because the
-  M2CryptoWin32/64 packages do not support Python 2.6.
+  M2Crypto package does not support Python 2.6.
 
 
 .. _`Installing to Linux`:
 
 Installing to Linux
 ^^^^^^^^^^^^^^^^^^^
 
+* Prerequisites:
+
+  - The Python environment into which you want to install must be the current
+    Python environment, and must have at least the following Python packages
+    installed:
+
+    - setuptools
+    - wheel (<0.30.0 on Python 2.6)
+    - pip
+    - pbr
+
 * Download the ``pywbem_os_setup.sh`` script from one of these sources:
 
   - :download:`pywbem_os_setup.sh <../pywbem_os_setup.sh>` on this site
@@ -184,6 +195,17 @@ Installing to Linux
 Installing to OS-X
 ^^^^^^^^^^^^^^^^^^
 
+* Prerequisites:
+
+  - The Python environment into which you want to install must be the current
+    Python environment, and must have at least the following Python packages
+    installed:
+
+    - setuptools
+    - wheel (<0.30.0 on Python 2.6)
+    - pip
+    - pbr
+
 * Download the ``pywbem_os_setup.sh`` script from one of these sources:
 
   - :download:`pywbem_os_setup.sh <../pywbem_os_setup.sh>` on this site
@@ -205,7 +227,7 @@ Installing to OS-X
 * Only on Python 2 on newer OS-X releases:
 
   The ``pywbem_os_setup.sh`` script installs the ``openssl`` package needed by
-  the M2Crypto Python package. On newer OS-X releases, you may see a notice
+  the ``M2Crypto`` Python package. On newer OS-X releases, you may see a notice
   that the ``openssl`` package is "not linked" because the TLS implementation
   provided with OS-X is available. In that case, you need to set up the
   following environment variables for use by the pywbem package installation
@@ -235,31 +257,41 @@ This section describes the installation of pywbem into a native Windows
 environment (i.e. without using a UNIX-like environment; for that, see
 :ref:`Installing to a UNIX-like environment under Windows`):
 
-* Determine the bit size of the active Python environment, for example as
-  follows:
+* Prerequisites:
 
-  .. code-block:: bash
+  - The Python environment into which you want to install must be the current
+    Python environment, and must have at least the following Python packages
+    installed:
 
-      > python -c "import ctypes; print(ctypes.sizeof(ctypes.c_void_p)*8)"
-      64
+    - setuptools
+    - wheel
+    - pip
+    - pbr
 
-* Install the Windows build of M2Crypto into the active Python environment:
+  - The `Chocolatey package manager`_ must be installed and its "choco" command
+    available in the PATH.
 
-  For a 32-bit Python environment:
+  - Windows command prompt in administrator mode.
 
-  .. code-block:: bash
+* Download the ``pywbem_os_setup.bat`` script from one of these sources:
+
+  - :download:`pywbem_os_setup.bat <../pywbem_os_setup.bat>` on this site
+
+  - `pywbem_os_setup.bat <https://raw.githubusercontent.com/pywbem/pywbem/master/pywbem_os_setup.bat>`_
+    on the `master` branch of the pywbem Git repository
 
-      > pip install M2CryptoWin32
+  That script installs OS-level prerequisite packages needed by pywbem.
 
-  For a 64-bit Python environment:
+* Execute the ``pywbem_os_setup.bat`` script in an administrator command prompt:
 
   .. code-block:: bash
 
-      > pip install M2CryptoWin64
+      > pywbem_os_setup.bat
+
+  The script uses the `Chocolatey package manager`_.
 
-  Note that these Python packages are binary builds and therefore do not invoke
-  Swig at their installation time. Therefore, there are no prerequisite
-  OS-level packages to install.
+  This script will also install the ``M2Crypto`` Python package into the active
+  Python environment.
 
 * Install pywbem (and its other prerequisite Python packages) into the active
   Python environment:
@@ -268,6 +300,8 @@ environment (i.e. without using a UNIX-like environment; for that, see
 
       > pip install pywbem
 
+.. _`Chocolatey package manager`: https://chocolatey.org/
+
 
 .. _`Installing to a UNIX-like environment under Windows`:
 
@@ -281,19 +315,37 @@ Note that Unix-like environments on Windows bring their own Python, so in such
 an environment, you install into that Python, and not into the Python of
 Windows.
 
+* Prerequisites:
+
+  - The Python environment into which you want to install must be the current
+    Python environment, and must have at least the following Python packages
+    installed:
+
+    - setuptools
+    - wheel (<0.30.0 on Python 2.6)
+    - pip
+    - pbr
+
+  - Prerequisite OS-level packages must be available in the UNIX-like
+    environment.
+
+    For CygWin, these packages can be installed using the CygWin setup
+    program and are listed in
+    :ref:`Prerequisite operating system packages for install`.
+
+    For other UNIX-like environments, we did not compile a list of the required
+    packages. If you can help out here by providing the package names, please
+    tell us by `opening an issue <https://github.com/pywbem/pywbem/issues>`_).
+
 * Install pywbem (and its other prerequisite Python packages) into the active
   Python environment:
 
   .. code-block:: bash
 
       > pip install pywbem
 
-  If the Swig compilation during installation of M2Crypto fails, there may be
-  components of the UNIX-like environment missing. When using CygWin, these
-  may be installed using the CygWin setup program, and the packages are listed
-  in :ref:`Prerequisite operating system packages for install`. For other
-  UNIX-like environments, there should be similar packages (please tell us by
-  `opening an issue <https://github.com/pywbem/pywbem/issues>`_).
+  If the Swig compilation during installation of ``M2Crypto`` fails, there may
+  be components of the UNIX-like environment missing (see first item).
 
 
 .. _`Installing into a different Python environment`: