Added support for pywbem using requests

This is done in such a way that pywbemcli can work with pysbem versions before and after the pywbem change (i.e. pywbem 1.0.0). There is no need to increase the minimum required pywbem version to 1.0.0. Details: * Changed the `--ca-certs` general option to support the changes as of pywbem version 1.0.0 (new values 'system' and 'certifi', and default changed from a fixed set of directories to 'certifi'). The pywbem version is determined at run time and pywbem versions before 1.0.0 are still supported. * Download and invoke pywbem_os_setup.* script for installing pywbemtools only when using pywbem before 1.0.0 on Python 2. Note that the script is still downloaded and invoked for development of pywbemtools because pywbemtools needs some of the same packages pywbem needs for development. * Adjusted a test case in test_general_options.py to accomodate the different exception message when using pywbem version 1.0.0. * Adjusted the Makefile to the change in the downloaded pywbem_os_setup.* script by removing it in target clobber. This ensures that there is a way to get out of errors resulting from using a previously downloaded script that does not fit to the pywbem version that is used. * Adjusted the Makefile to specify the name for the downloaded pywbem_os_setup.* script. This avoids that wget creates numbered versions of the file if it already exists. Signed-off-by: Andreas Maier <andreas.r.maier@gmx.de>
pywbem · Dec 15, 2019 · 88b2b4c · 88b2b4c
1 parent e03c964
commit 88b2b4c
Show file tree

Hide file tree

Showing 9 changed files with 99 additions and 49 deletions.
diff --git a/Makefile b/Makefile
@@ -375,11 +375,11 @@ install_basic_$(pymn).done: Makefile pip_upgrade_$(pymn).done
 # Scripts are required to install the OS-level components of pywbem.
 # Makefile gets the required scripts from the pywbem project on GitHub.
 pywbem_os_setup.sh:
-	wget -q https://raw.githubusercontent.com/pywbem/pywbem/master/pywbem_os_setup.sh
+	wget -q -O pywbem_os_setup.sh https://raw.githubusercontent.com/pywbem/pywbem/master/pywbem_os_setup.sh
 	chmod 755 pywbem_os_setup.sh
 
 pywbem_os_setup.bat:
-	wget -q https://raw.githubusercontent.com/pywbem/pywbem/master/pywbem_os_setup.bat
+	wget -q -O pywbem_os_setup.bat https://raw.githubusercontent.com/pywbem/pywbem/master/pywbem_os_setup.bat
 
 .PHONY: install_os
 install_os: install_os_$(pymn).done
@@ -479,7 +479,7 @@ all: install develop build builddoc check pylint test
 .PHONY: clobber
 clobber: clean
 	@echo "makefile: Removing everything for a fresh start"
-	-$(call RM_FUNC,*.done epydoc.log $(dist_files) $(pywbemcli_module_path)/*cover)
+	-$(call RM_FUNC,*.done epydoc.log $(dist_files) $(pywbemcli_module_path)/*cover pywbem_os_setup.*)
 	-$(call RMDIR_FUNC,$(doc_build_dir) .tox $(coverage_html_dir))
 	@echo "makefile: Done removing everything for a fresh start"
 	@echo "makefile: Target $@ done."

diff --git a/README.rst b/README.rst
@@ -1,10 +1,6 @@
 pywbemtools: Python tools for communicating with WBEM servers
 =============================================================
 
-.. |os-setup-link| raw:: html
-
-    <a href="https://pywbemtools.readthedocs.io/en/stable/_downloads/pywbem_os_setup.sh">pywbem_os_setup.sh</a>
-
 .. image:: https://img.shields.io/pypi/v/pywbemtools.svg
     :target: https://pypi.python.org/pypi/pywbemtools/
     :alt: Version on Pypi
@@ -84,7 +80,8 @@ Requirements:
 2. Operating Systems: Linux, OS-X, native Windows, UNIX-like environments on
    Windows (e.g. Cygwin)
 
-3. When using Python 2, the following OS-level packages:
+3. When using a pywbem version before 1.0.0 on Python 2, the following
+   OS-level packages are needed:
 
    * On native Windows:
 
@@ -99,10 +96,10 @@ Requirements:
      - ``wget`` - Download tool. Can be installed using the OS-level package
        manager for the platform.
 
-
 Installation:
 
-* When using Python 2, install OS-level packages needed by the pywbem package:
+* When using a pywbem version before 1.0.0 on Python 2, install OS-level
+  packages needed by the pywbem package:
 
   - On native Windows:
 

diff --git a/README_PYPI.rst b/README_PYPI.rst
@@ -52,7 +52,8 @@ Requirements:
 2. Operating Systems: Linux, OS-X, native Windows, UNIX-like environments on
    Windows (e.g. Cygwin)
 
-3. When using Python 2, the following OS-level packages:
+3. When using a pywbem version before 1.0.0 on Python 2, the following
+   OS-level packages are needed:
 
    * On native Windows:
 
@@ -69,7 +70,8 @@ Requirements:
 
 Installation:
 
-* When using Python 2, install OS-level packages needed by the pywbem package:
+* When using a pywbem version before 1.0.0 on Python 2, install OS-level
+  packages needed by the pywbem package:
 
   - On native Windows:
 

diff --git a/docs/changes.rst b/docs/changes.rst
@@ -67,6 +67,11 @@ Released: not yet
   '--association' returns classes that are associations and '--no-association'
   returns only classes that are not associations.  See issue # 447
 
+* Changed the `--ca-certs` general option to support the changes as of
+  pywbem version 1.0.0 (new values 'system' and 'certifi', and default changed
+  from a fixed set of directories to 'certifi'). The pywbem version is
+  determined at run time and pywbem versions before 1.0.0 are still supported.
+
 **Cleanup:**
 
 * Test: Enabled Python warning suppression for PendingDeprecationWarning

diff --git a/docs/introduction.rst b/docs/introduction.rst
@@ -98,8 +98,6 @@ prerequisites and its setup is described in the :ref:`Pywbemtools development` c
 Installation prerequisites
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. _pywbem installation documentation: https://pywbem.readthedocs.io/en/stable/intro.html#installation
-
 The Python environment into which you want to install must have the following
 Python packages installed:
 
@@ -108,14 +106,11 @@ Python packages installed:
 - pip - generally installed with Python 3.x but may be a separate install
   with Python 2.7 and with Cygwin Python releases.
 
-Pywbemtools installs the pywbem package which requires a number of OS-level
-packages as documented in the `pywbem installation documentation`_.
-The pywbem package provides the ``pywbem_os_setup.sh/.bat`` scripts that
-install the needed OS-level packages. These scripts are only needed when
-using Python 2.
+Pywbemtools installs the pywbem package.
 
-To use these scripts, your system must have the following commands installed
-when using Python 2:
+When using pywbem versions before 1.0.0 on Python 2, pywbem requires a number
+of OS-level packages, and your system must have the following commands
+installed:
 
 * On native Windows:
 
@@ -130,13 +125,13 @@ when using Python 2:
   - ``wget`` - Download tool. Can be installed using the OS-level package
     manager for the platform.
 
-
 .. _`Installation with pip`:
 
 Installation with pip
 ^^^^^^^^^^^^^^^^^^^^^
 
-When using Python 2, install OS-level packages needed by the pywbem package:
+When using pywbem versions before 1.0.0 on Python 2, install the OS-level
+packages needed by the pywbem package as follows:
 
 * On native Windows:
 

diff --git a/docs/pywbemcli/cmdshelp.rst b/docs/pywbemcli/cmdshelp.rst
@@ -81,13 +81,16 @@ Help text for ``pywbemcli``:
                                       verify client bypasses verification.
                                       Default: EnvVar PYWBEMCLI_VERIFY, or "--
                                       verify".
-      --ca-certs FILE                 Path name of a file or directory containing
-                                      certificates that will be matched against
-                                      the server certificate presented by the WBEM
-                                      server during TLS/SSL handshake. Default:
-                                      EnvVar PYWBEMCLI_CA_CERTS, or [/etc/pki/ca-
+      --ca-certs CACERTS              CA certificates to be used for verifying the
+                                      server certificate presented by the WBEM
+                                      server during TLS/SSL handshake: FILE: use
+                                      the certs in the specified cert file; DIR:
+                                      use the certs in the specified cert
+                                      directory. Default: EnvVar
+                                      PYWBEMCLI_CA_CERTS, or the first existing
+                                      directory in: /etc/pki/ca-
                                       trust/extracted/openssl/ca-bundle.trust.crt,
-                                      /etc/ssl/certs, /etc/ssl/certificates].
+                                      /etc/ssl/certs, /etc/ssl/certificates
       -c, --certfile FILE             Path name of a PEM file containing a X.509
                                       client certificate that is used to enable
                                       TLS/SSL 2-way authentication by presenting

diff --git a/pywbemtools/pywbemcli/_pywbem_server.py b/pywbemtools/pywbemcli/_pywbem_server.py
@@ -23,6 +23,7 @@
 from collections import OrderedDict
 import click
 
+import pywbem
 from pywbem import WBEMServer, configure_loggers_from_string
 
 from .config import DEFAULT_URL_SCHEME, DEFAULT_CONNECTION_TIMEOUT, \
@@ -455,11 +456,22 @@ def create_connection(self, log=None, use_pull=None, pull_max_cnt=None,
             else:
                 no_verification = not self.verify
 
+            # Convert ca_certs command line option to ca_certs parameter
+            if hasattr(pywbem, 'CA_CERTS_SYSTEM'):
+                if self.ca_certs == 'certifi':
+                    ca_certs = None
+                elif self.ca_certs == 'system':
+                    ca_certs = pywbem.CA_CERTS_SYSTEM
+                else:
+                    ca_certs = self.ca_certs
+            else:
+                ca_certs = self.ca_certs
+
             conn = PYWBEMCLIConnection(
                 self.server, creds,
                 default_namespace=self.default_namespace,
                 no_verification=no_verification,
-                x509=x509_dict, ca_certs=self.ca_certs,
+                x509=x509_dict, ca_certs=ca_certs,
                 timeout=self.timeout,
                 use_pull_operations=use_pull,
                 stats_enabled=timestats)

diff --git a/pywbemtools/pywbemcli/pywbemcli.py b/pywbemtools/pywbemcli/pywbemcli.py
@@ -29,9 +29,8 @@
 from prompt_toolkit.history import FileHistory
 from prompt_toolkit.auto_suggest import AutoSuggestFromHistory
 
-
 import pywbem
-from pywbem import DEFAULT_CA_CERT_PATHS, LOGGER_SIMPLE_NAMES, \
+from pywbem import LOGGER_SIMPLE_NAMES, \
     LOG_DESTINATIONS, DEFAULT_LOG_DESTINATION, LOG_DETAIL_LEVELS, \
     DEFAULT_LOG_DETAIL_LEVEL
 
@@ -59,6 +58,26 @@
 # enable -h as additional help option
 CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])
 
+if hasattr(pywbem, 'CA_CERTS_SYSTEM'):
+    CA_CERTS_HELP = \
+        'CA certificates to be used for verifying the server certificate ' \
+        'presented by the WBEM server during TLS/SSL handshake: ' \
+        '"system": use system-provided certs; ' \
+        '"certifi": use certs from certifi Python package; ' \
+        'FILE: use the certs in the specified cert file; ' \
+        'DIR: use the certs in the specified cert directory. ' \
+        'Default: EnvVar {ev}, or "certifi".'. \
+        format(ev=PywbemServer.ca_certs_envvar)
+else:
+    CA_CERTS_HELP = \
+        'CA certificates to be used for verifying the server certificate ' \
+        'presented by the WBEM server during TLS/SSL handshake: ' \
+        'FILE: use the certs in the specified cert file; ' \
+        'DIR: use the certs in the specified cert directory. ' \
+        'Default: EnvVar {ev}, or the first existing directory in: {dirs}'. \
+        format(ev=PywbemServer.ca_certs_envvar,
+               dirs=', '.join(pywbem.DEFAULT_CA_CERT_PATHS))
+
 
 # pylint: disable=bad-continuation
 # PywbemcliTopGroup sets order commands listed in help output
@@ -119,15 +138,10 @@
                    'handshake. If --no-verify client bypasses verification. '
                    'Default: EnvVar {ev}, or "--verify".'.
                    format(ev=PywbemServer.verify_envvar))
-@click.option('--ca-certs', type=str, metavar="FILE",
+@click.option('--ca-certs', type=str, metavar="CACERTS",
               default=None,  # defaulted in code
               envvar=PywbemServer.ca_certs_envvar,
-              help='Path name of a file or directory containing certificates '
-                   'that will be matched against the server certificate '
-                   'presented by the WBEM server during TLS/SSL handshake. '
-                   'Default: EnvVar {ev}, or [{dirs}].'.
-                   format(ev=PywbemServer.ca_certs_envvar,
-                          dirs=', '.join(DEFAULT_CA_CERT_PATHS)))
+              help=CA_CERTS_HELP)
 @click.option('-c', '--certfile', type=str, metavar="FILE",
               # defaulted in code
               envvar=PywbemServer.certfile_envvar,
@@ -359,7 +373,7 @@ def create_server_instance(svr_name):
     resolved_default_namespace = default_namespace or DEFAULT_NAMESPACE
     resolved_timestats = timestats or DEFAULT_TIMESTATS
     resolved_verify = DEFAULT_VERIFY if verify is None else verify
-    resolved_ca_certs = DEFAULT_CA_CERT_PATHS if ca_certs is None else ca_certs
+    resolved_ca_certs = 'certifi' if ca_certs is None else ca_certs
 
     if server and svr_name:
         raise click.ClickException(

diff --git a/tests/unit/test_general_options.py b/tests/unit/test_general_options.py
@@ -25,6 +25,7 @@
 import os
 import sys
 import pytest
+import pywbem
 
 from .cli_test_extensions import CLITestsBase
 
@@ -37,6 +38,30 @@
 BAD_PY_ERR_STRTUP_PATH = os.path.join(SCRIPT_DIR, 'py_err_processatstartup.py')
 MOCK_PW_PROMPT_PATH = os.path.join(SCRIPT_DIR, 'mock_password_prompt.py')
 
+if hasattr(pywbem, 'CA_CERTS_SYSTEM'):
+    CA_CERTS_HELP = """
+  --ca-certs CACERTS              CA certificates to be used for verifying the
+                                  server certificate presented by the WBEM
+                                  server during TLS/SSL handshake: "system":
+                                  use system-provided certs; "certifi": use
+                                  certs from certifi Python package; FILE: use
+                                  the certs in the specified cert file; DIR:
+                                  use the certs in the specified cert
+                                  directory. Default: EnvVar
+                                  PYWBEMCLI_CA_CERTS, or "certifi"."""
+else:
+    CA_CERTS_HELP = """
+  --ca-certs CACERTS              CA certificates to be used for verifying the
+                                  server certificate presented by the WBEM
+                                  server during TLS/SSL handshake: FILE: use
+                                  the certs in the specified cert file; DIR:
+                                  use the certs in the specified cert
+                                  directory. Default: EnvVar
+                                  PYWBEMCLI_CA_CERTS, or the first existing
+                                  directory in: /etc/pki/ca-
+                                  trust/extracted/openssl/ca-bundle.trust.crt,
+                                  /etc/ssl/certs, /etc/ssl/certificates"""
+
 GENERAL_HELP = """
 Usage: pywbemcli [GENERAL-OPTIONS] COMMAND [ARGS]...
 
@@ -104,14 +129,9 @@
                                   server during TLS/SSL handshake. If --no-
                                   verify client bypasses verification.
                                   Default: EnvVar PYWBEMCLI_VERIFY, or "--
-                                  verify".
-  --ca-certs FILE                 Path name of a file or directory containing
-                                  certificates that will be matched against
-                                  the server certificate presented by the WBEM
-                                  server during TLS/SSL handshake. Default:
-                                  EnvVar PYWBEMCLI_CA_CERTS, or [/etc/pki/ca-
-                                  trust/extracted/openssl/ca-bundle.trust.crt,
-                                  /etc/ssl/certs, /etc/ssl/certificates].
+                                  verify".""" + \
+    CA_CERTS_HELP + \
+    """
   -c, --certfile FILE             Path name of a PEM file containing a X.509
                                   client certificate that is used to enable
                                   TLS/SSL 2-way authentication by presenting
@@ -278,7 +298,9 @@ class       Command group for CIM classes.
      {'general': ['-s', 'http://blah:abcd'],
       'cmdgrp': 'class',
       'args': ['get', 'blah']},
-     {'stderr': ['Error:', 'ConnectionError', 'Socket error'],
+     {'stderr': ['Error:', 'ConnectionError',
+                 'Failed to parse' if hasattr(pywbem, 'CA_CERTS_SYSTEM')
+                 else 'Socket error'],
       'rc': 1,
       'test': 'regex'},
      None, OK],