v2.1.1 changes

CHANGELOG.rst

@@ -1,91 +1,111 @@
 Changelog
 =========
 
+2.1.1 - in development
+----------------------
+
+* After running ``st2 pack install`` CLI command display which packs have been installed.
+  (improvement)
+* Update ``/v1/packs/register`` API endpoint so it throws on failure (e.g. invalid pack or resource
+  metadata). This way the default behavior is consistent with default
+  ``st2ctl reload --register-all`` behavior.
+
+  If user doesn't want the API endpoint to fail on failure, they can pass
+  ``"fail_on_failure": false`` attribute in the request payload. (improvement)
+* Throw a more user-friendly exception when registering packs (``st2ctl reload``) if pack ref /
+  name is invalid. (improvement)
+* ``core.http`` action now also supports HTTP basic auth and digest authentication by passing
+  ``username`` and ``password`` parameter to the action. (new feature)
+* Fix ``GET /v1/packs/<pack ref or id>`` API endpoint - make sure pack object is correctly returned
+  when pack ref doesn't match pack name. Previously, 404 not found was thrown. (bug fix)
+* Update local action runner so it supports and works with non-ascii (unicode) parameter keys and
+  values. (bug fix)
+
+  Contribution by Hiroyasu OHYAMA. #3116
+* Update ``packs.load`` action to also register triggers by default. (improvement)
+* Update ``/v1/packs/register`` API endpoint so it registers resources in the correct order which
+  is the same as order used in ``st2-register-content`` script. (bug fix)
+
 2.1.0 - December 05, 2016
 -------------------------
 
+* Pack management changes:
+
+  - Add new ``stackstorm_version`` and ``system`` fields to the pack.yaml metadata file. Value of the
+    first field can contain a specific StackStorm version with which the pack is designed to work
+    with (e.g. ``>=1.6.0,<2.2.0`` or ``>2.0.0``). This field is checked when installing / registering
+    a pack and installation is aborted if pack doesn't support the currently running StackStorm version.
+    Second field can contain an object with optional system / OS level dependencies.
+    (new feature)
+  - Add new ``contributors`` field to the pack metadata file. This field can contain a list of
+    people who have contributed to the pack. The format is ``Name <email>``, e.g.
+    ``Tomaz Muraus <tomaz@stackstorm.com>`` (new feature)
+  - Add support for default values and dynamic config values for nested config objects. (new feature, improvement)
+  - Add new ``st2-validate-pack-config`` tool for validating config file against a particular config
+    schema file. (new-feature)
+  - Improved pack validation - now when the packs are registered we check that:
+
+    + ``version`` attribute in the pack metadata file matches valid semver format (e.g
+      ``0.1.0``, ``2.0.0``, etc.)
+    + ``email`` attribute (if specified) contains a valid email address. (improvement)
+    + Only valid word characters (``a-z``, ``0-9`` and ``_``) used for action parameter
+      names. Previously, due to bug in the code, any character was allowed.
+
+    If validation fails, pack registration will fail. If you have an existing action or pack definition which
+    uses invalid characters, pack registration will fail. **You must update your packs**.
+  - For consistency with new pack name validation changes, sample ``hello-st2`` pack has been renamed
+    to ``hello_st2``.
+  - Fix ``packs.uninstall`` action so it also deletes ``configs`` and ``policies`` which belong to
+    the pack which is being uninstalled. (bug fix)
+  - Update ``packs.install`` action (``pack install`` command) to only load resources from the packs
+    which are being installed. Also update it and remove "restart sensor container" step from the
+    install workflow. This step hasn't been needed for a while now because sensor container
+    dynamically reads a list of available sensors from the database and starts the sub processes.
+    (improvement)
+  - Remove ``packs.info`` action because ``.gitinfo`` file has been deprecated with the new pack
+    management approach. Now pack directories are actual checkouts of the corresponding pack git
+    repositories so this file is not needed anymore.
+
 * Add new ``POST /v1/actionalias/match`` API endpoint which allows users to perform ChatOps action
   alias matching server-side. This makes it easier to build and maintain StackStorm ChatOps
   clients / adapters for various protocols and mediums. Clients can now be very thin wrappers
   around this new API endpoint.
 
   Also add two new corresponding CLI commands - ``st2 alias-execution match`` and
-  ``st2 alias-execution execute``. (new feature) #2895
-
-  Contributed by Anthony Shaw
-* Only allow valid word characters (``a-z``, ``0-9`` and ``_``) to be used for action parameter
-  names. Previously, due to bug in the code, any character was allowed.
-
-  If you have an existing action definition which uses parameter with an invalid character, the
-  parameter name needs to be adjusted otherwise action registration will fail. (bug fix,
-  improvement)
+  ``st2 alias-execution execute``. Contribution by Anthony Shaw. (new feature) #2895.
 * Adding ability to pass complex array types via CLI by first trying to
   seralize the array as JSON and then falling back to comma separated array.
 * Add new ``core.pause`` action. This action behaves like sleep and can be used inside the action
   chain or Mistral workflows where waiting / sleeping is desired before proceeding with a next
-  task. (new feature) #2933
-
-  Contribution by Paul Mulvihill.
-* Improve API exception handling and make sure 400 status code is returned instead of 500 on
-  mongoengine field validation error. (improvement)
-* Allow user to supply multiple resource ids using ``?id`` query parameter when filtering
-  "get all" API endpoint result set (e.g. `?id=1,2,3,4`). This allows for a better client and
-  servers performance when user is polling and interested in multiple resources such as polling on
-  multiple action executions. (improvement)
-* Actually validate pack metadata file when registering packs. (bug fix, improvement)
-* Improve pack validation - now when the packs are registered we require that
-  "version" attribute in the pack metadata file matches valid semver format and we also require
-  that "email" attribute (if specified) contains a valid email address. (improvement)
-* For consistency with new pack name validation changes, sample ``hello-st2`` pack has been renamed
-  to ``hello_st2``.
-* Add new ``stackstorm_version`` and ``system`` fields to the pack.yaml metadata file. Value of the
-  first field can contain a specific for StackStorm version with which the pack is designed to work
-  with (e.g. ``>=1.6.0,<2.2.0`` or ``>2.0.0``). This field is checked when installing / registering
-  a pack and installation is aborted if pack doesn't support StackStorm version which is currently
-  running. Second field can contain an object with optional system / OS level dependencies.
-  (new feature)
-* Require pack metadata ``version`` attribute to contain a valid semver version identifier (e.g
-  ``0.1.0``, ``2.0.0``, etc.). If version identifier is invalid, pack registration will fail.
-  (improvement)
+  task. Contribution by Paul Mulvihill. (new feature) #2933. 
 * When a policy cancels a request due to concurrency, it leaves end_timestamp set to None which
   the notifier expects to be a date. This causes an exception in "isotime.format()". A patch was
   released that catches this exception, and populates payload['end_timestamp'] with the equivalent
   of "datetime.now()" when the exception occurs.
 * Adding check for datastore Client expired tokens used in sensor container
-* Add new ``contributors`` field to the pack metadata file. This field can contain a list of
-  people who have contributed to the pack. The format is ``Name <email>``, e.g.
-  ``Tomaz Muraus <tomaz@stackstorm.com>`` (new feature)
-* Update ``packs.install`` action (``pack install`` command) to only load resources from the packs
-  which are being installed. Also update it and remove "restart sensor container" step from the
-  install workflow. This step hasn't been needed for a while now because sensor container
-  dynamically reads a list of available sensors from the database and starts the sub processes.
-  (improvement)
+* Improve API exception handling and make sure 400 status code is returned instead of 500 on
+  mongoengine field validation error. (improvement)
+* Throw a more user-friendly exception if rendering a dynamic configuration value inside the config
+  fails. (improvement)
 * Change st2api so that a full execution object is returned instead of an error message, when an
   API client requests cancellation of an execution that is already canceled
-* Remove ``packs.info`` action because ``.gitinfo`` file has been deprecated with the new pack
-  management approach. Now pack directories are actual checkouts of the corresponding pack git
-  repositories so this file is not needed anymore.
 * Speed up short-lived Python runner actions by up to 70%. This way done by re-organizing and
   re-factoring code to avoid expensive imports such as jsonschema, jinja2, kombu and mongoengine
   in the places where those imports are not actually needed and by various other optimizations.
   (improvement)
-* Add new ``st2-validate-pack-config`` tool for validating config file against a particular config
-  schema file. (new-feature)
-* Upgrade various internal Python library dependencies to the latest stable versions (gunicorn,
-  kombu, six, appscheduler, passlib, python-gnupg, semver, paramiko, python-keyczar, virtualenv).
 * Improve performance of ``GET /executions/views/filters`` by creating additional indexes on
   executions collection
-* Add support for default values and dynamic config values for nested config objects. (new feature,
-  improvement)
-* Throw a more user-friendly exception if rendering a dynamic configuration value inside the config
-  fails. (improvement)
+* Allow user to supply multiple resource ids using ``?id`` query parameter when filtering
+  "get all" API endpoint result set (e.g. `?id=1,2,3,4`). This allows for a better client and
+  servers performance when user is polling and interested in multiple resources such as polling on
+  multiple action executions. (improvement)
+* Upgrade various internal Python library dependencies to the latest stable versions (gunicorn,
+  kombu, six, appscheduler, passlib, python-gnupg, semver, paramiko, python-keyczar, virtualenv).
 * Add support for ssh config file for ParamikoSSHrunner. Now ``ssh_config_file_path`` can be set
   in st2 config and can be used to access remote hosts when ``use_ssh_config`` is set to
   ``True``. However, to access remote hosts, action paramters like username and
   password/private_key, if provided with action, will have precedence over the config file
   entry for the host. #2941 #3032 #3058 [Eric Edgar] (improvement)
-* Fix ``packs.uninstall`` action so it also deletes ``configs`` and ``policies`` which belong to
-  the pack which is being uninstalled. (bug fix)
 * Fix python action runner actions and make sure that modules from ``st2common/st2common/runners``
   directory don't pollute ``PYTHONPATH`` for python runner actions. (bug fix)
 

contrib/packs/actions/load.yaml

@@ -7,8 +7,8 @@
   parameters:
     register:
       type: "string"
-      default: "actions,aliases,sensors"
-      description: "Possible options are all, sensors, actions, rules, aliases."
+      default: "actions,aliases,sensors,triggers"
+      description: "Possible options are all, sensors, actions, rules, aliases, triggers, configs."
     packs:
       type: "array"
       description: "A list of packs to register / load resources from."

contrib/runners/http_runner/http_runner.py

@@ -19,6 +19,7 @@
 import uuid
 
 import requests
+from requests.auth import HTTPBasicAuth
 from oslo_config import cfg
 
 from st2common.runners.base import ActionRunner
@@ -40,6 +41,8 @@
 RUNNER_HTTP_PROXY = 'http_proxy'
 RUNNER_HTTPS_PROXY = 'https_proxy'
 RUNNER_VERIFY_SSL_CERT = 'verify_ssl_cert'
+RUNNER_USERNAME = 'username'
+RUNNER_PASSWORD = 'password'
 
 # Lookup constants for action params
 ACTION_AUTH = 'auth'
@@ -77,6 +80,8 @@ def pre_run(self):
 
         self._cookies = self.runner_parameters.get(RUNNER_COOKIES, None)
         self._allow_redirects = self.runner_parameters.get(RUNNER_ALLOW_REDIRECTS, False)
+        self._username = self.runner_parameters.get(RUNNER_USERNAME, None)
+        self._password = self.runner_parameters.get(RUNNER_PASSWORD, None)
         self._http_proxy = self.runner_parameters.get(RUNNER_HTTP_PROXY, None)
         self._https_proxy = self.runner_parameters.get(RUNNER_HTTPS_PROXY, None)
         self._verify_ssl_cert = self.runner_parameters.get(RUNNER_VERIFY_SSL_CERT, None)
@@ -133,7 +138,8 @@ def _get_http_client(self, action_parameters):
         return HTTPClient(url=self._url, method=method, body=body, params=params,
                           headers=headers, cookies=self._cookies, auth=auth,
                           timeout=timeout, allow_redirects=self._allow_redirects,
-                          proxies=proxies, files=files, verify=self._verify_ssl_cert)
+                          proxies=proxies, files=files, verify=self._verify_ssl_cert,
+                          username=self._username, password=self._password)
 
     @staticmethod
     def _get_result_status(status_code):
@@ -144,7 +150,7 @@ def _get_result_status(status_code):
 class HTTPClient(object):
     def __init__(self, url=None, method=None, body='', params=None, headers=None, cookies=None,
                  auth=None, timeout=60, allow_redirects=False, proxies=None,
-                 files=None, verify=False):
+                 files=None, verify=False, username=None, password=None):
         if url is None:
             raise Exception('URL must be specified.')
 
@@ -172,6 +178,8 @@ def __init__(self, url=None, method=None, body='', params=None, headers=None, co
         self.proxies = proxies
         self.files = files
         self.verify = verify
+        self.username = username
+        self.password = password
 
     def run(self):
         results = {}
@@ -191,6 +199,9 @@ def run(self):
             else:
                 data = self.body
 
+            if self.username or self.password:
+                self.auth = HTTPBasicAuth(self.username, self.password)
+
             resp = requests.request(
                 self.method,
                 self.url,

contrib/runners/http_runner/runner.yaml

@@ -21,10 +21,16 @@
     https_proxy:
       description: A URL of a HTTPs proxy to use (e.g. http://10.10.1.10:3128).
       type: string
+    password:
+      description: Password required by basic authentication.
+      type: string
     url:
       description: URL to the HTTP endpoint.
       required: true
       type: string
+    username:
+      description: Username required by basic authentication.
+      type: string
     verify_ssl_cert:
       default: true
       description: Certificate for HTTPS request is verified by default using requests

contrib/runners/http_runner/tests/unit/test_http_runner.py

@@ -125,3 +125,25 @@ def test_https_verify_false(self, mock_requests):
             'GET', url, allow_redirects=False, auth=None, cookies=None,
             data='', files=None, headers={}, params=None, proxies=None,
             timeout=60, verify=False)
+
+    @mock.patch('http_runner.requests')
+    def test_https_auth_basic(self, mock_requests):
+        url = 'https://127.0.0.1:8888'
+        username = 'misspiggy'
+        password = 'kermit'
+        client = HTTPClient(url=url, username=username, password=password)
+        mock_result = MockResult()
+
+        mock_result.text = 'muppet show'
+        mock_result.headers = {'Authorization': 'bWlzc3BpZ2d5Omtlcm1pdA=='}
+        mock_result.status_code = 200
+
+        mock_requests.request.return_value = mock_result
+        result = client.run()
+
+        self.assertEqual(result['headers'], mock_result.headers)
+
+        mock_requests.request.assert_called_once_with(
+            'GET', url, allow_redirects=False, auth=client.auth, cookies=None,
+            data='', files=None, headers={}, params=None, proxies=None,
+            timeout=60, verify=False)