Several changes, fixes and improvements on WBEMSubscriptionManager

Details on code changes:

- In order to support conditional creation of owned filters, destinations,
  and subscriptions, it became necessary to verify the properties of the
  instances, and not just their Name property or instance paths.
  This caused a number of changes, some of which are changes at the
  external interface of class `WBEMSubscriptionManager`:

  - Changed the internal instance variables that register owned instances, to
    hold the `CIMInstance` objects instead of the `CIMInstanceName` objects.
  - In all `_create_*()` functions, returned the instances instead of the
    instance paths, after retrieving the created instances using GetInstance in
    order to make sure we have the exact property values as created by the
    server (not just the ones we have passed in to CreateInstance).
  - In addition, these functions now use GetInstance to make sure the sub.mgr.
    has the correct properties in case the server implementation does not honor
    all properties provided in CreateInstance.
  - In addition, these functions now perform ModifyInstance for owned instances,
    when they find that an instance with the desired instance path exists.
  - In `add_*()` and `get_*()`, changed the return value to now return
    `CIMInstance` objects instead of the `CIMInstanceName` objects (or lists
    thereof).

- Added the possibility to directly specify a value for the `Name` property
  of indication filters.

- Added a `get_owned_destinations()` method that was missing so far, for
  consistency with similar methods for filters and subscriptions.

- Changed logic in `add_subscriptions()` to be recursive instead of iterative,
  for consistency with the other `add_*()` methods, particularly because that
  allows for lists witin lists (even though that is not documented).

- In `add_subscriptions()`, performed the check for not allowing permanent
  subscriptions on owned destinations for all destinations, before any
  creation is attempted. This makes the error behavior more transparent.

- In `remove_destinations()`, `remove_filter()`, and `remove_subscriptions()`,
  fixed a bug in the loop that iterates over the owned instances to delete
  the list items that match, that would have deleted the wrong item in case
  multiple items had matched.

- Removed default `None` from `subscription_manager_id` argument of
  `__init__()`, because it is stated and enforced to be not `None`.

Details on documentation related changes:

- Added a Jupyter notebook 'subscriptionmanager.ipynb' with a tutorial about
  the subscription manager.

- Improved the description of the ownership types in the module docstring.
  It now describes three types of ownership: owned, permanent, and static.

- Simplified the lengthy description of the `owned` argument in all methods
  that have it, based upon the better explanation in the module docstring.

- In the documentation of `get_owned_filters()` and
  `get_owned_subscriptions()`, removed the mentioning of raised exceptions,
  because these methods do not raise any exceptions.

Details on code quality changes:

- Removed pylint disable=line-too-long for some functions that meanwhile no
  longer needed it, and added it to some functions that missed it.

- Some clarifications in comments in the code.

docs/changes.rst

@@ -20,6 +20,13 @@ pywbem v0.10.0.dev0
 
 Released: Not yet
 
+Incompatible changes
+^^^^^^^^^^^^^^^^^^^^
+
+* All methods of the WBEMSubscriptionManager class that returned instance
+  paths (or lists thereof) in pywbem v0.9 now return the complete instances
+  (or lists thereof).
+
 Deprecations
 ^^^^^^^^^^^^
 
@@ -54,6 +61,13 @@ Enhancements
 * Added `git` as an OS-level dependency for development (it is used by GitPython
   when building the documentation).
 
+* The Name property of indication filters reated via the
+  WBEMSubscriptionManager class can now be controlled by the user.
+
+* Indication filter, listener destination and indication subscription
+  instances created via the WBEMSubscriptionManager class, that are "owned",
+  are now conditionally created, dependent on the owned instances that have
+  been discovered upon restart of the WBEMSubscriptionManager.
 
 Bug fixes
 ^^^^^^^^^

docs/notebooks/subscriptionmanager.ipynb

@@ -0,0 +1,197 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Subscription Manager"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a href=\"#\" onclick=\"history.back()\">&lt;--- Back</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The [`WBEMSubscriptionManager`](https://pywbem.readthedocs.io/en/latest/indication.html#pywbem.WBEMSubscriptionManager) class is a subscription manager that provides for creating and removing indication subscriptions, indication filters and listener destinations for multiple WBEM servers and multiple WBEM listeners and for getting information about existing indication subscriptions.\n",
+    "\n",
+    "A WBEM listener that is used with this subscription manager is identified through its URL, so it may be a the WBEM listener provided by pywbem (see class [`WBEMListener`](https://pywbem.readthedocs.io/en/latest/indication.html#pywbem.WBEMListener)) or any external WBEM listener.\n",
+    "\n",
+    "This tutorial presents a full blown example on how to set up and shut down a subscription manager and some filters, listener destinations and subscription, including error handling. The code sections shown below are meant to be concatenated into one script."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following code fragment creates a connection with the WBEM server that is the target for the subscriptions, i.e. the server that lateron will emit the indications. The code also defines the URL of a WBEM listener that will be the receiver of the indications. The WBEM listener is not subject of this tutorial, so we assume it just exists, for the purpose of this tutorial:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "from __future__ import print_function\n",
+    "import pywbem\n",
+    "\n",
+    "# The WBEM server that should emit the indications\n",
+    "server = 'http://myserver'\n",
+    "username = 'user'\n",
+    "password = 'password'\n",
+    "\n",
+    "# The URL of the WBEM listener\n",
+    "listener_url = 'http://mylistener'\n",
+    "\n",
+    "def main():\n",
+    "    conn = pywbem.WBEMConnection(server, (username, password),\n",
+    "                                 no_verification=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we use the [`WBEMServer`](https://pywbem.readthedocs.io/en/latest/server.html#pywbem.WBEMServer) class to determine the interop namespace of the WBEM server. Determination of the Interop namespace requires interaction with the WBEM server, so exceptions may happen.\n",
+    "\n",
+    "In this tutorial, we use a try-block to show where exceptions can happen, but we just re-raise them without doing any recovery."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "    server = pywbem.WBEMServer(conn)\n",
+    "    try:\n",
+    "        interop_ns = server.interop_ns\n",
+    "    except pywbem.Error:\n",
+    "        raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we create the subscription manager, and add the WBEM server with it. This causes interaction to happen with the WBEM server: The add_server() method determines whether the WBEM server has any listener destinations, indication filters, or subscriptions that are owned by this subscription manager. This is determined based upon the subscription manager ID ('fred' in this example). Such instances could exist for example if this script was used before and has been aborted or failed:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "    sub_mgr = WBEMSubscriptionManager(subscription_manager_id='fred')\n",
+    "\n",
+    "    try:\n",
+    "        server_id = sub_mgr.add_server(server)\n",
+    "    except pywbem.Error as exc:\n",
+    "        raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, we want to create a listener destination for our WBEM listener. We use \"owned\" listener destinations in order to benefit from the automatic recovery, conditional creation, and cleanup of owned instances (see [section WBEMSubscriptionManager](https://pywbem.readthedocs.io/en/latest/indication.html#wbemsubscriptionmanager) for details about owned and not-owned instances).\n",
+    "\n",
+    "Because the subscription manager has discovered instances owned by it already, the add_listener_destination() method creates a listener destination instance only if it does not exist yet. That makes our code easy, because we don't have to care about that. However, because the method possibly needs to create an instance in the WBEM server, we need to handle any exceptions:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "    try:\n",
+    "        dest_path = sub_mgr.add_listener_destinations(server_id, listener_url)\n",
+    "    except pywbem.Error as exc:\n",
+    "        raise"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this tutorial, we create a dynamic filter. We could have used a static (pre-existing) filter as well, of course.\n",
+    "\n",
+    "Suppose, the management profile we implement requires us to use a specific filter name, so we cannot make the filter \"owned\" (ownership requires that the subscription manager has control over the filter name).\n",
+    "\n",
+    "We further assume for the sake of simplicity of ths tutorial, that the subscription is active only as long as the script runs, and that we tear everything down at the end of the script. Because the filter is not-owned, we need to clean up the filter explicitly, and we use a try-block to make sure that will happen at the end, even if exceptions happen in between."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "try:\n",
+    "    filter_path = sub_mgr.add_filter(\n",
+    "        server_id=server_id,\n",
+    "        source_namespace=interop_ns,\n",
+    "        query_language=\"DMTF:CQL\"\n",
+    "        query=\"SELECT * FROM CIM_AlertIndication \" \\\n",
+    "              \"WHERE OwningEntity = 'DMTF' \" \\\n",
+    "              \"AND MessageID LIKE 'SVPC0123|SVPC0124|SVPC0125'\")\n",
+    "        owned=False,\n",
+    "        name=\"DMTF:System Virtualization:Alerts\")\n",
+    "\n",
+    "\n",
+    "\n",
+    "finally:\n",
+    "    sub_mgr.    \n",
+    "        \n",
+    "\n",
+    "    subscription_manager.add_subscription(server2_id, filter2_path)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a href=\"#\" onclick=\"history.back()\">&lt;--- Back</a>"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

examples/pegasusindicationtest.py

@@ -196,9 +196,10 @@ def run_test(svr_url, listener_host, user, password, http_listener_port, \
         old_subs = sub_mgr.get_all_subscriptions(server_id)
         # TODO filter for our sub mgr
         if len(old_subs) != 0 or len(old_filters) != 0:
-            sub_mgr.remove_subscriptions(server_id, old_subs)
+            sub_mgr.remove_subscriptions(server_id,
+                                         [inst.path for inst in old_subs])
             for filter_ in old_filters:
-                sub_mgr.remove_filter(server_id, filter_)
+                sub_mgr.remove_filter(server_id, filter_.path)
 
     except ConnectionError as ce:
         print('Connection Error %s with %s' % (ce, svr_url))
@@ -227,11 +228,11 @@ def run_test(svr_url, listener_host, user, password, http_listener_port, \
     sub_mgr.add_listener_destinations(server_id, listener_url)
 
     # Create a dynamic alert indication filter and subscribe for it
-    filter_path = sub_mgr.add_filter(
+    filter_ = sub_mgr.add_filter(
         server_id, TEST_CLASS_NAMESPACE,
         TEST_QUERY,
         query_language="DMTF:CQL")
-    subscription_paths = sub_mgr.add_subscriptions(server_id, filter_path)
+    subscriptions = sub_mgr.add_subscriptions(server_id, filter_.path)
 
     # Request server to create indications by invoking method
     # This is pegasus specific
@@ -259,8 +260,9 @@ def run_test(svr_url, listener_host, user, password, http_listener_port, \
             time.sleep(requested_indications/150)
 
 
-    sub_mgr.remove_subscriptions(server_id, subscription_paths)
-    sub_mgr.remove_filter(server_id, filter_path)
+    sub_mgr.remove_subscriptions(server_id,
+                                 [inst.path for inst in subscriptions])
+    sub_mgr.remove_filter(server_id, filter_.path)
     sub_mgr.remove_server(server_id)
     LISTENER.stop()