Regexp support for name filtering in find() and findall()

Details:

* Fixed that the 'find()' and 'findall()' methods now also support regular
  expression matching when the resource name is passed as a filter argument.
  (issue #1070)

* Improved the 'list()' methods by using the name-to-URI cache when the
  resource name is passed as a filter argument. (part of issue #1070)
  This is an incompatible change for code that accesses properties of
  the returned resource via resource.properties and needs to be fixed
  by changing the code to use either resource.get_property() or by
  setting full_properties=True on the list() method.

* Improved the 'delete()' methods of zhmcclient resource classes by setting
  the ceased-existence flag on the resource. This will cause optimized
  find-like methods that operate on local data to properly raise
  CeasedExistence when used on the deleted resource object.
  (part of issue #1070)
  Depending on the user code, this change may also be incompatible.

* Added unit test cases for find() and findall() to test the regular
  expression matching that is now supported.

* Adjusted unit test cases for the name-to-URI caching because they
  depend on the internal call structure.

* Adjusted testcases for the get_property() and prop() methods to
  accomodate for the changed ceased-existence and full-properties
  behavior.

Signed-off-by: Andreas Maier <maiera@de.ibm.com>

docs/changes.rst

@@ -44,6 +44,23 @@ Released: not yet
 * The 'base_url' property of the 'zhmcclient.Session' object is now 'None' when
   the session is in the logged-off state. (related to issue #1024)
 
+* The 'list()' methods of zhmcclient manager objects when invoked with
+  full_properties=False and with the resource name as the only filter argument
+  now return only a minimal set of properties for the returned resource:
+  'class', 'parent', 'name', 'object/element-id', 'object/element-uri'.
+  Previously, the full set of properties was returned in such a case.
+  Code that accesses one of the no longer returned properties via
+  'resource.properties' will now fail with KeyError. This can be fixed by
+  changing such code to access the property via 'resource.get_property()',
+  or by specifying 'full_properties=True' on the 'list()' method.
+  (part of issue #1070)
+
+* The 'delete()' methods of zhmcclient resource objects now also set the
+  ceased-existence flag on the resource object. This causes 'get_property()'
+  and prop()' when called for locally available properties to now raise
+  CeasedExistence. Previously, the locally available property value was
+  returned. (part of issue #1070)
+
 **Deprecations:**
 
 **Bug fixes:**
@@ -67,6 +84,10 @@ Released: not yet
 * Docs: Clarified in 'StorageGroup.list_candidate_adapter_ports()' that the
   method is only for FCP-type storage groups.
 
+* Fixed that the 'find()' and 'findall()' methods now also support regular
+  expression matching when the resource name is passed as a filter argument.
+  (issue #1070)
+
 **Enhancements:**
 
 * Added support for retrievel of firmware from an FTP server to the
@@ -110,6 +131,19 @@ Released: not yet
 * Mock support: Added mock support for the Logon and Logoff HMC operations.
   (related to issue #1024)
 
+* Improved the 'list()' methods of zhmcclient manager classes by using the
+  name-to-URI cache when the resource name is passed as a filter argument.
+  This improvement avoids retrieving the resource from the HMC when it can be
+  found in the name-to-URI case, and therefore the resource will have only a
+  minimal set of properties in that case. See the corresponding entry in the
+  Incompatibilities section. (part of issue #1070)
+
+* Improved the 'delete()' methods of zhmcclient resource classes by setting
+  the ceased-existence flag on the resource. This will cause optimized
+  find-like methods that operate on local data to properly raise
+  CeasedExistence when used on the deleted resource object.
+  (part of issue #1070)
+
 **Cleanup:**
 
 **Known issues:**

tests/common/utils.py

@@ -88,7 +88,11 @@ def assert_resources(resources, exp_resources, prop_names):
                 # (e.g. Crypto adapter does not have 'port-count')
                 prop_value = res.properties[prop_name]
                 exp_prop_value = exp_res.properties[prop_name]
-                assert prop_value == exp_prop_value
+                assert prop_value == exp_prop_value, (
+                    "Unexpected value for property {n!r}: got: {av!r}, "
+                    "expected: {ev!r}".
+                    format(n=prop_name, av=prop_value, ev=exp_prop_value)
+                )
 
 
 def info(capsys, format_str, format_args=None):

tests/end2end/test_auto_update.py

@@ -167,30 +167,32 @@ def test_autoupdate_prop(dpm_mode_cpcs):  # noqa: F811
 
             _ = repr(part_auto)
 
-            # Test that accessing properties of the first partition does not
-            # raise CeasedExistence
-
-            desc = part.get_property('description')
-            assert desc == new_desc
-
-            desc = part.prop('description')
-            assert desc == new_desc
+            # Test that accessing properties of the first partition also
+            # raises CeasedExistence (the ceased-existence state is set by
+            # the delete() method).
 
             ce = part.ceased_existence
-            assert ce is False
+            assert ce is True
+
+            with pytest.raises(zhmcclient.CeasedExistence) as exc_info:
+                _ = part.get_property('description')
+            exc = exc_info.value
+            assert exc.resource_uri == part.uri
 
-            # Test that using methods of the first partition that need to
-            # use the partition on the HMC raise HTTP 404.1
+            with pytest.raises(zhmcclient.CeasedExistence) as exc_info:
+                _ = part.prop('description')
+            exc = exc_info.value
+            assert exc.resource_uri == part.uri
 
-            with pytest.raises(zhmcclient.HTTPError) as exc_info:
+            with pytest.raises(zhmcclient.CeasedExistence) as exc_info:
                 part.pull_full_properties()
             exc = exc_info.value
-            assert exc.http_status == 404 and exc.reason == 1
+            assert exc.resource_uri == part.uri
 
-            with pytest.raises(zhmcclient.HTTPError) as exc_info:
+            with pytest.raises(zhmcclient.CeasedExistence) as exc_info:
                 _ = part.dump()
             exc = exc_info.value
-            assert exc.http_status == 404 and exc.reason == 1
+            assert exc.resource_uri == part.uri
 
         finally:
             # We want to make sure the test partition gets cleaned up after

tests/end2end/utils.py

@@ -384,7 +384,7 @@ def runtest_find_list(session, manager, name, server_prop, client_prop,
         assert len(found_res_list) == 1
         found_res = found_res_list[0]
         assert_res_props(found_res, exp_props, ignore_values=volatile_props,
-                         prop_names=list_props)
+                         prop_names=minimal_props)
 
     if client_prop:
         # The code to be tested: list() with client-side filter and short props
@@ -393,7 +393,7 @@ def runtest_find_list(session, manager, name, server_prop, client_prop,
         assert name in [_res.name for _res in found_res_list]
         found_res = [_res for _res in found_res_list if _res.name == name][0]
         assert_res_props(found_res, exp_props, ignore_values=volatile_props,
-                         prop_names=list_props)
+                         prop_names=minimal_props)
 
 
 def runtest_get_properties(