Refactor property_manager configuration and move lock_manager to own …

…package

CHANGELOG.md

@@ -5,9 +5,14 @@
 - Drop Python 2 support
 - Drop support for Python syntax in config files (wsgidav.conf)
 - Drop support for Microsoft Web Folders (option `dir_browser.ms_mount`).
-- DAVCollection, DAVNonCollection, DAVProvider are ABCs now
+- DAVCollection, DAVNonCollection, DAVProvider are now ABCs.
 - API enforces some named keyword args (`..., *, ...`)
-- Rename option `lock_manager` to `lock_storage`.
+- Rename option `lock_manager` to `lock_storage`
+- Moved `lock_manager` and `lock_storage` modules to  `lock_man` package.
+- `lock_storage`, `property_manager`, `provider_mapping`
+   can now be configured in the YAML file to use custom implementations using
+   this syntax:
+   `{ "class": <class_path>, "<arg_name>": <val>, "<arg_name>": <val>, , ... }`
 - hotfixes.re_encode_path_info is true by default (null evaluates to false now!)
 - Deprecate hotfixes.winxp_accept_root_share_login and hotfixes.win_accept_anonymous_options
 - Move logging options to 'logging' section

docs/source/api-doc.rst

@@ -31,8 +31,6 @@ Package ``wsgidav``
    wsgidav.error_printer
    wsgidav.fs_dav_provider
    wsgidav.http_authenticator
-   wsgidav.lock_manager
-   wsgidav.lock_storage
    wsgidav.middleware
    wsgidav.request_resolver
    wsgidav.request_server
@@ -61,6 +59,15 @@ Package ``wsgidav.prop_man``
    wsgidav.prop_man.mongo_property_manager
 
 
+Package ``wsgidav.lock_man``
+--------------------------
+.. autosummary::
+   :toctree: _autosummary
+
+   wsgidav.lock_man.lock_manager
+   wsgidav.lock_man.lock_storage
+
+
 Package ``wsgidav.samples``
 ---------------------------
 .. autosummary::

docs/source/obsolete_sample_wsgidav.conf

@@ -228,21 +228,21 @@ property_manager = True
 # Lock Manager
 #
 # Uncomment this lines to specify your own locks manager.
-# Default:        wsgidav.lock_storage.LockStorageDict
-# Also available: wsgidav.lock_storage.LockStorageShelve
+# Default:        wsgidav.lock_man.lock_storage.LockStorageDict
+# Also available: wsgidav.lock_man.lock_storage.LockStorageShelve
 #
 # Check the documentation on how to develop custom lock managers.
 # Note that the default LockStorageDict works in-memory, and thus is NOT
 # persistent.
 
 # Example: Use in-memory lock storage
 #          (this is the same as passing 'lock_manager = True', which is default)
-#from wsgidav.lock_storage import LockStorageDict
+#from wsgidav.lock_man.lock_storage import LockStorageDict
 #lock_manager = LockStorageDict()
 
 
 # Example: Use PERSISTENT shelve based lock manager
-#from wsgidav.lock_storage import LockStorageShelve
+#from wsgidav.lock_man.lock_storage import LockStorageShelve
 #lock_manager = LockStorageShelve("wsgidav-locks.shelve")
 
 

docs/source/reference_guide_architecture.rst

@@ -113,11 +113,11 @@ interface is implemented.
 Lock Managers
 -------------
 
-.. inheritance-diagram:: wsgidav.lock_manager wsgidav.lock_storage
+.. inheritance-diagram:: wsgidav.lock_man.lock_manager wsgidav.lock_man.lock_storage
    :parts: 2
    :private-bases:
 
-DAV providers have a :class:`~wsgidav.lock_manager.LockManager` to support
+DAV providers have a :class:`~wsgidav.lock_man.lock_manager.LockManager` to support
 exclusive and shared write locking.
 The lock manager uses a lock storage implementation for persistence.
 
@@ -127,8 +127,8 @@ dictionary, and a persistent one based on shelve::
     lock_storage.LockStorage
     lock_storage.ShelveLockStorage
 
-:class:`~wsgidav.lock_storage.LockStorage` is used by default, but
-:class:`~wsgidav.lock_storage.ShelveLockStorage` can be enabled by uncommenting
+:class:`~wsgidav.lock_man.lock_storage.LockStorage` is used by default, but
+:class:`~wsgidav.lock_man.lock_storage.ShelveLockStorage` can be enabled by uncommenting
 two lines in the configuration file.
 
 In addition, this may be replaced by a custom version, as long as the required

docs/source/user_guide_configure.rst

@@ -66,7 +66,7 @@ The options described below can be defined for the CLI either
   * or `JSON <https://www.json.org/>`_ syntax inside a wsgidav.json file
 
 .. note::
-   The three supported file formats are just different ways for the CLI to
+   The two supported file formats are just different ways for the CLI to
    generate a Python dict that is then passed to the
    :class:`~wsgidav.wsgidav_app.WsgiDAVApp` constructor.
 
@@ -76,7 +76,7 @@ For a start, copy
 :download:`YAML Sample Configuration<../../sample_wsgidav.yaml>`
 and edit it to your needs.
 (Alternatively use
-:download:`JSON Sample Configuration<../sample_wsgidav.json>`.)
+:download:`JSON Sample Configuration<../../sample_wsgidav.json>`.)
 
 
 Verbosity Level
@@ -228,9 +228,15 @@ editing depending on authentication.
 
 Three syntax variants are supported:
 
-1. ``<mount_path>: <folder_path>``
-2. ``<mount_path>: { "root": <folder_path>, "readonly": <bool> }``
-3. ``<mount_path>: { "provider": <class_path>, "args:" ..., "kwargs": ... }``
+1. ``<mount_path>: <folder_path>``:
+   use ``FilesystemProvider(folder_path)``
+2. ``<mount_path>: { "root": <folder_path>, "readonly": <bool> }``:
+   use ``FilesystemProvider(folder_path, readonly)``
+3. ``<mount_path>: { "class": <class_path>, "arg1": val1, "arg2": val2, ... }``
+   Instantiate a custom class (derrived from ``DAVProvider``) using named
+   kwargs.
+..
+   1. ``<mount_path>: { "provider": <class_path>, "args:" ..., "kwargs": ... }``
 
 For example::
 
@@ -241,21 +247,52 @@ For example::
             root: "/path/to/share2"
             readonly: true
         "/share3":
-            provider: path.to.CustomDAVProviderClass
+            class: path.to.CustomDAVProviderClass
+            path: '/path/to/share3'
+            another_arg: 42
+..
             args: ["/path/to/share3", "second_arg"]
             kwargs: {"another_arg": 42}
 
 
 Property Manager
 ----------------
 
-.. todo:: TODO
+The built-in ``PropertyManager``.
 
+Possible options are:
 
-Lock Manager
-------------
+- Disable locking, by passing ``property_manager: null``.
+- Enable default storage, which is implemented using a memory-based,
+  **not** persistent storage, by passing ``property_manager: true``.
+  (This is an alias for ``property_manager: wsgidav.prop_man.property_manager.PropertyManager``)
+- Enable an installed or custom storage
 
-.. todo:: TODO
+Example: Use a persistent shelve based property storage::
+
+    property_manager:
+        class: wsgidav.prop_man.property_manager.ShelvePropertyManager
+        storage_path: /path/to/wsgidav_locks.shelve
+
+
+Lock Manager and Storage
+------------------------
+
+The built-in ``LockManager`` requires a ``LockStorage`` instance.
+
+Possible options are:
+
+- Disable locking, by passing ``lock_storage: null``.
+- Enable default locking, which is implemented using a memory-based,
+  **not** persistent storage, by passing ``lock_storage: true``.
+  (This is an alias for ``lock_storage: wsgidav.lock_man.lock_storage.LockStorageDict``)
+- Enable an installed lock storage
+
+Example: Use a persistent shelve based lock storage::
+
+    lock_storage:
+        class: wsgidav.lock_man.lock_storage.LockStorageShelve
+        storage_path: /path/to/wsgidav_locks.shelve
 
 
 Domain Controller
@@ -376,6 +413,19 @@ line like::
     $ wsgidav ... --auth=pam-login
 
 
+Custom Domain Controllers
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A custom domain controller can be used like so::
+
+    http_authenticator:
+        domain_controller: path.to.CustomDomainController
+
+The constructor must accept two arguments::
+
+    def __init__(self, wsgidav_app, config)
+
+
 Sample ``wsgidav.yaml``
 -----------------------
 

sample_wsgidav.yaml

@@ -110,22 +110,23 @@ mount_path: null
 #: to read-only.
 #: Note that a DomainController may still restrict access completely or prevent
 #: editing depending on authentication.
-#: The following syntax variants are supported:
+#:
+#: The following syntax variants are supported to use FilesystemProvider:
 #:     <mount_path>: <folder_path>
 #: or
 #:     <mount_path>: { "root": <folder_path>, "readonly": <bool> }
-#: or
-#:     <mount_path>: { "provider": <class_path>, "args": [<pos-args>, ...] }
+#: or instantiate an arbitrary custom class:
+#:     <mount_path>: { "class": <class_path>, [<arg>: <val>, ...] }
 
 provider_mapping:
     "/": "/path/to/share1"
     "/pub":
-        root: "/path/to/share2"
+        root: '/path/to/share2'
         readonly: true
     "/share3":
-        provider: path.to.CustomDAVProviderClass
-        args: ["/path/to/share3", "second_arg"]
-        kwargs: {"another_arg": 42}
+        class: path.to.CustomDAVProviderClass
+        path: '/path/to/share3'
+        another_arg: 42
 
 
 # ==============================================================================
@@ -183,12 +184,13 @@ pam_dc:
 # null: (default) no support for dead properties
 # true: Use wsgidav.prop_man.property_manager.PropertyManager
 #       which is an in-memory property manager (NOT persistent)
-property_manager: true
+#
+# Example: Use persistent shelve based property manager
+#     property_manager:
+#         class: wsgidav.prop_man.property_manager.ShelvePropertyManager
+#         storage_path: 'wsgidav-props.shelve'
 
-# Example:
-# Use PERSISTENT shelve based property manager
-# from wsgidav.prop_man.property_manager import ShelvePropertyManager
-# property_manager: ShelvePropertyManager("wsgidav-props.shelve")
+property_manager: null
 
 #: Optional additional live property modification
 #: Note: by default live properties like file size and last-modified time are
@@ -209,16 +211,20 @@ mutable_live_props:
 # ----------------------------------------------------------------------------
 # Lock Manager Storage
 #
-# Default:        true, i.e. use wsgidav.lock_storage.LockStorageDict
-# Also available: wsgidav.lock_storage.LockStorageShelve
+# null: No lock support
+# true: (default) shortcut for
+#     lock_storage: wsgidav.lock_man.lock_storage.LockStorageDict
+#
+# Note that the default LockStorageDict works in-memory, so it is
+# NOT persistent.
+#
+# Example: Use persistent shelve based lock storage:
+#     lock_storage:
+#         class: wsgidav.lock_man.lock_storage.LockStorageShelve
+#         storage_path: /path/to/wsgidav_locks.shelve
 #
 # Check the documentation on how to develop custom lock storage.
-# Note that the default LockStorageDict works in-memory, so it is NOT
-# persistent.
-# Example:
-#   Use PERSISTENT shelve based lock storage
-#   from wsgidav.lock_storage import LockStorageShelve
-#   lock_storage = LockStorageShelve("wsgidav-locks.shelve")
+
 lock_storage: true
 
 

tests/test_lock_manager.py

@@ -9,9 +9,9 @@
 from tempfile import gettempdir
 from time import sleep
 
-from wsgidav import lock_manager, lock_storage
 from wsgidav.dav_error import DAVError
-from wsgidav.lock_storage_redis import LockStorageRedis
+from wsgidav.lock_man import lock_manager, lock_storage
+from wsgidav.lock_man.lock_storage_redis import LockStorageRedis
 
 # ========================================================================
 # BasicTest

tests/test_util.py

@@ -12,6 +12,7 @@
 from wsgidav.util import (
     BASE_LOGGER_NAME,
     checked_etag,
+    deep_update,
     get_module_logger,
     init_logging,
     is_child_uri,
@@ -94,6 +95,36 @@ def testBasics(self):
         assert parse_if_match_header(' "abc" , def ') == ["abc", "def"]
         assert parse_if_match_header(' W/"abc" , def ') == ["abc", "def"]
 
+        d_org = {"b": True, "d": {"i": 1, "t": (1, 2)}}
+        d_new = {}
+        assert deep_update(d_org.copy(), d_new) == d_org
+        assert deep_update(d_org.copy(), {"b": False}) == {
+            "b": False,
+            "d": {"i": 1, "t": (1, 2)},
+        }
+        assert deep_update(d_org.copy(), {"b": {"class": "c"}}) == {
+            "b": {"class": "c"},
+            "d": {"i": 1, "t": (1, 2)},
+        }
+
+        d_new = {
+            "user_mapping": {
+                "*": {
+                    "tester": {
+                        "password": "secret",
+                        "description": "",
+                        "roles": [],
+                    },
+                    "tester2": {
+                        "password": "secret2",
+                        "description": "",
+                        "roles": [],
+                    },
+                }
+            }
+        }
+        assert deep_update({"user_mapping": {}}, d_new) == d_new
+
 
 class LoggerTest(unittest.TestCase):
     """Test configurable logging."""

tox.ini

@@ -138,7 +138,7 @@ deps =
     readthedocs-sphinx-search
 commands =
     # http://www.sphinx-doc.org/en/master/man/sphinx-build.html
-    sphinx-build -b html sphinx sphinx-build
+    sphinx-build -b html source build
 
 
 [testenv:stressor]

wsgidav/dav_provider.py

@@ -1514,15 +1514,17 @@ def set_share_path(self, share_path):
         self.share_path = share_path
 
     def set_lock_manager(self, lock_manager):
-        assert not lock_manager or hasattr(
-            lock_manager, "check_write_permission"
-        ), "Must be compatible with wsgidav.lock_manager.LockManager"
+        if lock_manager and not hasattr(lock_manager, "check_write_permission"):
+            raise ValueError(
+                "Must be compatible with wsgidav.lock_man.lock_manager.LockManager"
+            )
         self.lock_manager = lock_manager
 
     def set_prop_manager(self, prop_manager):
-        assert not prop_manager or hasattr(
-            prop_manager, "copy_properties"
-        ), "Must be compatible with wsgidav.prop_man.property_manager.PropertyManager"
+        if prop_manager and not hasattr(prop_manager, "copy_properties"):
+            raise ValueError(
+                "Must be compatible with wsgidav.prop_man.property_manager.PropertyManager"
+            )
         self.prop_manager = prop_manager
 
     def ref_url_to_path(self, ref_url):

wsgidav/interfaces/lockmanagerinterface.py

@@ -5,14 +5,14 @@ class LockManagerInterface:
     """
     +----------------------------------------------------------------------+
     | TODO: document this interface                                        |
-    | For now, see wsgidav.lock_manager instead                            |
+    | For now, see wsgidav.lock_man.lock_manager instead                            |
     +----------------------------------------------------------------------+
 
     This class is an interface for a LockManager.
     Implementations for the lock manager in WsgiDAV include::
 
-        wsgidav.lock_manager.LockManager
-        wsgidav.lock_manager.ShelveLockManager
+        wsgidav.lock_man.lock_manager.LockManager
+        wsgidav.lock_man.lock_manager.ShelveLockManager
 
     All methods must be implemented.