Add porting guide and documentation for changes to argument spec validation

docs/docsite/rst/porting_guides/porting_guide_core_2.11.rst

@@ -29,22 +29,71 @@ Command Line
 * The ``ansible-galaxy login`` command has been removed, as the underlying API it used for GitHub auth has been shut down. Publishing roles or collections to Galaxy with ``ansible-galaxy`` now requires that a Galaxy API token be passed to the CLI using a token file (default location ``~/.ansible/galaxy_token``) or (insecurely) with the ``--token`` argument to ``ansible-galaxy``.
 
 
-Other:
+Deprecated
+==========
+
+The constant ``ansible.module_utils.basic._CHECK_ARGUMENT_TYPES_DISPATCHER`` is deprecated. Use :const:`ansible.module_utils.common.parameters.DEFAULT_TYPE_VALIDATORS` instead.
+
+
+Breaking Changes
+================
+
+Changes to ``AnsibleModule``
+----------------------------
+
+With the move to :class:`ArgumentSpecValidator <ansible.module_utils.common.arg_spec.ArgumentSpecValidator>` for performing argument spec validation, the following private methods in :class:`AnsibleModule <ansible.module_utils.basic.AnsibleModule>` have been removed:
+
+    - ``_check_argument_types()``
+    - ``_check_argument_values()``
+    - ``_check_arguments()``
+    - ``_check_mutually_exclusive()`` --> :func:`ansible.module_utils.common.validation.check_mutually_exclusive`
+    - ``_check_required_arguments()`` --> :func:`ansible.module_utils.common.validation.check_required_arguments`
+    - ``_check_required_by()`` --> :func:`ansible.module_utils.common.validation.check_required_by`
+    - ``_check_required_if()`` --> :func:`ansible.module_utils.common.validation.check_required_if`
+    - ``_check_required_one_of()`` --> :func:`ansible.module_utils.common.validation.check_required_one_of`
+    - ``_check_required_together()`` --> :func:`ansible.module_utils.common.validation.check_required_together`
+    - ``_check_type_bits()`` --> :func:`ansible.module_utils.common.validation.check_type_bits`
+    - ``_check_type_bool()`` --> :func:`ansible.module_utils.common.validation.check_type_bool`
+    - ``_check_type_bytes()`` --> :func:`ansible.module_utils.common.validation.check_type_bytes`
+    - ``_check_type_dict()`` --> :func:`ansible.module_utils.common.validation.check_type_dict`
+    - ``_check_type_float()`` --> :func:`ansible.module_utils.common.validation.check_type_float`
+    - ``_check_type_int()`` --> :func:`ansible.module_utils.common.validation.check_type_int`
+    - ``_check_type_jsonarg()`` --> :func:`ansible.module_utils.common.validation.check_type_jsonarg`
+    - ``_check_type_list()`` --> :func:`ansible.module_utils.common.validation.check_type_list`
+    - ``_check_type_path()`` --> :func:`ansible.module_utils.common.validation.check_type_path`
+    - ``_check_type_raw()`` --> :func:`ansible.module_utils.common.validation.check_type_raw`
+    - ``_check_type_str()`` --> :func:`ansible.module_utils.common.validation.check_type_str`
+    - ``_count_terms()`` --> :func:`ansible.module_utils.common.validation.count_terms`
+    - ``_get_wanted_type()``
+    - ``_handle_aliases()``
+    - ``_handle_no_log_values()``
+    - ``_handle_options()``
+    - ``_set_defaults()``
+    - ``_set_fallbacks()``
+
+Modules or plugins using these private methods should use the public functions in :mod:`ansible.module_utils.common.validation` or :meth:`ArgumentSpecValidator.validate() <ansible.module_utils.common.arg_spec.ArgumentSpecValidator.validate>` if no public function was listed above.
+
+
+Changes to :mod:`ansible.module_utils.common.parameters`
+--------------------------------------------------------
+
+The following functions in :mod:`ansible.module_utils.common.parameters` are now private and should not be used directly. Use :meth:`ArgumentSpecValidator.validate() <ansible.module_utils.common.arg_spec.ArgumentSpecValidator.validate>` instead.
+
+    - ``list_no_log_values``
+    - ``list_deprecations``
+    - ``handle_aliases``
+
+
+Other
 ======
 
 * **Upgrading**: If upgrading from ``ansible < 2.10`` or from ``ansible-base`` and using pip, you must ``pip uninstall ansible`` or ``pip uninstall ansible-base`` before installing ``ansible-core`` to avoid conflicts.
 * Python 3.8 on the controller node is a soft requirement for this release. ``ansible-core`` 2.11 still works with the same versions of Python that ``ansible-base`` 2.10 worked with, however 2.11 emits a warning when running on a controller node with a Python version less than 3.8. This warning can be disabled by setting ``ANSIBLE_CONTROLLER_PYTHON_WARNING=False`` in your environment. ``ansible-core`` 2.12 will require Python 3.8 or greater.
-* The configuration system now validates the ``choices`` field, so any settings that violate it and were ignored in 2.10 cause an error in 2.11. For example, `ANSIBLE_COLLECTIONS_ON_ANSIBLE_VERSION_MISMATCH=0` now causes an error (valid choices are ``ignore``, ``warn`` or ``error``).
+* The configuration system now validates the ``choices`` field, so any settings that violate it and were ignored in 2.10 cause an error in 2.11. For example, ``ANSIBLE_COLLECTIONS_ON_ANSIBLE_VERSION_MISMATCH=0`` now causes an error (valid choices are ``ignore``, ``warn`` or ``error``).
 * The ``ansible-galaxy`` command now uses ``resolvelib`` for resolving dependencies. In most cases this should not make a user-facing difference beyond being more performant, but we note it here for posterity and completeness.
 * If you import Python ``module_utils`` into any modules you maintain, you may now mark the import as optional during the module payload build by wrapping the ``import`` statement in a ``try`` or ``if`` block. This allows modules to use ``module_utils`` that may not be present in all versions of Ansible or a collection, and to perform arbitrary recovery or fallback actions during module runtime.
 
 
-Deprecated
-==========
-
-No notable changes
-
-
 Modules
 =======
 

docs/docsite/rst/reference_appendices/module_utils.rst

@@ -25,3 +25,50 @@ To use this functionality, include ``import ansible.module_utils.basic`` in your
 
 .. automodule:: ansible.module_utils.basic
    :members:
+
+
+Argument Spec
+---------------------
+
+Classes and functions for validating parameters against an argument spec.
+
+ArgumentSpecValidator
+=====================
+
+.. autoclass:: ansible.module_utils.common.arg_spec.ArgumentSpecValidator
+   :members:
+
+ValidationResult
+================
+
+.. autoclass:: ansible.module_utils.common.arg_spec.ValidationResult
+   :members:
+   :member-order: bysource
+   :private-members: _no_log_values  # This only works in sphinx >= 3.2. Otherwise it shows all private members with doc strings.
+
+Parameters
+==========
+
+.. automodule:: ansible.module_utils.common.parameters
+   :members:
+
+   .. py:data:: DEFAULT_TYPE_VALIDATORS
+
+     :class:`dict` of type names, such as ``'str'``, and the default function
+     used to check that type, :func:`~ansible.module_utils.common.validation.check_type_str` in this case.
+
+Validation
+==========
+
+Standalone functions for validating various parameter types.
+
+.. automodule:: ansible.module_utils.common.validation
+   :members:
+
+
+Errors
+------
+
+.. automodule:: ansible.module_utils.errors
+   :members:
+   :member-order: bysource

lib/ansible/module_utils/common/arg_spec.py

@@ -50,60 +50,53 @@
 class ValidationResult:
     """Result of argument spec validation.
 
-    :param parameters: Terms to be validated and coerced to the correct type.
-    :type parameters: dict
-
+    This is the object returned by :func:`ArgumentSpecValidator.validate()
+    <ansible.module_utils.common.arg_spec.ArgumentSpecValidator.validate()>`
+    containing the validated parameters and any errors.
     """
 
     def __init__(self, parameters):
+        """
+        :arg parameters: Terms to be validated and coerced to the correct type.
+        :type parameters: dict
+        """
         self._no_log_values = set()
+        """:class:`set` of values marked as ``no_log`` in the argument spec. This
+        is a temporary holding place for these values and may move in the future.
+        """
+
         self._unsupported_parameters = set()
         self._validated_parameters = deepcopy(parameters)
         self._deprecations = []
         self._warnings = []
         self.errors = AnsibleValidationErrorMultiple()
+        """
+        :class:`~ansible.module_utils.errors.AnsibleValidationErrorMultiple` containing all
+        :class:`~ansible.module_utils.errors.AnsibleValidationError` objects if there were
+        any failures during validation.
+        """
 
     @property
     def validated_parameters(self):
+        """Validated and coerced parameters."""
         return self._validated_parameters
 
     @property
     def unsupported_parameters(self):
+        """:class:`set` of unsupported parameter names."""
         return self._unsupported_parameters
 
     @property
     def error_messages(self):
+        """:class:`list` of all error messages from each exception in :attr:`errors`."""
         return self.errors.messages
 
 
 class ArgumentSpecValidator:
     """Argument spec validation class
 
     Creates a validator based on the ``argument_spec`` that can be used to
-    validate a number of parameters using the ``validate()`` method.
-
-    :param argument_spec: Specification of valid parameters and their type. May
-        include nested argument specs.
-    :type argument_spec: dict
-
-    :param mutually_exclusive: List or list of lists of terms that should not
-        be provided together.
-    :type mutually_exclusive: list, optional
-
-    :param required_together: List of lists of terms that are required together.
-    :type required_together: list, optional
-
-    :param required_one_of: List of lists of terms, one of which in each list
-        is required.
-    :type required_one_of: list, optional
-
-    :param required_if: List of lists of ``[parameter, value, [parameters]]`` where
-        one of [parameters] is required if ``parameter`` == ``value``.
-    :type required_if: list, optional
-
-    :param required_by: Dictionary of parameter names that contain a list of
-        parameters required by each key in the dictionary.
-    :type required_by: dict, optional
+    validate a number of parameters using the :meth:`validate` method.
     """
 
     def __init__(self, argument_spec,
@@ -114,6 +107,31 @@ def __init__(self, argument_spec,
                  required_by=None,
                  ):
 
+        """
+        :arg argument_spec: Specification of valid parameters and their type. May
+            include nested argument specs.
+        :type argument_spec: dict[str, dict]
+
+        :kwarg mutually_exclusive: List or list of lists of terms that should not
+            be provided together.
+        :type mutually_exclusive: list[str] or list[list[str]]
+
+        :kwarg required_together: List of lists of terms that are required together.
+        :type required_together: list[list[str]]
+
+        :kwarg required_one_of: List of lists of terms, one of which in each list
+            is required.
+        :type required_one_of: list[list[str]]
+
+        :kwarg required_if: List of lists of ``[parameter, value, [parameters]]`` where
+            one of ``[parameters]`` is required if ``parameter == value``.
+        :type required_if: list
+
+        :kwarg required_by: Dictionary of parameter names that contain a list of
+            parameters required by each key in the dictionary.
+        :type required_by: dict[str, list[str]]
+        """
+
         self._mutually_exclusive = mutually_exclusive
         self._required_together = required_together
         self._required_one_of = required_one_of
@@ -130,30 +148,37 @@ def __init__(self, argument_spec,
                 self._valid_parameter_names.update([key])
 
     def validate(self, parameters, *args, **kwargs):
-        """Validate module parameters against argument spec. Returns a
-        ValidationResult object.
+        """Validate ``parameters`` against argument spec.
 
-        Error messages in the ValidationResult may contain no_log values and should be
-        sanitized before logging or displaying.
+        Error messages in the :class:`ValidationResult` may contain no_log values and should be
+        sanitized with :func:`~ansible.module_utils.common.parameters.sanitize_keys` before logging or displaying.
 
-        :Example:
+        :arg parameters: Parameters to validate against the argument spec
+        :type parameters: dict[str, dict]
 
-        validator = ArgumentSpecValidator(argument_spec)
-        result = validator.validate(parameters)
+        :return: :class:`ValidationResult` containing validated parameters.
 
-        if result.error_messages:
-            sys.exit("Validation failed: {0}".format(", ".join(result.error_messages))
+        :Simple Example:
 
-        valid_params = result.validated_parameters
+            .. code-block:: text
 
-        :param argument_spec: Specification of parameters, type, and valid values
-        :type argument_spec: dict
+                argument_spec = {
+                    'name': {'type': 'str'},
+                    'age': {'type': 'int'},
+                }
 
-        :param parameters: Parameters provided to the role
-        :type parameters: dict
+                parameters = {
+                    'name': 'bo',
+                    'age': '42',
+                }
+
+                validator = ArgumentSpecValidator(argument_spec)
+                result = validator.validate(parameters)
+
+                if result.error_messages:
+                    sys.exit("Validation failed: {0}".format(", ".join(result.error_messages))
 
-        :return: Object containing validated parameters.
-        :rtype: ValidationResult
+                valid_params = result.validated_parameters
         """
 
         result = ValidationResult(parameters)
@@ -238,6 +263,12 @@ def validate(self, parameters, *args, **kwargs):
 
 
 class ModuleArgumentSpecValidator(ArgumentSpecValidator):
+    """Argument spec validation class used by :class:`AnsibleModule`.
+
+    This is not meant to be used outside of :class:`AnsibleModule`. Use
+    :class:`ArgumentSpecValidator` instead.
+    """
+
     def __init__(self, *args, **kwargs):
         super(ModuleArgumentSpecValidator, self).__init__(*args, **kwargs)