[Resolve #1252] Supporting resolvers in Hook and Resolver arguments, with new !substitute, !join, !split, and !select resolvers!

docs/_source/docs/resolvers.rst

@@ -5,6 +5,9 @@ Sceptre implements resolvers, which can be used to resolve a value of a
 CloudFormation ``parameter`` or ``sceptre_user_data`` value at runtime. This is
 most commonly used to chain the outputs of one Stack to the inputs of another.
 
+You can use resolvers with any resolvable property on a StackConfig, as well as in the arguments
+of hooks and other resolvers.
+
 If required, users can create their own resolvers, as described in the section
 on `Custom Resolvers`_.
 
@@ -52,6 +55,27 @@ file_contents
 
 **deprecated**: Consider using the `file`_ resolver instead.
 
+join
+~~~~
+
+This resolver allows you to join multiple strings together to form a single string. This is great
+for combining the outputs of multiple resolvers. This resolver works just like CloudFormation's
+``!Join`` intrinsic function.
+
+The argument for this resolver should be a list with two elements: (1) A string to join the elements
+on and (2) a list of items to join.
+
+Example:
+
+.. code-block:: yaml
+
+   parameters:
+     BaseUrl: !join
+       - ":"
+       - - !stack_output my/app/stack.yaml::HostName
+         - !stack_output my/other/stack.yaml::Port
+
+
 no_value
 ~~~~~~~~
 
@@ -81,6 +105,49 @@ Refer to `sceptre-resolver-cmd <https://github.com/Sceptre/sceptre-resolver-cmd/
 
 .. _stack_attr_resolver:
 
+select
+~~~~~~
+
+This resolver allows you to select a specific index of a list of items. This is great for combining
+with the ``!split`` resolver to obtain part of a string. This function works almost the same as
+CloudFormation's ``!Select`` intrinsic function, **except you can use this with negative indices to
+select from the end of a list**.
+
+The argument for this resolver should be a list with two elements: (1) A numerical index and (2) a
+list of items to select out of. If the index is negative, it will select from the end of the list.
+For example, "-1" would select the last element and "-2" would select the second-to-last element.
+
+Example:
+
+.. code-block:: yaml
+
+   sceptre_user_data:
+     # This selects the last element after you split the connection string on "/"
+     DatabaseName: !select
+       - -1
+       - !split ["/", !stack_output my/database/stack.yaml::ConnectionString]
+
+split
+~~~~~
+
+This resolver will split a value on a given delimiter string. This is great when combining with the
+``!select`` resolver. This function works the same as CloudFormation's ``!Split`` intrinsic function.
+
+Note: The return value of this resolver is a *list*, not a string. This will not work to set Stack
+configurations that expect strings, but it WILL work to set Stack configurations that expect lists.
+
+The argument for this resolver should be a list with two elements: (1) The delimiter to split on and
+(2) a string to split.
+
+Example:
+
+.. code-block:: yaml
+
+   notifications: !split
+     - ";"
+     - !stack_output my/sns/topics.yaml::SemicolonDelimitedArns
+
+
 stack_attr
 ~~~~~~~~~~
 
@@ -181,6 +248,32 @@ Example:
    parameters:
      VpcIdParameter: !stack_output_external prj-network-vpc::VpcIdOutput prod
 
+
+substitute
+~~~~~~~~~~
+
+This resolver allows you to create a string using Python string format syntax. This functions as a
+great way to combine together a number of resolver outputs into a single string. This functions very
+similarly to Cloudformation's ``!Sub`` intrinsic function.
+
+The argument to this resolver should be a two-element list: (1) Is the format string, using
+curly-brace templates to indicate variables, and (2) a dictionary where the keys are the format
+string's variable names and the values are the variable values.
+
+Example:
+
+.. code-block:: yaml
+
+   parameters:
+     ConnectionString: !substitute
+       - "postgres://{username}:{password}@{hostname}:{port}/{database}"
+       - username: {{ var.username }}
+         password: !ssm /my/ssm/password
+         hostname: !stack_output my/database/stack.yaml::HostName
+         port: !stack_output my/database/stack.yaml::Port
+         database: {{var.database}}
+
+
 Custom Resolvers
 ----------------
 
@@ -306,18 +399,22 @@ For details on calling AWS services or invoking AWS-related third party tools in
 
 Resolver arguments
 ^^^^^^^^^^^^^^^^^^
-Resolver arguments can be a simple string or a complex data structure.
+Resolver arguments can be a simple string or a complex data structure. You can even use
+other resolvers in the arguments to resolvers! (Note: Other resolvers can only be passed in
+arguments when they're passed in lists and dicts.)
 
 .. code-block:: yaml
 
    template:
      path: <...>
      type: <...>
-    parameters:
-      Param1: !ssm "/dev/DbPassword"
-      Param2: !ssm {"name": "/dev/DbPassword"}
-      Param3: !ssm
-        name: "/dev/DbPassword"
+   parameters:
+     Param1: !ssm "/dev/DbPassword"
+     Param2: !ssm {"name": "/dev/DbPassword"}
+     Param3: !ssm
+       name: "/dev/DbPassword"
+     Param4: !ssm
+       name: !stack_output my/other/stack.yaml::MySsmParameterName
 
 .. _Custom Resolvers: #custom-resolvers
 .. _this is great place to start: https://docs.python.org/3/distributing/

docs/_source/docs/stack_config.rst

@@ -99,7 +99,7 @@ and that Stack need not be added as an explicit dependency.
 
 hooks
 ~~~~~
-* Resolvable: No
+* Resolvable: No (but you can use resolvers _in_ hook arguments!)
 * Can be inherited from StackGroup: Yes
 * Inheritance strategy: Overrides parent if set
 
@@ -556,7 +556,8 @@ order:
 A common point of confusion tends to be around the distinction between **"render time"** (phase 3, when
 Jinja logic is applied) and **"resolve time"** (phase 6, when resolvers are resolved). You cannot use
 a resolver via Jinja during "render time", since the resolver won't exist or be ready to use yet. You can,
-however, use Jinja logic to indicate *whether*, *which*, or *how* a resolver is configured.
+however, use Jinja logic to indicate *whether*, *which*, or *how* a resolver is configured. You can
+also use resolvers like ``!substitute`` to interpolate resolved values when Jinja isn't available.
 
 For example, you **can** do something like this:
 
@@ -566,6 +567,11 @@ For example, you **can** do something like this:
      {% if var.use_my_parameter %}
        my_parameter: !stack_output {{ var.stack_name }}::{{ var.output_name }}
      {% endif %}
+       # !substitute will let you combine outputs of multiple resolvers into a single string
+       my_combined_parameter: !substitute
+         - "{fist_part} - {second_part}"
+         - first_part: !stack_output my/stack/name.yaml::Output
+         - second_part: {{ var.second_part }}
 
 Accessing resolved values in other fields
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

sceptre/config/reader.py

@@ -182,7 +182,7 @@ def constructor_factory(node_class):
             :rtype: func
             """
 
-            # This function signture is required by PyYAML
+            # This function signature is required by PyYAML
             def class_constructor(loader, node):
                 return node_class(
                     loader.construct_object(self.resolve_node_tag(loader, node))

sceptre/helpers.py

@@ -2,7 +2,7 @@
 from contextlib import contextmanager
 from datetime import datetime
 from os import sep
-from typing import Optional, Any, List
+from typing import Optional, Any, List, Tuple, Union
 
 import dateutil.parser
 import deprecation
@@ -68,6 +68,28 @@ def func_on_instance(key):
     return attr
 
 
+Container = Union[list, dict]
+Key = Union[str, int]
+
+
+def delete_keys_from_containers(keys_to_delete: List[Tuple[Container, Key]]):
+    """Removes the indicated keys/indexes from their paired containers."""
+    list_items_to_delete = []
+    for container, key in keys_to_delete:
+        if isinstance(container, list):
+            # If it's a list, we want to gather up the items to remove from the list.
+            # We don't want to modify the list length yet, since removals will change all the other
+            # list indexes. Instead, we'll get the actual items at those indexes to remove later.
+            list_items_to_delete.append((container, container[key]))
+        else:
+            del container[key]
+
+    # Finally, now that we have all the items we want to remove the lists, we'll remove those
+    # items specifically from the lists.
+    for containing_list, item in list_items_to_delete:
+        containing_list.remove(item)
+
+
 def normalise_path(path):
     """
     Converts a path to use correct path separator.

sceptre/hooks/__init__.py

@@ -1,39 +1,21 @@
 import abc
 import logging
 from functools import wraps
+from typing import TYPE_CHECKING
 
 from sceptre.helpers import _call_func_on_values
-from sceptre.logging import StackLoggerAdapter
-
-from typing import TYPE_CHECKING, Any
+from sceptre.resolvers import CustomYamlTagBase
 
 if TYPE_CHECKING:
     from sceptre.stack import Stack
 
 
-class Hook(abc.ABC):
+class Hook(CustomYamlTagBase, metaclass=abc.ABCMeta):
     """
-    Hook is an abstract base class that should be inherited by all hooks.
-
-    :param argument: The argument of the hook.
-    :param stack: The associated stack of the hook.
+    Hook is an abstract base class that should be subclassed by all hooks.
     """
 
-    def __init__(self, argument: Any = None, stack: "Stack" = None):
-        self.logger = logging.getLogger(__name__)
-
-        if stack is not None:
-            self.logger = StackLoggerAdapter(self.logger, stack.name)
-
-        self.argument = argument
-        self.stack = stack
-
-    def setup(self):
-        """
-        setup is a method that may be overwritten by inheriting classes. Allows
-        hooks to run so initalisation steps when config is first read.
-        """
-        pass  # pragma: no cover
+    logger = logging.getLogger(__name__)
 
     @abc.abstractmethod
     def run(self):
@@ -43,15 +25,6 @@ def run(self):
         """
         pass  # pragma: no cover
 
-    def clone(self, stack: "Stack") -> "Hook":
-        """
-        Produces a "fresh" copy of the Hook, with the specified stack.
-
-        :param stack: The stack to set on the cloned resolver
-        """
-        clone = type(self)(self.argument, stack)
-        return clone
-
 
 class HookProperty(object):
     """
@@ -84,7 +57,7 @@ def __set__(self, instance: "Stack", value):
         """
 
         def setup(attr, key, value: Hook):
-            attr[key] = clone = value.clone(instance)
+            attr[key] = clone = value.clone_for_stack(instance)
             clone.setup()
 
         _call_func_on_values(setup, value, Hook)