[Resolves #1187] Add configuration for inheritance strategies for lis…

…t and dict configs (#1187)

docs/_source/docs/stack_config.rst

@@ -15,13 +15,16 @@ particular Stack. The available keys are listed below.
 
 -  `template_path`_ or `template`_ *(required)*
 -  `dependencies`_ *(optional)*
+-  `dependencies_inheritance`_ *(optional)*
 -  `hooks`_ *(optional)*
+-  `hooks_inheritance`_ *(optional)*
 -  `ignore`_ *(optional)*
 -  `notifications`_ *(optional)*
 -  `obsolete`_ *(optional)*
 -  `on_failure`_ *(optional)*
 -  `disable_rollback`_ *(optional)*
 -  `parameters`_ *(optional)*
+-  `parameters_inheritance`_ *(optional)*
 -  `protected`_ *(optional)*
 -  `role_arn`_ *(optional)*
 -  `cloudformation_service_role`_ *(optional)*
@@ -30,8 +33,10 @@ particular Stack. The available keys are listed below.
 -  `iam_role_session_duration`_ *(optional)*
 -  `sceptre_role_session_duration`_ *(optional)*
 -  `sceptre_user_data`_ *(optional)*
+-  `sceptre_user_data_inheritance`_ *(optional)*
 -  `stack_name`_ *(optional)*
 -  `stack_tags`_ *(optional)*
+-  `stack_tags_inheritance`_ *(optional)*
 -  `stack_timeout`_ *(optional)*
 
 It is not possible to define both `template_path`_ and `template`_. If you do so,
@@ -97,6 +102,16 @@ and that Stack need not be added as an explicit dependency.
    situation by either (a) setting those ``dependencies`` on individual Stack Configs rather than the
    the StackGroup Config, or (b) moving those dependency stacks outside of the StackGroup.
 
+dependencies_inheritance
+~~~~~~~~~~~~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This configuration will override the default inheritance strategy of `dependencies`.
+
+Valid values for this config are: ``list_join``, and ``child_wins``.
+
 hooks
 ~~~~~
 * Resolvable: No (but you can use resolvers _in_ hook arguments!)
@@ -106,6 +121,16 @@ hooks
 A list of arbitrary shell or Python commands or scripts to run. Find out more
 in the :doc:`hooks` section.
 
+hooks_inheritance
+~~~~~~~~~~~~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This configuration will override the default inheritance strategy of `hooks`.
+
+Valid values for this config are: ``list_join``, and ``child_wins``.
+
 ignore
 ~~~~~~
 * Resolvable: No
@@ -292,6 +317,16 @@ Example:
        - !stack_output security-groups.yaml::BaseSecurityGroupId
        - !file_contents /file/with/security_group_id.txt
 
+parameters_inheritance
+~~~~~~~~~~~~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This configuration will override the default inheritance strategy of `parameters`.
+
+Valid values for this config are: ``dict_merge``, and ``child_wins``.
+
 protected
 ~~~~~~~~~
 * Resolvable: No
@@ -397,6 +432,16 @@ Represents data to be passed to the ``sceptre_handler(sceptre_user_data)``
 function in Python templates or accessible under ``sceptre_user_data`` variable
 key within Jinja2 templates.
 
+sceptre_user_data_inheritance
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This configuration will override the default inheritance strategy of `sceptre_user_data`.
+
+Valid values for this config are: ``dict_merge``, and ``child_wins``.
+
 stack_name
 ~~~~~~~~~~
 * Resolvable: No
@@ -440,6 +485,16 @@ stack_tags
 
 A dictionary of `CloudFormation Tags`_ to be applied to the Stack.
 
+stack_tags_inheritance
+~~~~~~~~~~~~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This configuration will override the default inheritance strategy of `stack_tags`.
+
+Valid values for this config are: ``dict_merge``, and ``child_wins``.
+
 stack_timeout
 ~~~~~~~~~~~~~
 * Resolvable: No

docs/_source/docs/stack_group_config.rst

@@ -175,7 +175,7 @@ configurations should be defined at a lower directory level.
 
 YAML files that define configuration settings with conflicting keys, the child
 configuration file will usually take precedence (see the specific config keys as documented
-for the inheritance strategy employed).
+for the inheritance strategy employed and `Inheritance Strategy Override`_).
 
 In the above directory structure, ``config/config.yaml`` will be read in first,
 followed by ``config/account-1/config.yaml``, followed by
@@ -185,6 +185,17 @@ For example, if you wanted the ``dev`` StackGroup to build to a different
 region, this setting could be specified in the ``config/dev/config.yaml`` file,
 and would only be applied to builds in the ``dev`` StackGroup.
 
+Inheritance Strategy Override
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The inheritance strategy of some properties may be overridden by the stack group config.
+
+Strategy options:
+
+ - ``list_join``: Child configs are appended to parent configs.
+ - ``dict_merge``: Child configs will be merged with parent configs, the child keys taking precedence.
+ - ``child_wins``: Overrides parent if set.
+
 .. _setting_dependencies_for_stack_groups:
 
 Setting Dependencies for StackGroups

sceptre/config/reader.py

@@ -39,16 +39,30 @@
 
 ConfigAttributes = collections.namedtuple("Attributes", "required optional")
 
+
+CONFIG_MERGE_STRATEGY_OVERRIDES = {
+    "dependencies": strategies.LIST_STRATEGIES,
+    "hooks": strategies.LIST_STRATEGIES,
+    "notifications": strategies.LIST_STRATEGIES,
+    "parameters": strategies.DICT_STRATEGIES,
+    "sceptre_user_data": strategies.DICT_STRATEGIES,
+    "stack_tags": strategies.DICT_STRATEGIES,
+}
+
 CONFIG_MERGE_STRATEGIES = {
     "dependencies": strategies.list_join,
+    "dependencies_inheritance": strategies.child_or_parent,
     "hooks": strategies.child_wins,
+    "hooks_inheritance": strategies.child_or_parent,
     "iam_role": strategies.child_wins,
     "sceptre_role": strategies.child_wins,
     "iam_role_session_duration": strategies.child_wins,
     "sceptre_role_session_duration": strategies.child_wins,
     "notifications": strategies.child_wins,
+    "notifications_inheritance": strategies.child_or_parent,
     "on_failure": strategies.child_wins,
     "parameters": strategies.child_wins,
+    "parameters_inheritance": strategies.child_or_parent,
     "profile": strategies.child_wins,
     "project_code": strategies.child_wins,
     "protect": strategies.child_wins,
@@ -57,8 +71,10 @@
     "role_arn": strategies.child_wins,
     "cloudformation_service_role": strategies.child_wins,
     "sceptre_user_data": strategies.child_wins,
+    "sceptre_user_data_inheritance": strategies.child_or_parent,
     "stack_name": strategies.child_wins,
     "stack_tags": strategies.child_wins,
+    "stack_tags_inheritance": strategies.child_or_parent,
     "stack_timeout": strategies.child_wins,
     "template_bucket_name": strategies.child_wins,
     "template_key_value": strategies.child_wins,
@@ -68,6 +84,7 @@
     "obsolete": strategies.child_wins,
 }
 
+
 STACK_GROUP_CONFIG_ATTRIBUTES = ConfigAttributes(
     {"project_code", "region"},
     {
@@ -84,21 +101,26 @@
         "template_path",
         "template",
         "dependencies",
+        "dependencies_inheritance",
         "hooks",
+        "hooks_inheritance",
         "iam_role",
         "sceptre_role",
         "iam_role_session_duration",
         "sceptre_role_session_duration",
         "notifications",
         "on_failure",
         "parameters",
+        "parameters_inheritance",
         "profile",
         "protect",
         "role_arn",
         "cloudformation_service_role",
         "sceptre_user_data",
+        "sceptre_user_data_inheritance",
         "stack_name",
         "stack_tags",
+        "stack_tags_inheritance",
         "stack_timeout",
     },
 )
@@ -352,11 +374,8 @@ def _read(self, rel_path, base_config=None):
 
         # Parse and read in the config files.
         this_config = self._recursive_read(directory_path, filename, config)
-
-        if "dependencies" in config or "dependencies" in this_config:
-            this_config["dependencies"] = CONFIG_MERGE_STRATEGIES["dependencies"](
-                this_config.get("dependencies"), config.get("dependencies")
-            )
+        # Apply merge strategies with the config that includes base_config values.
+        this_config.update(self._get_merge_with_stratgies(config, this_config))
         config.update(this_config)
 
         self._check_version(config)
@@ -395,16 +414,43 @@ def _recursive_read(
 
         # Read config file and overwrite inherited properties
         child_config = self._render(directory_path, filename, config_group) or {}
+        child_config.update(self._get_merge_with_stratgies(config, child_config))
+        config.update(child_config)
+        return config
 
-        for config_key, strategy in CONFIG_MERGE_STRATEGIES.items():
-            value = strategy(config.get(config_key), child_config.get(config_key))
+    def _get_merge_with_stratgies(self, left: dict, right: dict) -> dict:
+        """
+        Returns a new dict with only the merge values of the two inputs, using the
+        merge strategies defined for each key.
+        """
+        merge = {}
+
+        # Then apply the merge strategies to each item
+        for config_key, default_strategy in CONFIG_MERGE_STRATEGIES.items():
+            strategy = default_strategy
+            override_key = f"{config_key}_inheritance"
+            if override_key in CONFIG_MERGE_STRATEGIES:
+                name = CONFIG_MERGE_STRATEGIES[override_key](
+                    left.get(override_key), right.get(override_key)
+                )
+                if not name:
+                    pass
+                elif name not in strategies.SELECTABLE:
+                    raise SceptreException(
+                        f"{name!r} is not a valid inheritance strategy"
+                    )
+                elif name not in CONFIG_MERGE_STRATEGY_OVERRIDES[config_key]:
+                    raise SceptreException(
+                        f"{name!r} is not a valid inheritance strategy for {config_key}"
+                    )
+                else:
+                    strategy = strategies.SELECTABLE[name]
 
+            value = strategy(left.get(config_key), right.get(config_key))
             if value:
-                child_config[config_key] = value
-
-        config.update(child_config)
+                merge[config_key] = value
 
-        return config
+        return merge
 
     def _render(self, directory_path, basename, stack_group_config):
         """

sceptre/config/strategies.py

@@ -71,3 +71,25 @@ def child_wins(a, b):
     :returns: b
     """
     return b
+
+
+def child_or_parent(a, b):
+    """
+    Returns second arg if is not empty, else the first.
+
+    :param a: An object.
+    :type a: object
+    :param b: An object.
+    :type b: object
+    :returns: b
+    """
+    return b or a
+
+
+SELECTABLE = {
+    "list_join": list_join,
+    "dict_merge": dict_merge,
+    "child_wins": child_wins,
+}
+LIST_STRATEGIES = ["list_join", "child_wins"]
+DICT_STRATEGIES = ["dict_merge", "child_wins"]