Skip to content

v1.42.0
Compare
Choose a tag to compare
@aknysh aknysh released this 28 Jul 13:56
· 78 commits to master since this release
v1.42.0
6eef34e
This commit was created on GitHub.com and signed with GitHub鈥檚 verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

what

why

  • Sprig functions in Atmos Go templates in imports provide over 70 useful functions for Go鈥檚 template language

  • Atmos component validation (using atmos validate component command and OPA policies in YAML) now has access to all the outputs that the atmos describe component generates (before, some recently added output were not present to the OPA policies for validation)

  • Update and improve component dependencies calculation


The command atmos describe component now outputs three different dependency-related attributes:

  • imports - a list of all imports in the Atmos stack (this shows all imports in the stack, related to the component and not)

  • deps_all - a list of all component stack dependencies (stack config files where the component settings are defined, either inline or via imports)

  • deps - a list of component stack dependencies where the final values of all component configurations are defined
    (after the deep-merging and processing all the inheritance chains and all the base components)

deps:
- catalog/terraform/base-component-1
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-3
- catalog/terraform/spacelift-and-backend-override-1
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
deps_all:
- catalog/terraform/base-component-1
- catalog/terraform/base-component-3
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-3
- catalog/terraform/spacelift-and-backend-override-1
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
- orgs/cp/tenant1/test1/us-east-2
imports:
- catalog/helmfile/echo-server
- catalog/helmfile/infra-server
- catalog/helmfile/infra-server-override
- catalog/terraform/base-component-1
- catalog/terraform/base-component-2
- catalog/terraform/base-component-3
- catalog/terraform/base-component-4
- catalog/terraform/derived-component-1
- catalog/terraform/derived-component-2
- catalog/terraform/derived-component-3
- catalog/terraform/mixins/test-1
- catalog/terraform/mixins/test-2
- catalog/terraform/services/service-1
- catalog/terraform/services/service-1-override
- catalog/terraform/services/service-1-override-2
- catalog/terraform/services/service-2
- catalog/terraform/services/service-2-override
- catalog/terraform/services/service-2-override-2
- catalog/terraform/services/top-level-service-1
- catalog/terraform/services/top-level-service-2
- catalog/terraform/spacelift-and-backend-override-1
- catalog/terraform/test-component
- catalog/terraform/test-component-2
- catalog/terraform/test-component-override
- catalog/terraform/test-component-override-2
- catalog/terraform/test-component-override-3
- catalog/terraform/top-level-component1
- catalog/terraform/top-level-component2
- catalog/terraform/vpc
- mixins/region/us-east-2
- mixins/stage/test1
- orgs/cp/_defaults
- orgs/cp/tenant1/_defaults
- orgs/cp/tenant1/test1/_defaults

The difference between the imports, deps_all and deps outputs is as follows:

  • imports shows all imports in the stack for all components. This can be useful in GitHub actions (or other CI/CD systems) and in OPA validation policies to check whether an import is allowed in the stack or not

  • deps_all shows all component stack dependencies (imports and root-level stacks) where any configuration for the component is present. This also can be useful in CI/CD systems and OPA validation policies to check whether a user or a team is allowed to import a particular config file for the component in the stack

  • deps shows all the component stack dependencies where the FINAL values from all the component sections are defined (after the deep-merging and processing all the inheritance chains and all the base components). This is useful in CI/CD systems (e.g. Spacelift) to detect all the affected files that the component depends on (and trigger the component's stack if any of the files is affected). deps is usually a much smaller list than deps_all and can differ from it in the following ways:

    • The component can inherit configurations from many base components, see Component Inheritance, and import those base component configurations

    • The component can override all the default variables from the base components, and the final values are not dependent on the base components config anymore. For example, derived-component-3 import the base component base-component-4 configuration, inherits from it, and overrides all the variables:

    # Import the base component config
import:
  - catalog/terraform/base-component-4

components:
  terraform:
    derived-component-3:
      metadata:
        component: "test/test-component"  # Point to the Terraform component
        inherits:
          # Inherit all the values from the base component
          - base-component-4
      vars:
        # Override all the variables from the base component

    • Atmos detects that and does not include the base component base-component-4 config file into the deps output since the component does not directly depend on it (all values are coming from derived-component-3). This will help, for example, to not trigger the component's Spacelift stack unnecessary if only the base-component-4 changes, preventing the unrelated stack runs

    • In the above case, the deps_all output will include both derived-component-3 and base-component-4, but the deps output will only include derived-component-3


NOTE: Before this update, we had only the deps output (and used it in Spacelift policies to detect affected files and trigger the stacks), which was somewhere in between the new deps and deps_all outputs. It showed too much and too little at the same time:

  • After we introduced multiple and multi-level component inheritance (see https://atmos.tools/core-concepts/components/inheritance), the deps output was not updated to check for the base components that were completely overridden by the derived component, and all those base components were included in the output, triggering Spacelift stacks unnecessary

  • Atmos did not check the cases where only imports were used to import the component config from the catalog w/o having anything related to the component defined inline in other config files. Atmos was including all those YAML config files into the deps output, and if they were updated (e.g. by importing a completely different component into the stack), triggering the component's Spacelift stack unnecessary. Now all of that is added only to the deps_all output having the deps output much smaller

  • Now we are using the Sources of component variables and other sections to find out all the dependencies for the deps and deps_all outputs

Assets 17