Skip to content

Commit

Permalink
Allow cascading dependencies (#100)
Browse files Browse the repository at this point in the history 
Co-authored-by: dmattia <david@transcend.io>
  • Loading branch information
@dmattia
dmattia and dmattia committed Jan 1, 2021
1 parent 829ada1 commit 9618e2a
Show file tree
Hide file tree
Showing 11 changed files with 244 additions and 48 deletions.
8 changes: 7 additions & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
VERSION=0.12.0
VERSION=0.13.0
PATH_BUILD=build/
FILE_COMMAND=terragrunt-atlantis-config
FILE_ARCH=darwin_amd64
Expand Down Expand Up @@ -31,6 +31,12 @@ build-all: clean
-d=$(PATH_BUILD) \
-build-ldflags "-X main.VERSION=$(VERSION)"

.PHONY: gotestsum
gotestsum:
mkdir -p cmd/test_artifacts
gotestsum
rm -rf cmd/test_artifacts

.PHONY: test
test:
mkdir -p cmd/test_artifacts
Expand Down
59 changes: 23 additions & 36 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ Alternative: Install a stable versions via Homebrew:
brew install transcend-io/tap/terragrunt-atlantis-config
```

This module officially supports golang versions v1.13, v1.14, and v1.15
This module officially supports golang versions v1.13, v1.14, and v1.15, tested on CircleCI with each build
This module also officially supports both Windows and Nix-based file formats, tested on CircleCI with each build

Usage Examples:
Usage Examples (see below sections for all options):

```bash
# From the root of your repo
Expand All @@ -49,21 +50,6 @@ terragrunt-atlantis-config generate --root /some/path/to/your/repo/root

# output to a file
terragrunt-atlantis-config generate --autoplan --output ./atlantis.yaml

# enable auto plan
terragrunt-atlantis-config generate --autoplan

# enable auto merge
terragrunt-atlantis-config generate --automerge

# define the workflow
terragrunt-atlantis-config generate --workflow web --output ./atlantis.yaml

# ignore parent terragrunt configs (those which don't reference a terraform module)
terragrunt-atlantis-config generate --ignore-parent-terragrunt

# Enable the project name creation
terragrunt-atlantis-config generate --create-project-name
```

Finally, check the log output (or your output file) for the YAML.
Expand Down Expand Up @@ -95,10 +81,10 @@ In your `atlantis.yaml` file, you will end up seeing output like:
- autoplan:
enabled: false
when_modified:
- '*.hcl'
- '*.tf*'
- some_extra_dep
- ../../.gitignore
- "*.hcl"
- "*.tf*"
- some_extra_dep
- ../../.gitignore
dir: example-setup/extra_dependency
```

Expand All @@ -112,26 +98,27 @@ If you specify `extra_atlantis_dependencies` in the parent Terragrunt module, th

One way to customize the behavior of this module is through CLI flag values passed in at runtime. These settings will apply to all modules.

| Flag Name | Description | Default Value |
|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------|
| `--autoplan` | The default value for autoplan settings. Can be overriden by locals. | false |
| `--automerge` | Enables the automerge setting for a repo. | false |
| `--ignore-parent-terragrunt` | Ignore parent Terragrunt configs (those which don't reference a terraform module).<br>In most cases, this should be set to `true` | false |
| `--parallel` | Enables `plan`s and `apply`s to happen in parallel. Will typically be used with `--create-workspace` | true |
| `--create-workspace` | Use different auto-generated workspace for each project. Default is use default workspace for everything | false |
| `--create-project-name` | Add different auto-generated name for each project | false |
| `--preserve-workflows` | Preserves workflows from old output files. Useful if you want to define your workflow definitions on the client side | true |
| `--workflow` | Name of the workflow to be customized in the atlantis server. If empty, will be left out of output | "" |
| `--output` | Path of the file where configuration will be generated. Typically, you want a file named "atlantis.yaml". Default is to write to `stdout`. | "" |
| `--root` | Path to the root directory of the git repo you want to build config for. | current directory |
| `--terraform-version` | Default terraform version to specify for all modules. Can be overriden by locals | "" |
| Flag Name | Description | Default Value |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- |
| `--autoplan` | The default value for autoplan settings. Can be overriden by locals. | false |
| `--automerge` | Enables the automerge setting for a repo. | false |
| `--cascade-dependencies` | When true, dependencies will cascade, meaning that a module will be declared to depend not only on its dependencies, but all dependencies of its dependencies all the way down. | true |
| `--ignore-parent-terragrunt` | Ignore parent Terragrunt configs (those which don't reference a terraform module).<br>In most cases, this should be set to `true` | false |
| `--parallel` | Enables `plan`s and `apply`s to happen in parallel. Will typically be used with `--create-workspace` | true |
| `--create-workspace` | Use different auto-generated workspace for each project. Default is use default workspace for everything | false |
| `--create-project-name` | Add different auto-generated name for each project | false |
| `--preserve-workflows` | Preserves workflows from old output files. Useful if you want to define your workflow definitions on the client side | true |
| `--workflow` | Name of the workflow to be customized in the atlantis server. If empty, will be left out of output | "" |
| `--output` | Path of the file where configuration will be generated. Typically, you want a file named "atlantis.yaml". Default is to write to `stdout`. | "" |
| `--root` | Path to the root directory of the git repo you want to build config for. | current directory |
| `--terraform-version` | Default terraform version to specify for all modules. Can be overriden by locals | "" |

## All Locals

Another way to customize the output is to use `locals` values in your terragrunt modules. These can be set in either the parent or child terragrunt modules, and the settings will only affect the current module (or all child modules for parent locals).

| Locals Name | Description | type |
|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
| `atlantis_workflow` | The custom atlantis workflow name to use for a module | string |
| `atlantis_terraform_version` | Allows overriding the `--terraform-version` flag for a single module | string |
| `atlantis_autoplan` | Allows overriding the `--autoplan` flag for a single module | bool |
Expand Down Expand Up @@ -165,7 +152,7 @@ jobs:
id: atlantis_validator
uses: transcend-io/terragrunt-atlantis-config-github-action@v0.0.3
with:
version: v0.12.0
version: v0.13.0
extra_args: '--autoplan --parallel=false
```
Expand Down
80 changes: 75 additions & 5 deletions cmd/generate.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,31 +46,50 @@ func makePathAbsolute(path string, parentPath string) string {
return filepath.Join(parentDir, path)
}

// Parses the terragrunt config at <path> to find all modules it depends on
// Set up a cache for the getDependencies function
type getDependenciesOutput struct {
dependencies []string
err error
}

var getDependenciesCache = make(map[string]getDependenciesOutput)

// Parses the terragrunt config at `path` to find all modules it depends on
func getDependencies(path string, terragruntOptions *options.TerragruntOptions) ([]string, error) {
// Check if this path has already been computed
cachedResult, ok := getDependenciesCache[path]
if ok {
return cachedResult.dependencies, cachedResult.err
}

// if theres no terraform source and we're ignoring parent terragrunt configs
// return nils to indicate we should skip this project
isParent, err := isParentModule(path, terragruntOptions)
if err != nil {
getDependenciesCache[path] = getDependenciesOutput{nil, err}
return nil, err
}
if ignoreParentTerragrunt && isParent {
getDependenciesCache[path] = getDependenciesOutput{nil, nil}
return nil, nil
}

// Parse the HCL file
decodeTypes := []config.PartialDecodeSectionType{
config.DependencyBlock,
config.DependenciesBlock,
config.TerraformBlock,
}

parsedConfig, err := config.PartialParseConfigFile(path, terragruntOptions, nil, decodeTypes)
if err != nil {
getDependenciesCache[path] = getDependenciesOutput{nil, err}
return nil, err
}

// Parse out locals
locals, err := parseLocals(path, terragruntOptions, nil)
if err != nil {
getDependenciesCache[path] = getDependenciesOutput{nil, err}
return nil, err
}

Expand All @@ -82,8 +101,8 @@ func getDependencies(path string, terragruntOptions *options.TerragruntOptions)

// Get deps from `dependencies` and `dependency` blocks
if parsedConfig.Dependencies != nil {
for _, path := range parsedConfig.Dependencies.Paths {
dependencies = append(dependencies, filepath.Join(path, "terragrunt.hcl"))
for _, parsedPaths := range parsedConfig.Dependencies.Paths {
dependencies = append(dependencies, filepath.Join(parsedPaths, "terragrunt.hcl"))
}
}

Expand Down Expand Up @@ -128,7 +147,56 @@ func getDependencies(path string, terragruntOptions *options.TerragruntOptions)
}
}

return nonEmptyDeps, nil
// Recurse to find dependencies of all dependencies
cascadedDeps := []string{}
for _, dep := range nonEmptyDeps {
cascadedDeps = append(cascadedDeps, dep)

// The "cascading" feature is protected by a flag
if !cascadeDependencies {
continue
}

// To find the path to the dependency, we join three things:
// 1. The path to the current module, `path`
// 2. `..`, because `path` includes the `terragrunt.hcl` file extension, while the `dep` path is relative to the folder that file is in
// 3. the relative path from the current module to the dependency, `dep`
depPath := filepath.Join(path, "..", dep)
childDeps, err := getDependencies(depPath, terragruntOptions)
if err != nil {
continue
}

for _, childDep := range childDeps {
// If `childDep` is a relative path, it will be relative to `childDep`, as it is from the nested
// `getDependencies` call on the top level module's dependencies. So here we update any relative
// path to be from the top level module instead.
childDepAbsPath := childDep
if !filepath.IsAbs(childDep) {
childDepAbsPath, err = filepath.Abs(filepath.Join(depPath, "..", childDep))
if err != nil {
getDependenciesCache[path] = getDependenciesOutput{nil, err}
return nil, err
}
}
childDepAbsPath = filepath.ToSlash(childDepAbsPath)

// Ensure we are not adding a duplicate dependency
alreadyExists := false
for _, dep := range cascadedDeps {
if dep == childDepAbsPath {
alreadyExists = true
break
}
}
if !alreadyExists {
cascadedDeps = append(cascadedDeps, childDepAbsPath)
}
}
}

getDependenciesCache[path] = getDependenciesOutput{cascadedDeps, nil}
return cascadedDeps, nil
}

// Creates an AtlantisProject for a directory
Expand Down Expand Up @@ -339,6 +407,7 @@ var defaultTerraformVersion string
var defaultWorkflow string
var outputPath string
var preserveWorkflows bool
var cascadeDependencies bool

// generateCmd represents the generate command
var generateCmd = &cobra.Command{
Expand All @@ -363,6 +432,7 @@ func init() {
generateCmd.PersistentFlags().BoolVar(&createWorkspace, "create-workspace", false, "Use different workspace for each project. Default is use default workspace")
generateCmd.PersistentFlags().BoolVar(&createProjectName, "create-project-name", false, "Add different name for each project. Default is false")
generateCmd.PersistentFlags().BoolVar(&preserveWorkflows, "preserve-workflows", true, "Preserves workflows from old output files. Default is true")
generateCmd.PersistentFlags().BoolVar(&cascadeDependencies, "cascade-dependencies", true, "When true, dependencies will cascade, meaning that a module will be declared to depend not only on its dependencies, but all dependencies of its dependencies all the way down. Default is true")
generateCmd.PersistentFlags().StringVar(&defaultWorkflow, "workflow", "", "Name of the workflow to be customized in the atlantis server. Default is to not set")
generateCmd.PersistentFlags().StringVar(&outputPath, "output", "", "Path of the file where configuration will be generated. Default is not to write to file")
generateCmd.PersistentFlags().StringVar(&gitRoot, "root", pwd, "Path to the root directory of the git repo you want to build config for. Default is current dir")
Expand Down
32 changes: 27 additions & 5 deletions cmd/generate_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ import (
)

// Resets all flag values to their defaults in between tests
func resetDefaultFlags() error {
func resetForRun() error {
pwd, err := os.Getwd()
if err != nil {
return err
}

// reset caches
getDependenciesCache = make(map[string]getDependenciesOutput)

// reset flags
gitRoot = pwd
autoPlan = false
autoMerge = false
cascadeDependencies = true
ignoreParentTerragrunt = false
parallel = true
createWorkspace = false
Expand All @@ -32,7 +38,7 @@ func resetDefaultFlags() error {

// Runs a test, asserting the output produced matches a golden file
func runTest(t *testing.T, goldenFile string, args []string) {
err := resetDefaultFlags()
err := resetForRun()
if err != nil {
t.Error("Failed to reset default flags")
return
Expand All @@ -50,7 +56,7 @@ func runTest(t *testing.T, goldenFile string, args []string) {

content, err := RunWithFlags(filename, allArgs)
if err != nil {
t.Error("Failed to read file")
t.Error(err)
return
}

Expand All @@ -61,7 +67,7 @@ func runTest(t *testing.T, goldenFile string, args []string) {
}

if string(content) != string(goldenContents) {
t.Errorf("Content did not match golden file.\n\nExpected Content: %s\n\nContent: %s", string(goldenContents), string(content))
t.Errorf("Content did not match golden file.\n\nExpected (Golden file) Contents: \n%s\n\nGenerated Content: \n%s", string(goldenContents), string(content))
}
}

Expand Down Expand Up @@ -261,7 +267,7 @@ func TestTerraformVersionConfig(t *testing.T) {
}

func TestPreservingOldWorkflows(t *testing.T) {
err := resetDefaultFlags()
err := resetForRun()
if err != nil {
t.Error("Failed to reset default flags")
return
Expand Down Expand Up @@ -314,3 +320,19 @@ func TestEnablingAutomerge(t *testing.T) {
"--automerge",
})
}

func TestChainedDependencies(t *testing.T) {
runTest(t, filepath.Join("golden", "chained_dependency.yaml"), []string{
"--root",
filepath.Join("..", "test_examples", "chained_dependencies"),
"--cascade-dependencies",
})
}

func TestChainedDependenciesHiddenBehindFlag(t *testing.T) {
runTest(t, filepath.Join("golden", "chained_dependency_no_flag.yaml"), []string{
"--root",
filepath.Join("..", "test_examples", "chained_dependencies"),
"--cascade-dependencies=false",
})
}
34 changes: 34 additions & 0 deletions cmd/golden/chained_dependency.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
automerge: false
parallel_apply: true
parallel_plan: true
projects:
- autoplan:
enabled: false
when_modified:
- '*.hcl'
- '*.tf*'
dir: dependency
- autoplan:
enabled: false
when_modified:
- '*.hcl'
- '*.tf*'
- ../dependency/terragrunt.hcl
dir: depender
- autoplan:
enabled: false
when_modified:
- '*.hcl'
- '*.tf*'
- ../depender/terragrunt.hcl
- ../dependency/terragrunt.hcl
- nested/terragrunt.hcl
dir: depender_on_depender
- autoplan:
enabled: false
when_modified:
- '*.hcl'
- '*.tf*'
- ../../dependency/terragrunt.hcl
dir: depender_on_depender/nested
version: 3

0 comments on commit 9618e2a

Please sign in to comment.