Skip to content

Support parent-scoped multi-file stacks - #2787

Merged
Andriy Knysh (aknysh) merged 9 commits into
mainfrom
osterman/test-multi-file-stacks
Jul 29, 2026
Merged

Support parent-scoped multi-file stacks#2787
Andriy Knysh (aknysh) merged 9 commits into
mainfrom
osterman/test-multi-file-stacks

Conversation

@osterman

@osterman Erik Osterman (Cloud Posse) (osterman) commented Jul 23, 2026

Copy link
Copy Markdown
Member

what

  • Support one logical stack across multiple top-level parent manifests without merging parent scopes.
  • Keep each parent's metadata.inherits graph self-contained while canonicalizing equivalent imported duplicates by lexical parent path.
  • Add the top-level stack composition PRD and regression fixtures for naming, conflicts, isolation, explicit imports, and invalid inheritance.

why

  • Multiple top-level files that represented layers of the same logical stack were previously rejected or treated independently.
  • This enables aggregate component discovery without implicitly making another parent's imports part of a manifest's inheritance dependencies.
  • Shared inheritance bases remain intentional and reviewable through each parent's normal import graph.

references

  • docs/prd/top-level-stack-composition.md

Summary by CodeRabbit

  • New Features

    • Added parent-scoped multi-file stack discovery: multiple top-level parent manifests with the same stack identity are composed into one logical stack.
    • Kept parent-specific imports/globals/locals/component config isolated while aggregating components across parents.
    • Enabled controlled component inheritance within the composed logical stack.
  • Bug Fixes

    • Duplicate component configurations now resolve deterministically.
    • Conflicting duplicates and unsupported peer-only inheritance are rejected with clearer diagnostics.
  • Documentation

    • Added a PRD and published a blog post on parent-scoped multi-file stacks.
    • Updated the public roadmap to mark the feature as shipped.

@atmos-pro

atmos-pro Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

@github-actions github-actions Bot added size/l Large size PR and removed size/m Medium size PR labels Jul 23, 2026
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • ✅ Review completed - (🔄 Check again to review again)
📝 Walkthrough

Walkthrough

Atmos now groups top-level manifests by logical stack identity while preserving parent-scoped resolution. Equivalent duplicate components use a canonical source; conflicting configurations and peer-only inheritance produce explicit errors. Tests, fixtures, PRD, blog, and roadmap entries cover the behavior.

Changes

Top-Level Stack Composition

Layer / File(s) Summary
Duplicate component resolution
internal/exec/utils.go, errors/errors.go, internal/exec/describe_component.go
Component candidates are independently resolved and compared after provenance fields are removed. Equivalent configurations use a canonical candidate; conflicting matches return ErrDuplicateComponentConfig.
Composition validation
internal/exec/validate_stacks.go, internal/exec/validate_stacks_test.go
Validation uses per-file merge contexts, extracts component configurations from mapped manifests, compares duplicates, and returns typed errors for malformed mappings.
Composition scenario coverage
internal/exec/top_level_stack_composition_test.go, tests/fixtures/scenarios/top-level-stack-composition*/**
Fixtures and integration tests cover logical stack aggregation, imports, inheritance, parent scopes, duplicate conflicts, and isolated inheritance failures.
Behavior documentation and release metadata
docs/prd/top-level-stack-composition.md, website/blog/2026-07-25-parent-scoped-multi-file-stacks.mdx, website/src/data/roadmap.js
The PRD and blog document parent-scoped composition semantics, and roadmap data records the shipped milestone.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant StackManifest
  participant findComponentInStacks
  participant ProcessComponentConfig
  participant ValidateStacks
  StackManifest->>findComponentInStacks: provide candidate parent manifest
  findComponentInStacks->>ProcessComponentConfig: resolve parent-scoped component configuration
  ProcessComponentConfig-->>findComponentInStacks: resolved component candidate
  findComponentInStacks->>ValidateStacks: provide matching component configurations
  ValidateStacks-->>findComponentInStacks: equivalent result or duplicate error
Loading

Possibly related PRs

  • cloudposse/atmos#1835: Both changes prioritize explicit stack manifest names during stack identity resolution.

Suggested labels: minor

Suggested reviewers: aknysh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.64% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: enabling parent-scoped multi-file stacks.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch osterman/test-multi-file-stacks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (4)
internal/exec/validate_stacks.go (1)

232-232: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Terminate the new comment with a period.

As per coding guidelines, all Go comments must end with periods.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exec/validate_stacks.go` at line 232, Update the new comment
describing the merge context to end with a period, preserving its existing
wording.

Source: Coding guidelines

internal/exec/stack_processor_utils.go (3)

586-651: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Wrap errors with context before returning.

logicalStackIdentity (line 591) and m.DeepCopyMap (line 627) errors are returned raw, with no indication of which stack file failed. As per coding guidelines: "wrap errors, use errors.Join for multiple errors, %w for string context...".

♻️ Proposed fix
 		identity, err := logicalStackIdentity(atmosConfig, results[i].stackFileName, results[i].deepMergedConfig)
 		if err != nil {
-			return err
+			return fmt.Errorf("resolving logical stack identity for %q: %w", results[i].stackFileName, err)
 		}
 			deepMergedConfig, err := m.DeepCopyMap(results[index].deepMergedConfig)
 			if err != nil {
-				return err
+				return fmt.Errorf("copying deep-merged config for %q: %w", results[index].stackFileName, err)
 			}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exec/stack_processor_utils.go` around lines 586 - 651, Update
addPeerComponentsForInheritance to wrap errors from logicalStackIdentity and
m.DeepCopyMap with stack-file context before returning, using %w so the original
errors remain inspectable. Identify the failing stack via
results[i].stackFileName or results[index].stackFileName in the respective
loops.

Source: Coding guidelines


791-849: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Avoid deep-copying the whole stack config just to preserve component ownership.

originalConfigs[i] deep-copies the entire deepMergedConfig (vars, settings, env, locals, imports, everything), but retainOwnedComponents only ever reads originalConfig[cfg.ComponentsSectionName]. For stacks with large vars/settings sections this is unnecessary CPU/memory overhead on every file, on top of the deep copy already performed inside addPeerComponentsForInheritance.

⚡ Proposed fix
 	originalConfigs := make([]map[string]any, count)
 	for i := range processedResults {
-		originalConfig, err := m.DeepCopyMap(processedResults[i].deepMergedConfig)
-		if err != nil {
-			return nil, nil, nil, err
+		components, _ := processedResults[i].deepMergedConfig[cfg.ComponentsSectionName].(map[string]any)
+		originalComponents, err := m.DeepCopyMap(map[string]any{cfg.ComponentsSectionName: components})
+		if err != nil {
+			return nil, nil, nil, err
 		}
-		originalConfigs[i] = originalConfig
+		originalConfigs[i] = originalComponents
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exec/stack_processor_utils.go` around lines 791 - 849, Replace the
full deep-copy loop that builds originalConfigs with a copy containing only each
processed result’s component ownership section required by
retainOwnedComponents. Preserve the existing indexing and error behavior, and
keep retainOwnedComponents(finalConfig, originalConfigs[i]) unchanged unless
needed to consume the reduced configuration.

549-868: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Break up stack_processor_utils.go to stay under the file-length guideline.

This file is 3,049 lines, well above the stated below 600 lines cap. Move the peer-inheritance helpers and related composition logic into a focused helper/test file, such as stack_processor_composition.go, and keep this file near or under the limit.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/exec/stack_processor_utils.go` around lines 549 - 868, Split the
peer-inheritance and composition logic out of stack_processor_utils.go into a
focused file such as stack_processor_composition.go, moving
logicalStackIdentity, addPeerComponentsForInheritance, and retainOwnedComponents
together with any required imports and tests. Update ProcessYAMLConfigFiles to
continue calling these helpers without changing behavior, and reduce
stack_processor_utils.go to the stated below-600-line guideline.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/prd/top-level-stack-composition.md`:
- Around line 16-22: Update the ordered-list continuation indentation in the
logical stack requirements, especially items 1–5 and the corresponding section
around items 86–95, from 3 spaces to 4 spaces. Preserve the text and list
structure while ensuring continuation lines use indentation compatible with the
editorconfig.

In `@internal/exec/stack_processor_utils.go`:
- Around line 653-674: Update retainOwnedComponents so it still processes
finalComponents when originalConfig has no components section; only return early
when finalComponents is unavailable. Treat a nil originalComponents map as empty
while checking each componentType, allowing all injected peer components to be
removed when the manifest originally owned none.
- Around line 565-580: Update logicalStackIdentity to handle a nil atmosConfig
before accessing atmosConfig.Stacks or other configuration fields, returning the
appropriate stack-file-based identity for that case. Preserve the existing
manifest-name, name-template, and name-pattern precedence when atmosConfig is
non-nil.

In `@internal/exec/top_level_stack_composition_test.go`:
- Around line 13-16: Update initTopLevelStackCompositionConfig to construct the
fixture directory with filepath.Join using the existing parent, tests, fixtures,
scenarios, and fixture components, then pass that result to t.Chdir; add the
necessary filepath import.

In `@internal/exec/validate_stacks.go`:
- Around line 398-413: Update the validation errors in the stack manifest lookup
flow around stackConfig, components, componentTypeConfig, and componentConfig to
use suitable sentinel errors from errors/errors.go, adding them if necessary.
Wrap each sentinel with the existing manifest or component context so callers
can classify failures with errors.Is() while retaining the dynamic details.

---

Nitpick comments:
In `@internal/exec/stack_processor_utils.go`:
- Around line 586-651: Update addPeerComponentsForInheritance to wrap errors
from logicalStackIdentity and m.DeepCopyMap with stack-file context before
returning, using %w so the original errors remain inspectable. Identify the
failing stack via results[i].stackFileName or results[index].stackFileName in
the respective loops.
- Around line 791-849: Replace the full deep-copy loop that builds
originalConfigs with a copy containing only each processed result’s component
ownership section required by retainOwnedComponents. Preserve the existing
indexing and error behavior, and keep retainOwnedComponents(finalConfig,
originalConfigs[i]) unchanged unless needed to consume the reduced
configuration.
- Around line 549-868: Split the peer-inheritance and composition logic out of
stack_processor_utils.go into a focused file such as
stack_processor_composition.go, moving logicalStackIdentity,
addPeerComponentsForInheritance, and retainOwnedComponents together with any
required imports and tests. Update ProcessYAMLConfigFiles to continue calling
these helpers without changing behavior, and reduce stack_processor_utils.go to
the stated below-600-line guideline.

In `@internal/exec/validate_stacks.go`:
- Line 232: Update the new comment describing the merge context to end with a
period, preserving its existing wording.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3c37e459-1dae-4957-81b7-96375a7d1d08

📥 Commits

Reviewing files that changed from the base of the PR and between 6426ad2 and d46609e.

📒 Files selected for processing (30)
  • docs/prd/top-level-stack-composition.md
  • errors/errors.go
  • internal/exec/describe_component.go
  • internal/exec/stack_processor_utils.go
  • internal/exec/top_level_stack_composition_test.go
  • internal/exec/utils.go
  • internal/exec/validate_stacks.go
  • tests/fixtures/scenarios/top-level-stack-composition-conflict/atmos.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-conflict/components/terraform/mock/main.tf
  • tests/fixtures/scenarios/top-level-stack-composition-conflict/stacks/catalog/shared.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-conflict/stacks/parents/01-network.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-conflict/stacks/parents/02-platform.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-explicit-name/atmos.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-explicit-name/components/terraform/mock/main.tf
  • tests/fixtures/scenarios/top-level-stack-composition-explicit-name/stacks/catalog/shared.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-explicit-name/stacks/parents/01-network.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-explicit-name/stacks/parents/02-platform.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-isolated-inheritance/atmos.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-isolated-inheritance/components/terraform/mock/main.tf
  • tests/fixtures/scenarios/top-level-stack-composition-isolated-inheritance/stacks/parents/01-network.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-isolated-inheritance/stacks/parents/02-platform.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-parent-scope/atmos.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-parent-scope/components/terraform/mock/main.tf
  • tests/fixtures/scenarios/top-level-stack-composition-parent-scope/stacks/parents/01-network.yaml
  • tests/fixtures/scenarios/top-level-stack-composition-parent-scope/stacks/parents/02-platform.yaml
  • tests/fixtures/scenarios/top-level-stack-composition/atmos.yaml
  • tests/fixtures/scenarios/top-level-stack-composition/components/terraform/mock/main.tf
  • tests/fixtures/scenarios/top-level-stack-composition/stacks/catalog/shared.yaml
  • tests/fixtures/scenarios/top-level-stack-composition/stacks/parents/01-network.yaml
  • tests/fixtures/scenarios/top-level-stack-composition/stacks/parents/02-platform.yaml

Comment thread docs/prd/top-level-stack-composition.md
Comment thread internal/exec/stack_processor_utils.go Outdated
Comment thread internal/exec/stack_processor_utils.go Outdated
Comment thread internal/exec/top_level_stack_composition_test.go
Comment thread internal/exec/validate_stacks.go Outdated
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@fc4960a). Learn more about missing BASE report.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2787   +/-   ##
=======================================
  Coverage        ?   81.84%           
=======================================
  Files           ?     1793           
  Lines           ?   173241           
  Branches        ?        0           
=======================================
  Hits            ?   141787           
  Misses          ?    23656           
  Partials        ?     7798           
Flag Coverage Δ
unittests 81.84% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
errors/errors.go 100.00% <ø> (ø)
internal/exec/describe_component.go 79.31% <100.00%> (ø)
internal/exec/utils.go 85.22% <100.00%> (ø)
internal/exec/validate_stacks.go 82.02% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 23, 2026
@osterman Erik Osterman (Cloud Posse) (osterman) added minor New features that do not break anything and removed patch A minor, backward compatible change labels Jul 25, 2026
@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

Note

Release Documentation Complete ✅

  • Changelog: website/blog/2026-07-25-parent-scoped-multi-file-stacks.mdx
  • Roadmap: website/src/data/roadmap.js

Thank you!

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 27, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 27, 2026
checkComponentStackMap validates terraform and helmfile component
types with dedicated sentinel errors; every other component type
falls through to the generic ErrInvalidComponentsSection default
case, which was untested and showed zero coverage on the newly added
lines in internal/exec/validate_stacks.go.
@aknysh
Andriy Knysh (aknysh) merged commit 8f56f56 into main Jul 29, 2026
142 of 143 checks passed
@atmos-pro

atmos-pro Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@aknysh
Andriy Knysh (aknysh) deleted the osterman/test-multi-file-stacks branch July 29, 2026 00:06
@github-actions

Copy link
Copy Markdown

These changes were released in v1.225.0-rc.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minor New features that do not break anything size/l Large size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants