Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/actions/build-website/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,15 @@ runs:
tag: v0.18.0
cache: enable

- name: Install Atmos
uses: cloudposse/github-action-setup-atmos@v2
with:
install-wrapper: false

- name: "Render Documentation for Terraform Components"
shell: bash
env:
PUBLIC_REPO_ACCESS_TOKEN: ${{ inputs.repo_access_token }}
PUBLIC_REPO_ACCESS_TOKEN: ${{ inputs.repo_access_token }}
run: |
./scripts/render-docs-for-components.sh

Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,7 @@ content/reference-architecture/**
!content/components/library/aws/_category_.json
!content/github-actions/library/actions/_category_.json
changelog/*

.atmos
.python-version
yarn.lock
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,9 @@ real-clean:

lint:
npx docusaurus-mdx-checker --cwd docs

readme/build:
@atmos docs generate readme

readme:
@atmos docs generate readme
50 changes: 39 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@

<!-- markdownlint-disable -->
# Developer Documentation <a href="https://cpco.io/homepage?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content="><img align="right" src="https://cloudposse.com/logo-300x69.svg" width="150" /></a>
<a href="https://github.com/cloudposse/docs/releases"><img src="https://img.shields.io/github/release/cloudposse/docs.svg?style=for-the-badge" alt="Latest Release"/></a><a href="https://slack.cloudposse.com"><img src="https://slack.cloudposse.com/for-the-badge.svg" alt="Slack Community"/></a>


<a href="https://github.com/cloudposse/docs/releases"><img src="https://img.shields.io/github/release/cloudposse/docs.svg?style=for-the-badge" alt="Latest Release"/></a><a href="https://slack.cloudposse.com"><img src="https://slack.cloudposse.com/for-the-badge.svg" alt="Slack Community"/></a><a href="https://cloudposse.com/support/"><img src="https://img.shields.io/badge/Get_Support-success.svg?style=for-the-badge" alt="Get Support"/></a>


<!-- markdownlint-restore -->

<!--
Expand Down Expand Up @@ -55,17 +59,9 @@ make start



<!-- markdownlint-disable -->
## Makefile Targets
```text
Available targets:

help Help screen
help/all Display help for all targets
help/short This help short screen

```
<!-- markdownlint-restore -->



## Related Projects
Expand Down Expand Up @@ -100,6 +96,38 @@ In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

**NOTE:** Be sure to merge the latest changes from "upstream" before making a pull request!


## Running Terraform Tests

We use [Atmos](https://atmos.tools) to streamline how Terraform tests are run. It centralizes configuration and wraps common test workflows with easy-to-use commands.

All tests are located in the [`test/`](test) folder.

Under the hood, tests are powered by Terratest together with our internal [Test Helpers](https://github.com/cloudposse/test-helpers) library, providing robust infrastructure validation.

Setup dependencies:
- Install Atmos ([installation guide](https://atmos.tools/install/))
- Install Go [1.24+ or newer](https://go.dev/doc/install)
- Install Terraform or OpenTofu

To run tests:

- Run all tests:
```sh
atmos test run
```
- Clean up test artifacts:
```sh
atmos test clean
```
- Explore additional test options:
```sh
atmos test --help
```
The configuration for test commands is centrally managed. To review what's being imported, see the [`atmos.yaml`](https://raw.githubusercontent.com/cloudposse/.github/refs/heads/main/.github/atmos/terraform-module.yaml) file.

Learn more about our [automated testing in our documentation](https://docs.cloudposse.com/community/contribute/automated-testing/) or implementing [custom commands](https://atmos.tools/core-concepts/custom-commands/) with atmos.

### 🌎 Slack Community

Join our [Open Source Community](https://cpco.io/slack?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content=slack) on Slack. It's **FREE** for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally *sweet* infrastructure.
Expand Down Expand Up @@ -137,7 +165,7 @@ All other trademarks referenced herein are the property of their respective owne


---
Copyright © 2017-2024 [Cloud Posse, LLC](https://cpco.io/copyright)
Copyright © 2017-2025 [Cloud Posse, LLC](https://cpco.io/copyright)


<a href="https://cloudposse.com/readme/footer/link?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content=readme_footer_link"><img alt="README footer" src="https://cloudposse.com/readme/footer/img"/></a>
Expand Down
3 changes: 0 additions & 3 deletions README.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ description: |-

Let's jump right in! Here's how to get started with our documentation.

include:
- "docs/targets.md"

usage: |-
1. Build dependencies
```
Expand Down
12 changes: 12 additions & 0 deletions atmos.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Atmos Configuration — powered by https://atmos.tools
#
# This configuration enables centralized, DRY, and consistent project scaffolding using Atmos.
#
# Included features:
# - Organizational custom commands: https://atmos.tools/core-concepts/custom-commands
# - Automated README generation: https://atmos.tools/cli/commands/docs/generate
#

# Import shared configuration used by all modules
import:
- https://raw.githubusercontent.com/cloudposse/.github/refs/heads/main/.github/atmos/default.yaml
7 changes: 5 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions scripts/docs-collator/AbstractFetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
DOCS_DIR = "docs"
TARGETS_MD = "targets.md"
README_YAML = "README.yaml"
ATMOS_YAML = "atmos.yaml"


class MissingReadmeYamlException(Exception):
Expand All @@ -29,3 +30,7 @@ def _fetch_docs(self, repo, module_download_dir, submodule_dir=""):
continue

self.github_provider.fetch_file(repo, remote_file, module_download_dir)

def _fetch_atmos_yaml(self, repo, module_download_dir):
self.github_provider.fetch_file(repo, ATMOS_YAML, module_download_dir)

8 changes: 8 additions & 0 deletions scripts/docs-collator/GitHubActionFetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
ACTIONS_DIR = "actions"
README_YAML = "README.yaml"
README_MD = "README.md"
ACTION_YAML = "action.yml"


class GitHubActionFetcher(AbstractFetcher):
Expand All @@ -23,12 +24,19 @@ def fetch(self, repo):

self._fetch_readme_yaml(repo, repo_download_dir)

self._fetch_action_yaml(repo, repo_download_dir)

self._fetch_atmos_yaml(repo, repo_download_dir)

if DOCS_DIR in remote_files:
self._fetch_docs(repo, repo_download_dir)

if ACTIONS_DIR in remote_files:
self.__fetch_sub_actions(repo, repo_download_dir)

def _fetch_action_yaml(self, repo, module_download_dir):
self.github_provider.fetch_file(repo, ACTION_YAML, module_download_dir)

def __fetch_sub_actions(self, repo, module_download_dir):
remote_files = self.github_provider.list_repo_dir(repo, ACTIONS_DIR)

Expand Down
20 changes: 10 additions & 10 deletions scripts/docs-collator/GitHubActionRenderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@
from utils import rendering, templating

TARGETS_MD = "targets.md"
README_YAML = "README.yaml"
README_MD = "README.md"
ATMOS_YAML = "atmos.yaml"
INDEX_CATEGORY_JSON = "_category_.json"
README_TEMPLATE = "readme.md"
DOC_SUBFOLDER = "actions"

SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
Expand Down Expand Up @@ -45,25 +44,26 @@ def render(self, repo):
self._copy_extra_resources_for_docs(repo_download_dir, module_docs_dir)

def __render_readme(self, module_download_dir, module_docs_dir):
readme_yaml_file = os.path.join(module_download_dir, README_YAML)
readme_md_file = os.path.join(module_download_dir, README_MD)
readme_tmpl_file = os.path.join(TEMPLATES_DIR, README_TEMPLATE)
atmos_yaml_file = os.path.join(TEMPLATES_DIR, ATMOS_YAML)

io.create_dirs(module_docs_dir)

response = subprocess.run(
[
"make",
"atmos",
"--config",
f"{atmos_yaml_file}",
"docs",
"generate",
"readme",
f"README_TEMPLATE_FILE={readme_tmpl_file}",
f"README_FILE={readme_md_file}",
f"README_YAML={readme_yaml_file}",
f"README_TEMPLATE_YAML={readme_yaml_file}",
f"README_INCLUDES={module_download_dir}",
],
capture_output=True,
cwd=module_download_dir,
)

logging.info(response.stderr.decode("utf-8"))

if response.returncode != 0:
error_message = response.stderr.decode("utf-8")
raise TerraformDocsRenderingError(error_message)
Expand Down
4 changes: 4 additions & 0 deletions scripts/docs-collator/ModuleFetcher.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os

from AbstractFetcher import AbstractFetcher, MissingReadmeYamlException
import logging

DOCS_DIR = "docs"
IMAGES_DIR = "images"
Expand Down Expand Up @@ -56,12 +57,15 @@ def __fetch_submodules(self, repo, module_download_dir):
for readme_file in readme_files.values():
self.github_provider.fetch_file(repo, readme_file, module_download_dir)
if os.path.basename(readme_file) == README_YAML:
logging.info(f"Fetching submodule docs for: {os.path.dirname(readme_file)}")
self._fetch_docs(
repo,
module_download_dir,
submodule_dir=os.path.dirname(readme_file),
)

self.__fetch_terraform_files(repo, module_download_dir, remote_files)

def __fetch_terraform_files(self, repo, module_download_dir, remote_files):
for remote_file in remote_files:
# We only need variables and output files to render terraform-docs.
Expand Down
44 changes: 21 additions & 23 deletions scripts/docs-collator/ModuleRenderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@
DOCS_DIR = "docs"
IMAGES_DIR = "images"
SUBMODULES_DIR = "modules"
TARGETS_MD = "targets.md"
ATMOS_YAML = "atmos.yaml"
README_YAML = "README.yaml"
README_MD = "README.md"
INDEX_CATEGORY_JSON = "_category_.json"
MODULES_README_TEMPLATE = "readme.md"

SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
TEMPLATES_DIR = os.path.join(SCRIPT_DIR, "templates/modules")
Expand Down Expand Up @@ -59,45 +58,44 @@ def render(self, repo):
# self.__create_index_for_provider(repo)
# self.__create_indexes_for_subfolders(repo)

def __render_readme(self, module_download_dir, module_docs_dir):
def __render_readme(self, module_download_dir, module_docs_dir, template_name="readme"):
readme_yaml_file = os.path.join(module_download_dir, README_YAML)
readme_md_file = os.path.join(module_download_dir, README_MD)
readme_tmpl_file = os.path.join(TEMPLATES_DIR, MODULES_README_TEMPLATE)
atmos_yaml_file = os.path.join(TEMPLATES_DIR, ATMOS_YAML)

io.create_dirs(module_docs_dir)

# Re-render terraform docs with this repo's terraform-docs template for modules.
# This replaces docs/terraform.md for the given module in place
logging.debug(f"Rendering terraform docs for: {module_download_dir}")
rendering.render_terraform_docs(
module_download_dir, os.path.join(TEMPLATES_DIR, "terraform-docs.yml")
)

# Run the make readme command in the module directory to compile README.md
logging.debug(f"Rendering README.md for: {module_download_dir}")
logging.debug(f"make readme")
logging.debug(f"README_TEMPLATE_FILE: {readme_tmpl_file}")
logging.debug(f"atmos docs generate readme --config {atmos_yaml_file}")
logging.debug(f"ATMOS_YAML_FILE: {atmos_yaml_file}")
logging.debug(f"README_FILE: {readme_md_file}")
logging.debug(f"README_YAML: {readme_yaml_file}")
logging.debug(f"README_TEMPLATE_YAML: {readme_yaml_file}")
logging.debug(f"README_INCLUDES: {module_download_dir}")
logging.debug(f"README_MD: {readme_md_file}")
response = subprocess.run(
[
"make",
"readme",
f"README_TEMPLATE_FILE={readme_tmpl_file}",
f"README_FILE={readme_md_file}",
f"README_YAML={readme_yaml_file}",
f"README_TEMPLATE_YAML={readme_yaml_file}",
f"README_INCLUDES={module_download_dir}",
"atmos",
"--config",
f"{atmos_yaml_file}",
"docs",
"generate",
template_name,
],
capture_output=True,
cwd=module_download_dir,
)

if response.returncode != 0:
error_message = response.stderr.decode("utf-8")
raise TerraformDocsRenderingError(error_message)

# Re-render terraform docs with this repo's terraform-docs template for modules.
# This replaces docs/terraform.md for the given module in place
logging.debug(f"Rendering terraform docs for: {module_download_dir}")
rendering.render_terraform_docs(
module_download_dir, os.path.join(TEMPLATES_DIR, "terraform-docs.yml")
)

logging.info(f"Rendered: {readme_md_file}")

def __copy_extra_resources_for_images(self, module_download_dir, module_docs_dir):
Expand Down Expand Up @@ -143,7 +141,7 @@ def __copy_extra_resources_for_submodules(
submodule_dir = os.path.dirname(readme)
readme_md = os.path.join(submodule_dir, README_MD)
self._pre_rendering_fixes(repo, module_download_dir, rel_dir)
self.__render_readme(submodule_dir, dest_dir)
self.__render_readme(submodule_dir, dest_dir, "submodule-readme")
self._post_rendering_fixes(repo, readme_md, rel_dir)
io.copy_file(readme_md, os.path.join(dest_dir, README_MD))
continue
Expand Down
1 change: 1 addition & 0 deletions scripts/docs-collator/component/fetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def fetch(self, repo):

self._fetch_readme_yaml(repo, module_download_dir)
self._fetch_readme_md(repo, module_download_dir)
self._fetch_atmos_yaml(repo, module_download_dir)

if CHANGELOG_MD in remote_files:
self._fetch_changelog_md(repo, module_download_dir)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
docs:
generate:
readme:
base-dir: .
input:
- "./README.yaml"
- get_support: true
template: "../../../scripts/docs-collator/component/renderer/templates/components/yml/readme.md"
output: "./README.md"
terraform:
source: ./src
enabled: true
format: "markdown table"
show_providers: false
show_inputs: true
show_outputs: true
sort_by: "name"
hide_empty: false
indent_level: 2
Loading
Loading