diff --git a/.claude/commands/add-borders.md b/.claude/commands/add-borders.md new file mode 100644 index 000000000000..761cfad232cf --- /dev/null +++ b/.claude/commands/add-borders.md @@ -0,0 +1,104 @@ +--- +description: Add 1px grey borders to PNG images in a documentation file. +--- + +# Usage + +`/add-borders ` + +Analyze PNG images referenced in the specified markdown file at {{arg}} and add 1px #999999 (medium grey) borders to any images that don't already have them. This improves visual clarity and consistency of screenshots and diagrams in documentation. + +--- + +## Process + +### 1. Verify prerequisites + +Before running the border tool, ensure dependencies are installed: + +```bash +cd /workspaces/src/pulumi/docs/scripts/image-borders && pipenv install +``` + +If this is the first run or if Pillow is not yet installed, the installation will complete automatically. + +### 2. Run the border addition tool + +Execute the Python script with the provided documentation file: + +```bash +cd /workspaces/src/pulumi/docs && pipenv run python scripts/image-borders/add_borders.py {{arg}} +``` + +The script will: +- Parse the markdown file to find all PNG image references +- Check each image's edge pixels to detect existing borders +- Add a 1px #999999 border to images that don't have one +- Skip images that already have a grey border (within tolerance) +- Display a summary of modified and skipped images + +### 3. Review the results + +The script outputs: +- **Modified**: Images that received new borders +- **Skipped**: Images that already had borders + +Example output: +``` +Found 3 PNG image(s) + +✓ Added border: content/docs/esc/assets/create-environment.png +✓ Added border: content/docs/esc/assets/editor-before-save.png +⊘ Skipped (has border): content/docs/esc/assets/editor-after-save.png + +Summary: + Modified: 2 + Skipped: 1 +``` + +### 4. Verify the changes + +After borders are added: +1. View the modified images to ensure borders look correct +2. Check that border width and color (#CCCCCC) are appropriate +3. Verify the images still display correctly in the documentation + +You can preview the documentation locally: +```bash +make serve +``` + +Then navigate to the affected page at http://localhost:1313 + +### 5. Commit the changes (if appropriate) + +If the borders improve the documentation quality, commit the modified images: + +```bash +git add +git commit -m "Add 1px borders to documentation images for visual clarity" +``` + +--- + +## Options + +The script supports additional options: + +- `--dry-run` - Show what would be done without making changes +- `--repo-root ` - Specify repository root (auto-detected by default) + +Example with dry run: +```bash +pipenv run python scripts/image-borders/add_borders.py {{arg}} --dry-run +``` + +--- + +## Notes + +- The script uses edge pixel analysis to detect existing borders with 80% confidence threshold +- Handles RGBA images (preserves transparency) +- Uses #999999 (medium grey) for visibility on both white and light backgrounds +- Only processes PNG files (ignores JPG, GIF, SVG, etc.) +- Modifies images in place (use git to revert if needed) diff --git a/.claude/commands/glow-up.md b/.claude/commands/glow-up.md index c1aeffbe8f3f..e84c3f651aff 100644 --- a/.claude/commands/glow-up.md +++ b/.claude/commands/glow-up.md @@ -76,8 +76,13 @@ Read the entire target file and perform comprehensive analysis: For each image or diagram referenced in the document: -1. Use the Read tool to view the image -2. Analyze the image for the following issues: +1. **Check for missing borders on PNG images**: Run the border detection script with dry-run mode to deterministically identify which PNG images need borders: + ```bash + cd /workspaces/src/pulumi/docs/scripts/image-borders && pipenv run python add_borders.py --dry-run + ``` + The script will report which images would be modified vs skipped. + +2. Regardless of existing borders, use the Read tool to view the image and analyze the image for the following issues: **Screenshots:** - Missing or inadequate alt text (accessibility requirement) @@ -163,6 +168,7 @@ Develop a detailed plan organized by issue category: - Images to convert to Mermaid/GoAT - Unnecessary images to remove - File format or size optimizations + - PNG images that need 1px grey borders for visual clarity **7. Content enhancements** — List specific recommendations: - Unclear sections that need rewriting @@ -204,7 +210,8 @@ Once approved, implement the changes in logical order: 4. **Style improvements** (remove filler, simplify sentences, active voice) 5. **Link improvements** (improve link text, add cross-references) 6. **Image improvements** (add alt text, remove unnecessary images, flag outdated screenshots) -7. **Content enhancements** (rewrite unclear sections, add context) +7. **Add borders to PNG images** (run `/add-borders ` if PNG images need borders) +8. **Content enhancements** (rewrite unclear sections, add context) Use the TodoWrite tool to track progress through the improvements. diff --git a/Pipfile b/Pipfile new file mode 100644 index 000000000000..0757494bb360 --- /dev/null +++ b/Pipfile @@ -0,0 +1,11 @@ +[[source]] +url = "https://pypi.org/simple" +verify_ssl = true +name = "pypi" + +[packages] + +[dev-packages] + +[requires] +python_version = "3.11" diff --git a/STYLE-GUIDE.md b/STYLE-GUIDE.md index 6f4b53073e9f..837277dd9df6 100644 --- a/STYLE-GUIDE.md +++ b/STYLE-GUIDE.md @@ -77,6 +77,14 @@ The symbol is not needed in regular in-text links within documentation pages. --- +## Images and Media + +- Use relative paths for images stored in the same directory or a subdirectory. +- Provide descriptive alt text for all images. +- For partial screenshots where the image may be hard to distinguish from the page background, add a 1px gray #999999 border. + +--- + ## Notes / Callouts Use the `{{ notes }}` shortcode sparingly. Supported levels: diff --git a/content/docs/deployments/deployments/cloud-credentials.md b/content/docs/deployments/deployments/cloud-credentials.md index 4546574115da..000d40407aa6 100644 --- a/content/docs/deployments/deployments/cloud-credentials.md +++ b/content/docs/deployments/deployments/cloud-credentials.md @@ -17,7 +17,7 @@ menu: In order for a Pulumi IaC operation like `update` or `preview` work, the Pulumi CLI must be able to access credentials that will allow it to perform the necessary CRUD operations on the resources in your stack. In order for Pulumi Deployments to access the necessary cloud credentials to run your Pulumi operation there are two common approaches you can take: 1. **Use [Pulumi Deployments' OIDC integration](/docs/deployments/deployments/oidc/)** where possible (we support [AWS](/docs/deployments/deployments/oidc/aws/), [Azure](/docs/deployments/deployments/oidc/azure/), and [Google Cloud](/docs/deployments/deployments/oidc/gcp/)), and store any remaining required secrets or configuration in [Pulumi Deployments Environment Variables](/docs/deployments/deployments/reference/#deployment-settings). -2. **Use [Pulumi ESC](/docs/esc/)** to define an Environment (or Environments), and [import the environment(s) into your stack](/docs/esc/get-started/integrate-with-pulumi-iac/). +2. **Use [Pulumi ESC](/docs/esc/)** to define an Environment (or Environments), and [import the environment(s) into your stack](/docs/esc/guides/integrate-with-pulumi-iac/). ## Choosing between Pulumi ESC Environments and Pulumi Deployments OIDC diff --git a/content/docs/esc/get-started/esc-base-url.png b/content/docs/esc/assets/esc-base-url.png similarity index 100% rename from content/docs/esc/get-started/esc-base-url.png rename to content/docs/esc/assets/esc-base-url.png diff --git a/content/docs/esc/assets/esc-create-environment.png b/content/docs/esc/assets/esc-create-environment.png deleted file mode 100644 index 3d56457de72f..000000000000 Binary files a/content/docs/esc/assets/esc-create-environment.png and /dev/null differ diff --git a/content/docs/esc/concepts/_index.md b/content/docs/esc/concepts/_index.md index f81c53dd1726..db089d5d3b41 100644 --- a/content/docs/esc/concepts/_index.md +++ b/content/docs/esc/concepts/_index.md @@ -2,7 +2,7 @@ title: Concepts title_tag: Pulumi ESC Concepts h1: Pulumi ESC Concepts -meta_desc: Pulumi ESC allows you to compose and manage hierarchical collections of configuration and secrets and consume them in various ways. +meta_desc: Learn the core concepts of Pulumi ESC including environments, providers, composition, and the evaluation model. menu: esc: parent: esc-home @@ -12,58 +12,127 @@ aliases: - /docs/concepts/environments/ --- -Pulumi ESC (Environments, Secrets, and Configuration) simplifies how organizations manage secrets and configurations across multiple environments. It enables teams to compose collections of configuration and secrets called *environments*, which can be consumed by various infrastructure and application services. ESC helps ensure security, consistency, and efficiency in handling secrets and configuration. +Pulumi ESC (Environments, Secrets, and Configuration) is a secrets and configuration management system built around four core concepts: environments, sources, targets, and management. Understanding these concepts will help you build a mental model of how ESC works. -Pulumi ESC is offered both as a fully managed cloud service in [Pulumi Cloud](/docs/pulumi-cloud/) and self-hosted for scenarios that require isolated environments. ESC has native integration with several products and other Pulumi products, including Pulumi Infrastructure as Code (IaC). The [pulumi/esc project](https://github.com/pulumi/esc) is open source, and contains the evaluation engine for environments, the `esc` CLI. +Pulumi ESC is available as a fully managed service in [Pulumi Cloud](/docs/pulumi-cloud/) and can be self-hosted for isolated environments. The [pulumi/esc project](https://github.com/pulumi/esc) is open source and contains the evaluation engine and CLI. {{< figure src="/docs/esc/assets/pulumi_esc.png" caption="Figure: The Pulumi ESC ecosystem.">}} -The diagram above shows four key concepts: +## Core concepts -1. ***Environments***: ESC environments are collections of secrets and configuration. Environments are composable from multiple environments and draw from both static and dynamic sources and can be shared to many targets. +### Environments -2. ***Sources***: ESC can *input* configuration and secrets from a variety of sources, and it has an extensible plugin model for integrating third-party sources. ESC's built-in support for dynamic secret providers, allows for best-practices like generating short-term credentials via OIDC, and dynamically pulling secret values at the time of use, for all major cloud providers. +An environment is a named collection of configuration values and secrets defined in YAML. Environments are the fundamental unit of organization in ESC. Each environment: -3. ***Targets***: ESC *outputs* configuration and secrets to a variety of popular targets. Pulumi ESC has a rich API that allows for easy integration, and provides standard output mechanisms like environment variables and key files. +- Has a unique name within a project (e.g., `my-org/my-project/production`) +- Contains a YAML document that defines values, imports other environments, and configures providers +- Is versioned—every change creates a new version +- Can be tagged for easier reference (e.g., `stable`, `v1.2.3`) +- Is evaluated when opened, resolving all dynamic values and imports -4. ***Management***: ESC environments are centrally managed in Pulumi Cloud, and can be permissioned with RBAC, versioned, tagged, and audited. ESC secrets are encrypted in flight and at rest. +Environments are composable: one environment can import others, inheriting and overriding their values. This enables organizing configuration hierarchically to match team boundaries and security requirements. -## The ESC Approach +### Sources -Pulumi ESC takes a distinct approach to managing secrets and configuration that is different from other secret managers. ESC emphasizes flexibility and an open-ecosystem approach to integrations and is specifically designed for managing secrets and configurations across complex multi-cloud environments. Whether used in conjunction with [Pulumi IaC](/docs/iac/) or as a standalone tool, ESC helps streamline operations, reduce duplication, and enhance security for teams across a wide range of use cases. +ESC pulls configuration and secrets from multiple sources through an extensible provider system: -### Centralized management, composability, and reusability +- **Static values** - Key-value pairs defined directly in the environment YAML +- **Dynamic providers** - Plugins that generate or retrieve values at evaluation time: + - OIDC providers generate short-lived cloud credentials (AWS, Azure, GCP, Vault) + - Secret providers retrieve values from external systems (AWS Secrets Manager, Azure Key Vault, 1Password) + - Login providers authenticate with cloud services -Pulumi ESC aggregates secrets and configuration from different sources into *environments*. These environments can be composed from other environments, allowing for flexible organization and reuse without duplication. +Dynamic providers execute when an environment is opened, not when it's defined. This ensures credentials and secrets are fresh and never stored. -Pulumi ESC provides fine-grained **Role-Based Access Control (RBAC)**, ensuring that only authorized users and teams can access specific environments and retrieve secrets. Audit logs track who accessed or changed the configurations, enhancing security and accountability. +### Targets + +ESC outputs configuration and secrets to multiple targets: + +- **Environment variables** - Inject secrets into processes via the `esc run` command +- **Configuration APIs** - Access values programmatically through SDKs (TypeScript, Python, Go) +- **Pulumi IaC** - Reference environments in `Pulumi..yaml` files +- **CLI output** - View values with `esc env open` or `esc env get` +- **Kubernetes** - Sync secrets to Kubernetes clusters via the ESC operator +- **Other tools** - Integrate with Terraform, Docker, and developer tools + +The same environment can output to multiple targets simultaneously, centralizing secrets management across different tools. + +### Management + +Environments are centrally managed in Pulumi Cloud with: + +- **Role-Based Access Control (RBAC)** - Control who can read, write, or delete environments +- **Audit logs** - Track all access and modifications to environments +- **Versioning** - Every change creates a new immutable version +- **Tagging** - Label versions for rollback or staged deployments +- **Encryption** - All secrets are encrypted at rest and in transit +- **Customer-managed keys** - Optionally encrypt with your own keys + +## Environment composition + +Environments support composition through imports, allowing you to build complex configurations from reusable pieces. Consider this example: {{< figure src="team_environments.png" caption="Figure: Composable ESC environments facilitate team-based organization.">}} -Pulumi ESC environments can be structured to reflect an organization’s team structure, security boundaries, or deployment targets. The above diagram shows an example of how a customer might organize and compose different security environments. +An organization might structure environments by team and security boundaries: + +- **Central platform team** - Owns common configuration like OIDC settings and feature flags +- **Billing service team** - Manages payment processor secrets +- **Communications team** - Manages secrets for mailing and texting services + +Each team's environment can be independently permissioned and versioned. Application environments import these base environments and add application-specific configuration. When environments import others, values are merged using JSON Merge Patch semantics—later values override earlier ones. + +This structure: + +- Reduces duplication by sharing common configuration +- Enforces security boundaries through separate permissions per environment +- Allows teams to update their environments without affecting others +- Enables complex applications to compose from multiple security contexts + +## Static vs. dynamic values + +ESC supports both static and dynamic values in the same environment: + +**Static values** are defined directly in the YAML and evaluated once: -Imagine a hypothetical development organization comprised of a few teams: +```yaml +values: + region: us-west-2 + apiEndpoint: https://api.example.com +``` -* The *billing service* team, that manages secrets/config for the payment processor -* The *communications* team, that manages secrets/config for the mailing service and texting service. -* The *central platform* team, that owns most common config, including OIDC config and the keys/config for the feature flag system. +**Dynamic values** are generated or retrieved when the environment is opened: -Permissions to these environments can be defined separately to minimize security exposure. They can be reused independently or composed into more complex application scenarios. +```yaml +values: + aws: + login: + fn::open::aws-login: + oidc: + roleArn: arn:aws:iam::123456789012:role/my-role + environmentVariables: + AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId} +``` -### Dynamic vs static configurations +Dynamic providers execute during evaluation, generating fresh credentials or retrieving current values. This ensures credentials are short-lived and secrets are never stale. -Pulumi ESC supports both **static** configurations (e.g. simple key-value pairs) and **dynamic** configurations (values retrieved from third-party providers) in the same environment. This allows teams to mix-and-match the type of configuration they need. +## Configuration as code -Pulumi ESC also supports **dynamic secret providers**, such as AWS OIDC, Azure KeyVault, GCP Secrets Manager, and more. This allows teams to pull short-lived credentials or other secrets dynamically from external sources. +ESC environments are defined as YAML documents that can be: -More detail on dynamic secret providers is available in [Adding OIDC and secrets providers](/docs/esc/environments/working-with-environments/#using-secrets-providers-and-oidc). The [providers list](/docs/esc/integrations/) details the currently supported integrations. +- Edited directly in the Pulumi Cloud console +- Modified via the `esc` CLI +- Managed programmatically through SDKs (TypeScript, Python, Go) +- Automated in CI/CD pipelines via the API -### Configuration-as-Code, automation, and integration everywhere +Environments support: -Like our other products, Pulumi ESC uses an "as-code" approach to configuration and secrets. ESC environments can be composed, managed, and accessed using code written in TypeScript, JavaScript, Go, Python, or YAML. The `esc` CLI and our full-featured API allows for scripted use in automated environments like CI/CD. This reduces copy/paste style duplication of credentials and allows for management from a single source of truth. ESC is already deeply integrated into Pulumi IaC and Pulumi Cloud, and provides a number of third-party product integrations both as secrets providers and consumers. +- **Interpolation** - Reference other values with `${path.to.value}` syntax +- **Functions** - Transform values with built-in functions (`fn::secret`, `fn::toJSON`) +- **Type safety** - Values maintain their types (strings, numbers, objects, arrays) -## Learn More +## Learn more -* [How Pulumi ESC Works](/docs/esc/concepts/how-esc-works) -* The [ESC providers](/docs/esc/integrations/) list -* [Environments Overview](/docs/esc/environments/) +- [How Pulumi ESC works](/docs/esc/concepts/how-esc-works) - Architecture and evaluation model +- [ESC providers](/docs/esc/integrations/) - Full list of integrations +- [Environments reference](/docs/esc/environments/) - Complete syntax documentation diff --git a/content/docs/esc/environments/_index.md b/content/docs/esc/environments/_index.md index 2546b7810fd2..7c19eb04987c 100644 --- a/content/docs/esc/environments/_index.md +++ b/content/docs/esc/environments/_index.md @@ -7,7 +7,7 @@ menu: esc: parent: esc-home identifier: esc-environments - weight: 4 + weight: 5 --- Pulumi ESC (Environments, Secrets, and Configuration) lets you define collections of configuration settings and secrets called _environments_ and use them in any application or service. Environments are YAML documents composed of static key-value pairs, programmatic expressions, dynamically retrieved values from supported providers including all major clouds through OpenID Connect (OIDC), and other Pulumi ESC environments. diff --git a/content/docs/esc/get-started/_index.md b/content/docs/esc/get-started/_index.md index 76687df4bbd4..be045e837165 100644 --- a/content/docs/esc/get-started/_index.md +++ b/content/docs/esc/get-started/_index.md @@ -1,32 +1,142 @@ --- -title: Get started +title: Get Started title_tag: Get Started with Pulumi ESC (Environments, Secrets, and Configuration) -h1: Get Started with Pulumi ESC (Environments, Secrets, and Configuration) -meta_desc: Learn how to manage secrets and hierarchical configuration with Pulumi ESC. +h1: Get Started with Pulumi ESC +meta_desc: Get started with Pulumi ESC in 5 minutes. Create an environment, store a secret, and retrieve it programmatically. +weight: 1 menu: esc: parent: esc-home identifier: esc-get-started - weight: 2 aliases: - /docs/pulumi-cloud/esc/get-started/ + - /docs/esc/get-started/begin/ + - /docs/esc/get-started/create-environment/ + - /docs/pulumi-cloud/esc/get-started/begin/ + - /docs/pulumi-cloud/esc/get-started/create-environment/ --- -In a typical application or infrastructure development workflow, there's often a need to maintain multiple environments such as development, staging, and production. Each of these environments might have its own set of configuration values: API endpoints, database connection strings, third-party secrets, and more. +Pulumi ESC (Environments, Secrets, and Configuration) is a centralized secrets and configuration management service. In this quick start, you'll create your first environment, store a secret, and retrieve it programmatically. -Hardcoding these values or keeping them inside source code is a security risk and makes managing configurations complex. [Pulumi ESC (Environments, Secrets and Configuration)](/docs/esc/) offers a centralized store to manage configuration data, plain-text data, and secrets. +## Prerequisites -In this tutorial, we’ll demonstrate how to use Pulumi ESC as well as the power of this service in managing configuration and secrets. +1. **Create a Pulumi account** at [app.pulumi.com](https://app.pulumi.com) +1. **Install the ESC CLI** -Before you begin, you can watch the following video which provides a high level overview of how Pulumi ESC works: +{{< chooser os "macos,windows,linux" >}} -
- +{{% choosable os macos %}} + +```bash +brew update && brew install pulumi/tap/esc +``` + +{{% /choosable %}} + +{{% choosable os linux %}} + +```bash +curl -fsSL https://get.pulumi.com/esc/install.sh | sh +``` + +{{% /choosable %}} + +{{% choosable os windows %}} + +
+
+

Windows binary download

+

+amd64 +

+
-{{< get-started-stepper >}} +{{% /choosable %}} + +{{% /chooser %}} + +See the [ESC installation docs](/docs/install/esc/) for more options. + +## Create your first environment + +1. **Log in** to the ESC CLI: + +```bash +esc login +``` + +You'll be prompted to log in via your browser or with an access token. Follow the instructions to authenticate. + +1. **Create an environment** in the Pulumi Cloud console: + + 1. Open [Pulumi Cloud](https://app.pulumi.com) and log in + 1. Select **Environments** in the left navigation + 1. Select **+ Create Environment** + 1. Choose **New Environment** + 1. For **Project name**, enter: `my-project` + 1. For **Environment name**, enter: `dev` + 1. Select **Create Environment** + + ![Create Environment dialog in Pulumi Cloud showing fields for project name (my-project) and environment name (dev)](./images/esc-create-environment.png) + + Your environment is now created and ready to store configuration and secrets. + +## Store configuration and secrets + +Add both plaintext configuration and encrypted secrets to your environment: + +1. In the **Environment definition** editor, erase the contents and replace them with the following YAML: + + ```yaml + values: + region: us-west-2 + apiKey: + fn::secret: demo-secret-123 + ``` + + This defines two values: `region` (a plaintext value) and `apiKey` (a secret value, denoted with `fn::secret`). + + ![ESC environment editor showing YAML definition with region and apiKey values before saving](./images/esc-env-edit-pre-save.png) + +1. Select **Save** + + Watch what happens: ESC automatically encrypts the secret value. The plaintext `demo-secret-123` is replaced with an encrypted value. + + ![ESC environment editor showing encrypted secret value and preview displayed as [secret] after saving](./images/esc-env-edit-post-save.png) + +## Retrieve your configuration and secrets + +Open your environment to retrieve all values, including decrypted secrets: + +```bash +esc env open my-project/dev +``` + +You should see output like: + +```json +{ + "apiKey": "demo-secret-123", + "region": "us-west-2" +} +``` + +You've created an environment, stored configuration and secrets, and retrieved them interactively with the CLI. Notice that the secret is automatically decrypted when you open the environment. + +## What's next? + +### Use ESC in your infrastructure code + +Pulumi users typically integrate ESC with their IaC workflows to centralize secrets and configuration across all stacks. If you already use Pulumi IaC, learn how to reference ESC environments in your Pulumi programs: + +- **[Integrate ESC with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/)** + +New to Pulumi IaC? Start with the [Pulumi IaC Get Started guide](/docs/get-started/) first. + +### Explore other use cases + +- **[Understand the concepts](/docs/esc/concepts/)** - Learn how ESC works under the hood +- **[Set up OIDC](/docs/esc/guides/setting-up-oidc/)** - Generate short-lived cloud credentials dynamically +- **[Pull secrets from external sources](/docs/esc/guides/external-secrets/)** - Integrate with AWS Secrets Manager, Azure Key Vault, and more +- **[Compose environments](/docs/esc/guides/importing-environments/)** - Share configuration across teams and projects diff --git a/content/docs/esc/get-started/begin.md b/content/docs/esc/get-started/begin.md deleted file mode 100644 index 4eebd8f10484..000000000000 --- a/content/docs/esc/get-started/begin.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -title_tag: Before You Begin | Pulumi ESC -title: Before you begin -h1: "Pulumi ESC: Before you begin" -meta_desc: This page provides an overview on how to get started with Pulumi ESC. -weight: 2 -menu: - esc: - parent: esc-get-started - identifier: esc-get-started-begin -aliases: ---- - -Before you get started using Pulumi ESC, let's run through a few quick steps to ensure your environment is set up correctly. - -### Create a Pulumi account - -Pulumi ESC is a service of Pulumi Cloud, meaning you will need to create a Pulumi account to be able to use it. To do so, navigate to the [Pulumi Cloud console](https://app.pulumi.com) and create a new account. Once created, you can [optionally create an access token](/docs/administration/access-identity/access-tokens/). Doing so will provide you an alternative way to sign into the Pulumi Cloud via the CLI. The token can also be used to automate your usage of the Pulumi Cloud using the REST API. - -### Install the Pulumi ESC CLI - -{{< notes type="info" >}} -Pulumi ESC can be used with or without Pulumi IaC. This means that if you already have the [Pulumi IaC CLI](/docs/cli/) installed, you do not need to install the Pulumi ESC CLI, and you may substitute `pulumi env` anywhere you see the `esc env` command in the rest of this tutorial. -{{< /notes >}} - -Use the below option to install the Pulumi ESC CLI based on your operating system. - -{{< chooser os "macos,windows,linux" >}} - -{{% choosable os macos %}} - -```bash -$ brew update && brew install pulumi/tap/esc -``` - -{{% /choosable %}} - -{{% choosable os linux %}} - -```bash -$ curl -fsSL https://get.pulumi.com/esc/install.sh | sh -``` - -{{% /choosable %}} - -{{% choosable os windows %}} - -
-
-

Windows Binary Download

-

-amd64 -

-
-
- -{{% /choosable %}} - -{{% /chooser %}} - -You can explore more installation options by visiting the [ESC installation docs](/docs/install/esc/). - -### Login to the ESC CLI - -Run the following command to log into the CLI: - -```bash -esc login -``` - -You will be prompted to log in to the Pulumi Cloud using either the browser or by optionally providing an access token. - -```bash -$ esc login -Manage your Pulumi ESC environments by logging in. -Run `esc --help` for alternative login options. -Enter your access token from https://app.pulumi.com/account/tokens - or hit to log in using your browser : -Logged in to https://api.pulumi.com/ as your-pulumi-org (https://app.pulumi.com/your-pulumi-org) -``` - -### [Optional] Configure OpenID Connect (OIDC) - -Pulumi supports [OpenID Connect (OIDC) integration](/docs/esc/environments/configuring-oidc) across various services including Pulumi ESC. OIDC enables secure interactions between Pulumi and cloud providers by leveraging signed, short-lived tokens issued by the Pulumi Cloud. Use one of the following guides below to configure OIDC between Pulumi ESC and your chosen cloud provider: - -- [OIDC Configuration for AWS](/docs/esc/environments/configuring-oidc/aws/) -- [OIDC Configuration for Azure](/docs/esc/environments/configuring-oidc/azure/) -- [OIDC Configuration for Google Cloud](/docs/esc/environments/configuring-oidc/gcp/) - -This is an optional step that is not required to get started with Pulumi ESC. There are some steps in this series that will require OIDC configuration to complete, but that will be indicated on the relevant pages. - -In the next section, you will start your journey with Pulumi ESC by creating a new environment. - -{{< get-started-stepper >}} diff --git a/content/docs/esc/get-started/create-environment.md b/content/docs/esc/get-started/create-environment.md deleted file mode 100644 index 0e1935d89075..000000000000 --- a/content/docs/esc/get-started/create-environment.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -title_tag: Create a New Environment | Pulumi ESC -title: Create environment -h1: "Pulumi ESC: Create A New Environment" -meta_desc: This page provides an overview on how to create a new Pulumi ESC environment. -weight: 3 -menu: - esc: - parent: esc-get-started - identifier: esc-get-started-create-environment -aliases: ---- - -## Overview - -In Pulumi ESC, an environment is a collection of configuration intended to capture the configuration values needed to work with a particular environment. -Environments belong to a Project, which is a collection of related environments. - -An environment can be created one of two ways: - -- via the Pulumi Cloud console -- via the CLI - -This tutorial will walk you through how to create a new environment. - -## Create an environment - -### Create via the console - -To create an environment via the console, navigate to [Pulumi Cloud](https://app.pulumi.com) and select the **Environments** link in the left-hand menu. - -You will be directed to the Environments landing page. To create a new environment, click the **Create Environment** button. Enter a name for the project your environment should belong in (e.g. `my-project`) and a name for your environment (e.g., `dev-environment` for a development environment) and then click **Create Environment**. You will then be directed to the environment definition page. - -![Creating a new environment in the Pulumi ESC console](/docs/esc/assets/esc-create-environment.png) - -### Create via the CLI - -To create an environment via the CLI, use the `esc env init` command as shown below, where `` is optional and defaults to your Pulumi Cloud username. - -```bash -esc env init [/]/ -``` - -Note that environment names must be unique within a project and may only contain alphanumeric characters, hyphens, underscores, and periods. If you specify -an existing project the new environment will be created within it, otherwise a new project will be created. - -```bash -$ esc env init my-project/dev-environment -Environment created. -``` - -You can validate that your environment was created by running the `esc env ls` command which will list all of the environments that you have access to. - -```bash -$ esc env ls -myorg/my-project/dev-environment -``` - -In the next section, you will learn how to store configuration values and secrets in your environment. - -{{< get-started-stepper >}} diff --git a/content/docs/esc/get-started/images/esc-create-environment.png b/content/docs/esc/get-started/images/esc-create-environment.png new file mode 100644 index 000000000000..d59d6fe58a78 Binary files /dev/null and b/content/docs/esc/get-started/images/esc-create-environment.png differ diff --git a/content/docs/esc/get-started/images/esc-env-edit-post-save.png b/content/docs/esc/get-started/images/esc-env-edit-post-save.png new file mode 100644 index 000000000000..aafa175b51f8 Binary files /dev/null and b/content/docs/esc/get-started/images/esc-env-edit-post-save.png differ diff --git a/content/docs/esc/get-started/images/esc-env-edit-pre-save.png b/content/docs/esc/get-started/images/esc-env-edit-pre-save.png new file mode 100644 index 000000000000..a76dc409cb5b Binary files /dev/null and b/content/docs/esc/get-started/images/esc-env-edit-pre-save.png differ diff --git a/content/docs/esc/get-started/integrate-with-pulumi-iac.md b/content/docs/esc/get-started/integrate-with-pulumi-iac.md deleted file mode 100644 index 44ad8e871a84..000000000000 --- a/content/docs/esc/get-started/integrate-with-pulumi-iac.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -title_tag: Integrate with Pulumi IaC | Pulumi ESC -title: Integrate with Pulumi IaC -h1: "Pulumi ESC: Integrate with Pulumi IaC" -meta_desc: This page provides an overview on how to use Pulumi ESC with Pulumi IaC. -weight: 8 -menu: - esc: - parent: esc-get-started - identifier: esc-get-started-integrate-with-pulumi-iac -aliases: ---- - -## Overview - -Pulumi ESC works independently of [Pulumi Infrastructure as Code (IaC)](/docs/get-started/), providing developers the flexibility to centrally manage their environment configuration regardless of how or where those environments are created. - -Pulumi ESC also integrates seamlessly with Pulumi IaC, and this tutorial will demonstrate how to leverage Pulumi ESC in your own Pulumi programs. This works everywhere, including Pulumi Deployments and GitHub Actions, without any additional work or changes. - -## Prerequisites - -To complete the steps in this tutorial, you will need to install the following prerequisites: - -- the [Pulumi IaC CLI](/docs/cli/) -- one of [Pulumi's supported language runtimes](/docs/languages-sdks/) - -### Create and Configure a New Project - -Once the prerequisites are installed, run the `pulumi new ` command in an empty folder to [create a new Pulumi project](/docs/cli/commands/pulumi_new/), making sure to replace `` with the supported language of your choice: - -```bash -# example using python -$ pulumi new python -This command will walk you through creating a new Pulumi project. - -Enter a value or leave blank to accept the (default), and press . -Press ^C at any time to quit. - -project name (pulumi-esc-iac): -project description (A minimal Python Pulumi program): -Created project 'pulumi-esc-iac' - -Please enter your desired stack name. -To create a stack in an organization, use the format / (e.g. `acmecorp/dev`). -stack name (dev): pulumi/dev -Created stack 'dev' - -Installing dependencies... - -Creating virtual environment... -Finished creating virtual environment -Updating pip, setuptools, and wheel in virtual environment... -... -... -Finished installing dependencies - -Your new project is ready to go! - -To perform an initial deployment, run `pulumi up` -``` - -## Access configuration locally - -In a Pulumi project, you can locally [store and retrieve configuration values](/docs/concepts/config/) using the `pulumi config set [value]` command. Run the following command to create a config value with a key of `myEnvironment` and a value of `development`: - -```bash -$ pulumi config set myEnvironment development -``` - -Now run the following command to create a secret config value with a key of `myPassword` and a value of `demo-password-123`: - -```bash -$ pulumi config set myPassword demo-password-123 --secret -``` - -These values will be stored in your project's stack settings file `Pulumi..yaml` as shown below: - -```yaml -# Contents of Pulumi..yaml file -config: - pulumi-esc-iac:myEnvironment: development - pulumi-esc-iac:myPassword: - secure: AAABADd5YzRaVuzxM08i5z2CJ3LGkQau5e5Lhk+1Gtj37qv6zKkFr8KxmN6X+w/XMg== -``` - -You can retrieve the value of this configuration via the CLI by running the `pulumi config get ` command as shown below: - -```bash -# example output -$ pulumi config get myEnvironment -development -``` - -You can also import and access this configuration value from directly within your Pulumi program as shown below: - -{{< example-program path="aws-import-export-pulumi-config" >}} - -Run the `pulumi preview` command as shown below to validate the retrieval of the configuration: - -```bash -$ pulumi preview -Previewing update (dev) - -Loading policy packs... - - Type Name Plan - + pulumi:pulumi:Stack pulumi-esc-iac-dev create - -Policies: - ✅ pulumi-internal-policies@v0.0.6 - -Outputs: # the outputted config values - Environment: "development" - Password : [secret] - -Resources: - + 1 to create -``` - -Defining the configuration via the project stack settings file may be fine when dealing with a singular project, but it can become very challenging to maintain securely and consistently across multiple projects. Centralizing these configuration values using Pulumi ESC provides more scalability without increasing administrative overhead along the way. - -## Access configuration from ESC - -To centralize this configuration and make it accessible to your Pulumi program, you will need to add a second-level key named `pulumiConfig` in your environment file that will expose the values nested underneath it to Pulumi IaC. Add the following configuration to your environment file: - -```yaml -values: - pulumiConfig: - myEnvironment: development -``` - -From here, you will need to import your environment file into your Pulumi project. To do this, return to your `Pulumi..yaml` file and update it to import your environment as shown below, making sure to replace the value of `/` with the identifier of your own environment: - -```yaml -environment: - - / -``` - -This will import any configuration values that you have defined under the `pulumiConfig` key in your environment file and make them accessible to your Pulumi project. - -Save the file and then run the `pulumi config get ` command as shown below to quickly test that the import is working: - -```bash -# example output -$ pulumi config get myEnvironment -development -``` - -Now run the `pulumi preview` command again to test that the imported configuration value is accessible from within your Pulumi program: - -```bash -$ pulumi preview -Previewing update (dev) - -Loading policy packs... - - Type Name Plan - + pulumi:pulumi:Stack pulumi-esc-iac-dev create - -Policies: - ✅ pulumi-internal-policies@v0.0.6 - -Outputs: - Value: "development" - -Resources: - + 1 to create -``` - -See the Pulumi documentation on [Accessing Configuration from Code](/docs/concepts/config/#code) for more details. - -{{< get-started-stepper >}} diff --git a/content/docs/esc/get-started/store-and-retrieve-secrets.md b/content/docs/esc/get-started/store-and-retrieve-secrets.md deleted file mode 100644 index ff4c28ff9662..000000000000 --- a/content/docs/esc/get-started/store-and-retrieve-secrets.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -title_tag: Store and Retrieve Secrets | Pulumi ESC -title: Store and retrieve secrets -h1: "Pulumi ESC: Store and Retrieve Secrets" -meta_desc: This page provides an overview on how to store and retrieve secrets in Pulumi ESC. -weight: 4 -menu: - esc: - parent: esc-get-started - identifier: esc-get-started-store-retrieve-secrets -aliases: ---- - -In an environment file, values are defined as a series of key-value pairs in YAML format. All variables will be defined under a top-level key named `values`. These values can be strings, numbers, or arrays, and they can be manually provided, dynamically generated from external sources, or referenced from other values in the file. They can also be stored in plain-text or as secrets. - -```yaml -values: - myKey1: "myValue1" - myNestedKey: - myKey2: "myValue2" - myNumber: 1 - myPassword: - fn::secret: - ciphertext: ZXN.... -``` - -You can store and retrieve values in an environment via one of the following methods: - -- the Pulumi Cloud console Document view -- the Pulumi Cloud console Table view -- the ESC CLI - -## Store Environment values - -### Store via the document view - -To store values in your environment using the Document view, first click on the name of the environment to open it. You will be presented with a split pane view. The left side is the YAML-based code document view, and this is where you will write the definition of your environment configuration. The right side will show a preview of your configuration in JSON format. - -![Open environment in Pulumi ESC console](/docs/esc/assets/esc-open-env.png) - -Next, delete the placeholder text in the environment file and add the following simple configuration definition in its place: - -```yaml -values: - myEnvironment: "development" - myPassword: - fn::secret: "demo-password-123" -``` - -As shown above, you can specify that a value should be stored as a secret by using the `fn::secret` function. Once you have added the configuration, click the **Save** button located at the bottom of the editor. - -{{< video title="Adding values to the environment in the Pulumi ESC console" src="https://www.pulumi.com/uploads/esc-add-env-values.mp4" autoplay="true" loop="true" >}} - -The **Environment preview** pane on the right-hand side will then update to show your added configuration in JSON format. You will notice that the value of "myPassword" has been hidden from view in both the definition and preview panes. - -### Store via the Table view - -To store values in your environment using the Document view, first click on the name of the environment to open it. From the **Editor** menu, click the **Table view** button to switch the editor from the document YAML view to the table view. - -{{< video title="Adding values to the environment in the Pulumi ESC console" src="https://www.pulumi.com/uploads/esc-open-env-table-view.mp4" autoplay="true" loop="true" >}} - -Next, under the **Configuration** section, you will see fields labeled **Path** and **Value**. You will define your configuration values using these fields. In the **Path** field, type in `myEnvironment`, and in the **Value** field, type in `development`. Then click **Done** and **Save**. - -![Adding values to the environment in the Pulumi ESC console](/docs/esc/assets/esc-add-env-values.png) - -Next, click the **+Secret/config** button to create a new configuration. This time, you will create a configuration that will be stored as a secret. To do so, enter `myPassword` for the value of **Path** and `demo-password-123` for the value of **Value**. Select the **secret** option from the drop-down to indicate that this configuration will be stored as a secret, then click **Done** and **Save**. - -![Adding values to the environment in the Pulumi ESC console](/docs/esc/assets/esc-add-secret-value.png) - -You will notice that the value of `myPassword` is hidden from view after saving. - -### Store via the CLI - -To store values or update an existing value via the CLI, use the `esc env set` command as shown below, where `` is optional and defaults to your Pulumi Cloud username: - -```bash -esc env set [/]/ -``` - -To demonstrate how this works, add the following simple configuration definition to your environment using the following command, making sure to replace the value of `my-project/dev-environment` with the name of your own environment: - -```bash -esc env set my-project/dev-environment myEnvironment development -esc env set my-project/dev-environment myPassword demo-password-123 --secret -``` - -As shown above, you can specify that a value should be stored as a secret by using the `--secret` flag. - -Alternatively, you can directly [edit your environment file with a code editor](/docs/pulumi-cloud/esc/environments/#with-the-pulumi-esc-cli) using the following command, making sure to replace `/` with the identifier of your own environment (e.g. `my-project/dev-environment`): - -```bash -esc env edit / -``` - -Using this method enables you to add your configuration values in the same way that you would [via the console](/docs/esc/get-started/store-and-retrieve-secrets/#store-via-the-console). - -## Retrieve Environment values - -### Retrieve via the Document view - -To retrieve values using the Document view, click the **Open** button. This will return any statically defined plain-text values and definitions. - -![Clicking the open button in the Pulumi ESC console](/docs/esc/assets/esc-open-environment.png) - -As shown above, it does not return the value of secrets defined, nor does it resolve values that are dynamically generated from a provider. To view these values, you will need to click the **Show secrets** slider. - -![Clicking the show secrets slider the Pulumi ESC console](/docs/esc/assets/esc-show-secret-document-view.png) - -### Retrieve via the Table view - -Non-secret configuration values remain visible in the Table view after their creation, but secret values are automatically hidden. To reveal the value of a secret using the Table view, click the small eye icon. - -{{< video title="Clicking the show secrets slider the Pulumi ESC console" src="https://www.pulumi.com/uploads/esc-show-secret-table-view.mp4" autoplay="true" loop="true" >}} - -### Retrieve via the CLI - -The CLI has a built-in `get` command that enables you to retrieve a single value from your environment. The format of the full command looks like the following: - -```bash -esc env get [/]/ -``` - -To retrieve the value of the `myEnvironment` variable you created earlier, the command to do so would look like the following, making sure to replace the value of `my-project/dev-environment` with the identifier of your own environment: - -```bash -esc env get my-project/dev-environment myEnvironment -``` - -Running this command should return the following response: - -```bash -$ esc env get my-project/dev-environment myEnvironment - - Value - - "development" - - Definition - - development - - Defined at - - • my-project/dev-environment:2:8 -``` - -It is also possible to retrieve all values in an environment. To do so, run the `esc env get` command without specifying a value as shown below: - -```bash -esc env get my-project/dev-environment -``` - -Running this command should return the following response: - -```bash -$ esc env get my-project/dev-environment - - Value - - { - "myEnvironment": "development", - "myPassword": "[secret]" - } - - Definition - - values: - myEnvironment: "development" - myPassword: - fn::secret: - ciphertext: ZXNjeAA.... - -``` - -The `esc env get` command only returns statically defined plain-text values and definitions. This means that it does not return the value of any defined secrets, nor does it resolve values that are dynamically generated from a provider. To view these values, you must run the `esc env open` command as shown below. This will open the environment and resolve any secrets or dynamically retrieved values: - -```bash -$ esc env open my-project/dev-environment - -{ - "myEnvironment": "development", - "myPassword": "demo-password-123" -} - -``` - -In the next section, you will learn how to import configuration values from other environments. - -{{< get-started-stepper >}} diff --git a/content/docs/esc/guides/_index.md b/content/docs/esc/guides/_index.md new file mode 100644 index 000000000000..1a09482f77d1 --- /dev/null +++ b/content/docs/esc/guides/_index.md @@ -0,0 +1,34 @@ +--- +title: Guides +title_tag: Pulumi ESC Guides +h1: Pulumi ESC Guides +meta_desc: Step-by-step guides for common Pulumi ESC tasks including IaC integration, OIDC setup, and secrets management. +menu: + esc: + parent: esc-home + identifier: esc-guides + weight: 4 +--- + +These guides walk you through common Pulumi ESC tasks. Each guide is self-contained and provides step-by-step instructions for accomplishing a specific goal. + +## Getting started guides + +- **[Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/)** - Use ESC environments in your Pulumi infrastructure code +- **[Managing secrets](/docs/esc/guides/managing-secrets/)** - Store, retrieve, and organize secrets in ESC environments +- **[Running commands with esc run](/docs/esc/guides/running-commands-with-esc/)** - Inject secrets into any command or script + +## Advanced configuration + +- **[Importing environments](/docs/esc/guides/importing-environments/)** - Compose environments and share configuration across teams +- **[Setting up OIDC](/docs/esc/guides/setting-up-oidc/)** - Generate short-lived cloud credentials dynamically +- **[Integrating external secrets](/docs/esc/guides/external-secrets/)** - Pull secrets from AWS Secrets Manager, Azure Key Vault, and more + +## Reference documentation + +For comprehensive syntax and feature documentation, see: + +- [Environments reference](/docs/esc/environments/) - Complete environment syntax and features +- [Integrations](/docs/esc/integrations/) - Provider-specific documentation +- [CLI reference](/docs/esc/cli/) - Command-line tool documentation +- [API and SDKs](/docs/esc/development/) - Programmatic access to ESC diff --git a/content/docs/esc/get-started/retrieve-external-secrets.md b/content/docs/esc/guides/external-secrets.md similarity index 74% rename from content/docs/esc/get-started/retrieve-external-secrets.md rename to content/docs/esc/guides/external-secrets.md index 4fd95dfcc735..599689892ba1 100644 --- a/content/docs/esc/get-started/retrieve-external-secrets.md +++ b/content/docs/esc/guides/external-secrets.md @@ -1,14 +1,16 @@ --- -title_tag: Retrieve External Secrets | Pulumi ESC -title: Retrieve secrets from external sources -h1: "Pulumi ESC: Retrieve Secrets from External Sources" -meta_desc: This page provides an overview on how to retrieve secrets from external sources. -weight: 7 +title_tag: Integrating External Secrets | Pulumi ESC +title: Integrating External Secrets +h1: Integrating External Secrets with Pulumi ESC +meta_desc: Learn how to integrate Pulumi ESC with external secret providers like AWS Secrets Manager, Azure Key Vault, and Google Cloud Secret Manager. menu: esc: - parent: esc-get-started - identifier: esc-get-started-retrieve-external-secrets + parent: esc-guides + identifier: esc-guides-external-secrets + weight: 6 aliases: + - /docs/esc/get-started/retrieve-external-secrets/ + - /docs/pulumi-cloud/esc/get-started/retrieve-external-secrets/ --- ## Overview @@ -23,7 +25,7 @@ This enables you to centralize both new and existing secrets and configurations ## Import external secrets -If you have not done so already, make sure you have [configured OIDC connectivity](/docs/esc/get-started/begin/#configure-openid-connect-oidc) between Pulumi and a supported provider listed below. During the configuration, you will need to make sure that you add the permissions necessary to interact with secrets in your chosen provider. +If you have not done so already, make sure you have [configured OIDC connectivity](/docs/esc/guides/setting-up-oidc/) between Pulumi and a supported provider listed below. During the configuration, you will need to make sure that you add the permissions necessary to interact with secrets in your chosen provider. {{% chooser cloud "aws,azure,gcp" / %}} @@ -76,14 +78,14 @@ values: secretId: my-app-secret ``` -You can validate this configuration by [opening the environment via the ESC console](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-console), clicking the **Open** button and then clicking the **Show secrets** slider. +You can validate this configuration by [opening the environment via the ESC console](/docs/esc/guides/managing-secrets/#via-the-pulumi-cloud-console), clicking the **Open** button and then clicking the **Show secrets** slider. -Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. +Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/guides/managing-secrets/#via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. ```bash $ esc env open pulumi/demos/aws-secrets-example @@ -173,14 +175,14 @@ values: name: my-app-secret ``` -You can validate this configuration by [opening the environment via the ESC console](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-console), clicking the **Open** button and then clicking the **Show secrets** slider. +You can validate this configuration by [opening the environment via the ESC console](/docs/esc/guides/managing-secrets/#via-the-pulumi-cloud-console), clicking the **Open** button and then clicking the **Show secrets** slider. -Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. +Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/guides/managing-secrets/#via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. ```bash $ esc env open pulumi/demos/azure-secrets-example @@ -269,14 +271,14 @@ values: name: my-app-secret ``` -You can validate this configuration by [opening the environment via the ESC console](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-console), clicking the **Open** button and then clicking the **Show secrets** slider. +You can validate this configuration by [opening the environment via the ESC console](/docs/esc/guides/managing-secrets/#via-the-pulumi-cloud-console), clicking the **Open** button and then clicking the **Show secrets** slider. -Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. +Alternatively, you can validate the configuration by [opening the environment via the ESC CLI](/docs/esc/guides/managing-secrets/#via-the-cli). Run the `esc env open //` command, making sure to replace the values of ``, ``, and `` with the names of your Pulumi organization, ESC project, and environment respectively. ```bash $ esc env open pulumi/demos/gcp-secrets-example @@ -315,4 +317,16 @@ values: {{% /choosable %}} -{{< get-started-stepper >}} +## Next steps + +- [Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/) - Use external secrets in your infrastructure code +- [Setting up OIDC](/docs/esc/guides/setting-up-oidc/) - Learn more about configuring OIDC for dynamic credentials +- [Managing secrets](/docs/esc/guides/managing-secrets/) - Understand how to organize and retrieve secrets + +### Provider-specific documentation + +For complete documentation on each secrets provider: + +- [AWS Secrets Manager integration](/docs/esc/integrations/dynamic-secrets/aws-secrets/) +- [Azure Key Vault integration](/docs/esc/integrations/dynamic-secrets/azure-secrets/) +- [Google Cloud Secret Manager integration](/docs/esc/integrations/dynamic-secrets/gcp-secrets/) diff --git a/content/docs/esc/get-started/import-environments.md b/content/docs/esc/guides/importing-environments.md similarity index 74% rename from content/docs/esc/get-started/import-environments.md rename to content/docs/esc/guides/importing-environments.md index 512ef8773bfa..078f48a3311a 100644 --- a/content/docs/esc/get-started/import-environments.md +++ b/content/docs/esc/guides/importing-environments.md @@ -1,14 +1,16 @@ --- title_tag: Import Environments | Pulumi ESC -title: Import environments -h1: "Pulumi ESC: Import Environments" -meta_desc: This page provides an overview on how to import environments in Pulumi ESC. -weight: 5 +title: Importing Environments +h1: Importing Environments +meta_desc: Learn how to import and compose Pulumi ESC environments to share configuration across teams and projects. menu: esc: - parent: esc-get-started - identifier: esc-get-started-import-environments + parent: esc-guides + identifier: esc-guides-importing-environments + weight: 5 aliases: + - /docs/esc/get-started/import-environments/ + - /docs/pulumi-cloud/esc/get-started/import-environments/ --- ## Overview @@ -21,7 +23,7 @@ There may be scenarios where the value you need to retrieve is stored in a diffe In the development environment, you might be integrating with the sandbox or development endpoint of the third-party service, while in the testing environment, you might be integrating with the production endpoint. Both endpoints may share the same base URL. -![Image showing same base URL and different endpoints](/docs/esc/get-started/esc-base-url.png) +![Image showing same base URL and different endpoints](/docs/esc/assets/esc-base-url.png) Since this base URL would be the same across environments, it would be more efficient to define it once in one place rather than multiple times across multiple environment files. @@ -72,21 +74,23 @@ You should now see `"ENDPOINT_URL": "https://wordsapiv1.p.rapidapi.com/"` in the ![Importing the global config environment in the Pulumi Console](/docs/esc/assets/esc-import-environments2.png) {{% notes type="info" %}} -You can test this out by retrieving the imported value via the console or the CLI. Refer to the [Store and Retrieve Secrets guide](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-environment-values) for the steps on how to do this. +You can test this out by retrieving the imported value via the console or the CLI. See [Managing secrets](/docs/esc/guides/managing-secrets/#retrieving-secrets) for details on retrieving values. {{% /notes %}} ### Import via the Table view -Alternatively, you can import environments using the Table view of the Pulumi console. Navigate to the **Table view** of your environment and click the **Import** button under the **Imports** section. You will be presented with a dropdown list of environments that you can import. Search for the name of the environment you want to import and select it. Then click **Import**. +Alternatively, you can import environments using the Table view of the Pulumi console. Navigate to the **Table view** of your environment and select the **Import** button under the **Imports** section. You will be presented with a dropdown list of environments that you can import. Search for the name of the environment you want to import and select it. Then select **Import**. ![Importing the global config environment in the Pulumi Console](/docs/esc/assets/esc-import-pulumi-console.png) -To view the imported values, you will need to [open your environment](/docs/esc/get-started/store-and-retrieve-secrets/#retrieve-environment-values). +To view the imported values, you will need to [open your environment](/docs/esc/guides/managing-secrets/#retrieving-secrets). {{% notes type="info" %}} You can import multiple environments and reorder them. The order of the imported environments is important because if you have two environments that have the same name for a key/path, then the last imported environment will override the value of that key/paths for any environments that are imported before it. This behavior can be useful for scenarios where you need to intentionally override the value of a particular key/path. {{% /notes %}} -In the next section, you will learn how to run local commands without manually configuring local secrets. +## Next steps -{{< get-started-stepper >}} +- [Managing secrets](/docs/esc/guides/managing-secrets/) - Learn more about storing and retrieving values +- [Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/) - Use composed environments in your infrastructure code +- [Composition and imports reference](/docs/esc/environments/imports/) - Complete documentation on import behavior and JSON Merge Patch semantics diff --git a/content/docs/esc/guides/integrate-with-pulumi-iac.md b/content/docs/esc/guides/integrate-with-pulumi-iac.md new file mode 100644 index 000000000000..6afc56deac03 --- /dev/null +++ b/content/docs/esc/guides/integrate-with-pulumi-iac.md @@ -0,0 +1,161 @@ +--- +title_tag: Integrate ESC with Pulumi IaC +title: Integrate with Pulumi IaC +h1: Integrate ESC with Pulumi IaC +meta_desc: Learn how to use Pulumi ESC environments in your Pulumi infrastructure as code projects to centralize secrets and configuration. +menu: + esc: + parent: esc-guides + identifier: esc-guides-integrate-pulumi-iac + weight: 1 +aliases: + - /docs/esc/get-started/integrate-with-pulumi-iac/ + - /docs/pulumi-cloud/esc/get-started/integrate-with-pulumi-iac/ +--- + +This guide shows you how to integrate Pulumi ESC with your Pulumi IaC projects to centralize configuration and secrets across all your stacks. + +{{< notes type="info" >}} +**This guide is for existing Pulumi IaC users.** If you're new to Pulumi IaC, start with the [Pulumi IaC Get Started guide](/docs/get-started/) first. +{{< /notes >}} + +If you completed the [ESC Get Started guide](/docs/esc/get-started/), you already have an environment with values like `region` and `apiKey`. This guide shows you how to reference that same environment from your Pulumi stack configuration. + +## Prerequisites + +- [Pulumi CLI](/docs/install/) installed +- [Pulumi account](https://app.pulumi.com/signup) created +- An existing Pulumi project (or create one with `pulumi new`) +- An ESC environment with configuration values (see [Get Started](/docs/esc/get-started/) to create one) + +## Add ESC to your Pulumi stack + +### Step 1: Reference your ESC environment + +In your stack configuration file (`Pulumi..yaml`), add an `environment` block that references your ESC environment: + +```yaml +environment: + - / +``` + +For example, if your ESC environment is `my-project/dev`: + +```yaml +environment: + - my-project/dev +``` + +You can also reference multiple environments, which will be merged in order (later values override earlier ones): + +```yaml +environment: + - my-project/common + - my-project/dev +``` + +### Step 2: Define configuration in your ESC environment + +ESC environments are YAML documents that you can edit using the CLI or Pulumi Cloud console. Use the CLI to edit your environment: + +```bash +esc env edit / +``` + +You can also edit environments in the [Pulumi Cloud console](https://app.pulumi.com) if you prefer a visual editor. + +In your ESC environment, use the `pulumiConfig` block to expose values to Pulumi IaC. If you're using the environment from the Get Started guide, wrap your existing values in a `pulumiConfig` block: + +```yaml +values: + pulumiConfig: + region: us-west-2 + apiKey: + fn::secret: demo-secret-123 +``` + +Values in `pulumiConfig` can be strings, numbers, booleans, or secrets (using `fn::secret`). + +The `pulumiConfig` block is the bridge between ESC and Pulumi IaC. Any values you define under `pulumiConfig` become available in your Pulumi program through the standard Configuration API. + +This allows you to centralize all your configuration and secrets in ESC while accessing them through familiar Pulumi config patterns like `config.get()` or `config.require()`. + +### Step 3: Access configuration in your code + +Use Pulumi's standard Configuration API to access these values in your infrastructure code: + +{{< example-program path="aws-import-export-pulumi-config" >}} + +Your Pulumi program now retrieves configuration and secrets from ESC. Run `pulumi preview` or `pulumi up` to see it in action. + +## Common patterns + +### Using dynamic cloud credentials + +To share AWS OIDC credentials across multiple stacks, configure your ESC environment to generate short-lived credentials: + +```yaml +values: + aws: + login: + fn::open::aws-login: + oidc: + roleArn: arn:aws:iam::123456789012:role/pulumi-deployment-role + sessionName: pulumi-session + pulumiConfig: + aws:region: ${aws.login.region} + environmentVariables: + AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId} + AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey} + AWS_SESSION_TOKEN: ${aws.login.sessionToken} +``` + +This pattern works everywhere Pulumi runs: locally, in CI/CD, Pulumi Deployments, and GitHub Actions. Similar patterns are available for Azure (`fn::open::azure-login`) and GCP (`fn::open::gcp-login`). + +Learn more in [Setting up OIDC](/docs/esc/guides/setting-up-oidc/). + +### Managing API keys and secrets + +Pull third-party API keys from external secret stores: + +```yaml +values: + pulumiConfig: + myApp:datadogApiKey: + fn::secret: + fn::open::azure-secrets: + login: ${azure.login} + get: + secretId: https://my-keyvault.vault.azure.net/secrets/datadog-api-key +``` + +Learn more in [Integrating external secrets](/docs/esc/guides/external-secrets/). + +### Environment-specific configuration + +Compose environments to share common configuration while overriding values per environment: + +```yaml +# common environment +values: + pulumiConfig: + myApp:instanceType: t3.micro + myApp:replicas: 1 + +# production environment (imports common) +imports: + - common +values: + pulumiConfig: + myApp:instanceType: t3.large # override for production + myApp:replicas: 3 # override for production +``` + +Learn more in [Importing environments](/docs/esc/guides/importing-environments/). + +## Next steps + +- [Setting up OIDC](/docs/esc/guides/setting-up-oidc/) - Generate dynamic cloud credentials +- [Integrating external secrets](/docs/esc/guides/external-secrets/) - Pull from AWS, Azure, GCP vaults +- [Importing environments](/docs/esc/guides/importing-environments/) - Compose configuration hierarchies +- [Pulumi IaC integration reference](/docs/esc/integrations/infrastructure/pulumi-iac/) - Complete integration documentation diff --git a/content/docs/esc/guides/managing-secrets.md b/content/docs/esc/guides/managing-secrets.md new file mode 100644 index 000000000000..a0ec878f917a --- /dev/null +++ b/content/docs/esc/guides/managing-secrets.md @@ -0,0 +1,194 @@ +--- +title_tag: Managing Secrets in Pulumi ESC +title: Managing Secrets +h1: Managing Secrets in Pulumi ESC +meta_desc: Learn how to store, retrieve, and organize secrets in Pulumi ESC environments using the CLI and Pulumi Cloud console. +menu: + esc: + parent: esc-guides + identifier: esc-guides-managing-secrets + weight: 2 +aliases: + - /docs/esc/get-started/store-and-retrieve-secrets/ + - /docs/pulumi-cloud/esc/get-started/store-and-retrieve-secrets/ +--- + +This guide shows you how to store and retrieve secrets in Pulumi ESC environments. Secrets are encrypted values that ESC stores securely and hides from view by default. + +## Prerequisites + +- [ESC CLI](/docs/install/esc/) installed +- [Pulumi account](https://app.pulumi.com/signup) created +- An ESC environment (create one with `esc env init //`) + +## Understanding ESC values + +ESC environments store configuration as key-value pairs in YAML format. All values are defined under a top-level `values` key: + +```yaml +values: + apiEndpoint: https://api.example.com + region: us-west-2 + apiKey: + fn::secret: my-secret-value +``` + +Values can be: + +- **Plain text** - Regular configuration values (region, endpoint URLs) +- **Secrets** - Encrypted values marked with `fn::secret` +- **Structured data** - Objects and arrays +- **Dynamic values** - Generated from providers (covered in other guides) + +## Storing secrets + +### Via the CLI + +Add a secret to your environment using the `--secret` flag: + +```bash +esc env set // apiKey my-secret-value --secret +``` + +For example: + +```bash +esc env set my-org/my-project/dev apiKey demo-secret-123 --secret +``` + +This encrypts the value before storing it. + +### Via the Pulumi Cloud console + +1. Navigate to [Pulumi Cloud](https://app.pulumi.com) +1. Select **Environments** in the left navigation +1. Select your environment +1. In the editor, add your secret using the `fn::secret` function: + +```yaml +values: + apiKey: + fn::secret: my-secret-value +``` + +1. Select **Save** + +The console will encrypt the secret and replace the plaintext value with a ciphertext reference: + +```yaml +values: + apiKey: + fn::secret: + ciphertext: ZXNjeAA... +``` + +## Retrieving secrets + +### Via the CLI + +Open your environment to retrieve all values, including secrets: + +```bash +esc env open // +``` + +This returns all values in JSON format with secrets revealed: + +```json +{ + "apiKey": "my-secret-value", + "region": "us-west-2" +} +``` + +To retrieve a single value: + +```bash +esc env get // apiKey +``` + +### Via the Pulumi Cloud console + +1. Select your environment in Pulumi Cloud +1. Select **Open** to evaluate the environment +1. Toggle **Show secrets** to reveal encrypted values + +## Organizing secrets + +### Nested structure + +Group related secrets using nested keys: + +```yaml +values: + database: + host: db.example.com + password: + fn::secret: db-password-123 + port: 5432 + api: + key: + fn::secret: api-key-456 + endpoint: https://api.example.com +``` + +Access nested values with dot notation: + +```bash +esc env get my-org/my-project/dev database.password +``` + +### Multiple environments + +Organize secrets by environment (dev, staging, production) using separate ESC environments: + +- `my-org/my-project/dev` - Development secrets +- `my-org/my-project/staging` - Staging secrets +- `my-org/my-project/prod` - Production secrets + +Each environment can have different RBAC permissions, ensuring production secrets are only accessible to authorized users. + +## Best practices + +### Use secrets for sensitive data + +Mark these values as secrets: + +- API keys and tokens +- Database passwords +- Private keys and certificates +- OAuth client secrets + +### Use plain text for non-sensitive data + +These can be plain text: + +- Region names +- Public endpoints +- Feature flags +- Port numbers + +### Rotate secrets regularly + +Update secrets by setting new values: + +```bash +esc env set my-org/my-project/prod apiKey new-secret-value --secret +``` + +ESC versions every change, allowing you to roll back if needed. + +### Control access with RBAC + +Use [Role-Based Access Control](/docs/esc/administration/access-control/) to limit who can read or write secrets: + +- Grant teams read-only access to production secrets +- Allow developers full access to development secrets +- Use service accounts for CI/CD access + +## Next steps + +- [Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/) - Use secrets in your infrastructure code +- [Integrating external secrets](/docs/esc/guides/external-secrets/) - Pull secrets from AWS, Azure, GCP vaults +- [Running commands with esc run](/docs/esc/guides/running-commands-with-esc/) - Inject secrets into any command +- [Access control reference](/docs/esc/environments/access-control/) - Complete RBAC documentation diff --git a/content/docs/esc/guides/running-commands-with-esc.md b/content/docs/esc/guides/running-commands-with-esc.md new file mode 100644 index 000000000000..fe4f3d022386 --- /dev/null +++ b/content/docs/esc/guides/running-commands-with-esc.md @@ -0,0 +1,225 @@ +--- +title_tag: Running Commands with ESC +title: Running Commands with esc run +h1: Running Commands with esc run +meta_desc: Learn how to use esc run to inject secrets and configuration from Pulumi ESC into any command or script. +menu: + esc: + parent: esc-guides + identifier: esc-guides-running-commands + weight: 3 +--- + +This guide shows you how to use `esc run` to inject secrets and configuration from ESC environments into any command or script as environment variables. + +## Prerequisites + +- [ESC CLI](/docs/install/esc/) installed +- [Pulumi account](https://app.pulumi.com/signup) created +- An ESC environment with values (see [Managing secrets](/docs/esc/guides/managing-secrets/)) + +## Basic usage + +The `esc run` command opens an ESC environment, exports values as environment variables, and runs a command with those variables: + +```bash +esc run // -- +``` + +For example, if your environment contains: + +```yaml +values: + environmentVariables: + DATABASE_URL: postgres://localhost/mydb + API_KEY: + fn::secret: my-api-key-123 +``` + +Run a script with these variables: + +```bash +esc run my-org/my-project/dev -- node app.js +``` + +The `app.js` script will have access to `DATABASE_URL` and `API_KEY` as environment variables. + +## Using the environmentVariables block + +ESC exports values to environment variables using the `environmentVariables` block: + +```yaml +values: + database: + host: db.example.com + password: + fn::secret: db-password-123 + + environmentVariables: + DB_HOST: ${database.host} + DB_PASSWORD: ${database.password} +``` + +The `environmentVariables` block: + +- Maps ESC values to environment variable names +- Uses interpolation (`${...}`) to reference other values +- Automatically includes secrets (they're revealed when the environment is opened) + +## Common patterns + +### Running cloud CLI commands + +Inject cloud credentials into CLI commands: + +```yaml +values: + aws: + login: + fn::open::aws-login: + oidc: + roleArn: arn:aws:iam::123456789012:role/my-role + sessionName: esc-session + + environmentVariables: + AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId} + AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey} + AWS_SESSION_TOKEN: ${aws.login.sessionToken} +``` + +Run AWS CLI commands: + +```bash +esc run my-org/my-project/aws-prod -- aws s3 ls +esc run my-org/my-project/aws-prod -- aws ec2 describe-instances +``` + +The same pattern works for other cloud CLIs: + +```bash +esc run my-org/my-project/azure-prod -- az vm list +esc run my-org/my-project/gcp-prod -- gcloud compute instances list +``` + +### Running tests with secrets + +Configure test environments with secrets: + +```yaml +values: + environmentVariables: + TEST_DATABASE_URL: postgres://localhost/test_db + TEST_API_KEY: + fn::secret: test-api-key + TEST_WEBHOOK_SECRET: + fn::secret: test-webhook-secret +``` + +Run tests: + +```bash +esc run my-org/my-project/test -- npm test +esc run my-org/my-project/test -- pytest +esc run my-org/my-project/test -- go test ./... +``` + +### Running deployment scripts + +Inject credentials for deployment scripts: + +```yaml +values: + environmentVariables: + DEPLOY_TOKEN: + fn::secret: github-deploy-token + DOCKER_REGISTRY: myregistry.azurecr.io + DOCKER_USERNAME: myuser + DOCKER_PASSWORD: + fn::secret: registry-password +``` + +Run deployment: + +```bash +esc run my-org/my-project/prod -- ./deploy.sh +``` + +### Running interactive shells + +Start a shell session with environment variables loaded: + +```bash +esc run my-org/my-project/dev -- bash +``` + +All environment variables from the ESC environment are available in the shell. This is useful for: + +- Debugging issues locally with production-like configuration +- Running multiple commands without repeating `esc run` +- Interactive exploration of cloud resources + +Exit the shell to clear the environment variables. + +## Combining with other tools + +### Docker + +Run containers with secrets: + +```bash +esc run my-org/my-project/dev -- docker run --env DATABASE_URL myapp +``` + +Or use with Docker Compose: + +```bash +esc run my-org/my-project/dev -- docker-compose up +``` + +### Make + +Run Makefile targets with secrets: + +```bash +esc run my-org/my-project/dev -- make deploy +``` + +### CI/CD + +Use in CI/CD pipelines to inject secrets without storing them in CI configuration: + +```bash +# GitHub Actions +- run: esc run my-org/my-project/prod -- ./deploy.sh + +# GitLab CI +script: + - esc run my-org/my-project/prod -- ./deploy.sh +``` + +## Security considerations + +### Secrets are revealed during execution + +When you run `esc run`, secrets are revealed and passed as environment variables to the command. The command and any child processes can access these secrets. + +### Secrets are temporary + +Environment variables only exist for the duration of the command. Once the command exits, the secrets are no longer available. + +### Use appropriate environments + +Use separate environments for different security contexts: + +- Dev environment for local development +- CI environment for continuous integration +- Prod environment for production deployments + +Configure RBAC to control who can run commands with production secrets. + +## Next steps + +- [Managing secrets](/docs/esc/guides/managing-secrets/) - Store and organize secrets +- [Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/) - Use ESC in infrastructure code +- [Setting up OIDC](/docs/esc/guides/setting-up-oidc/) - Generate dynamic cloud credentials +- [CLI reference](/docs/esc/cli/commands/esc_run/) - Complete `esc run` documentation diff --git a/content/docs/esc/get-started/use-short-term-credentials.md b/content/docs/esc/guides/setting-up-oidc.md similarity index 64% rename from content/docs/esc/get-started/use-short-term-credentials.md rename to content/docs/esc/guides/setting-up-oidc.md index 004995b92aec..855d3033d8f2 100644 --- a/content/docs/esc/get-started/use-short-term-credentials.md +++ b/content/docs/esc/guides/setting-up-oidc.md @@ -1,19 +1,28 @@ --- -title_tag: Use Short Term Cloud Credentials | Pulumi ESC -title: Use short term cloud credentials -h1: "Use Short Term Cloud Credentials to Run Commands Without Local Secrets" -meta_desc: This page provides an overview on how to get short term cloud credentials and run commands without using local secrets using the "esc run" command. -weight: 6 +title_tag: Setting Up OIDC | Pulumi ESC +title: Setting Up OIDC +h1: Setting Up OIDC with Pulumi ESC +meta_desc: Learn how to configure OpenID Connect (OIDC) with Pulumi ESC to generate short-lived cloud credentials dynamically. menu: esc: - parent: esc-get-started - identifier: esc-get-started-use-short-term-credentials + parent: esc-guides + identifier: esc-guides-setting-up-oidc + weight: 4 aliases: + - /docs/esc/get-started/use-short-term-credentials/ + - /docs/pulumi-cloud/esc/get-started/use-short-term-credentials/ --- -Managing cloud credentials presents significant challenges for organizations of all sizes. Static, long-lived credentials, especially those stored in local environments introduce security risks and operational issues. Pulumi ESC’s built-in support for [dynamic login providers](/docs/esc/integrations/dynamic-login-credentials/), allows you to generate short-term, scoped credentials via OIDC. These credentials can then be used in your CLI workflows, CI/CD, Pulumi IaC, and more! +This guide shows you how to configure OpenID Connect (OIDC) between Pulumi ESC and AWS to generate short-lived credentials dynamically. Using [dynamic login providers](/docs/esc/integrations/dynamic-login-credentials/), you can eliminate static credentials and generate temporary, scoped credentials on demand. These credentials work with CLI workflows, CI/CD pipelines, Pulumi IaC, and any tool that uses AWS credentials. -In this example, you will use the `esc run` command to execute AWS CLI operations without having to manually configure AWS credentials in your local environment. +This guide demonstrates using `esc run` to execute AWS CLI commands with dynamically generated credentials, without configuring static AWS credentials locally. + +## Prerequisites + +- [Pulumi account](https://app.pulumi.com/signup) created +- [ESC CLI](/docs/esc-cli/) installed +- AWS account with administrative access to create IAM roles and OIDC providers +- [AWS CLI](https://aws.amazon.com/cli/) installed (for testing the integration) ## Create the AWS OIDC configuration @@ -29,29 +38,27 @@ To use dynamic credentials, you need to configure OpenID Connect (OIDC) between 3. Select **OpenID Connect** as the provider type 4. For the Provider URL, enter: `https://api.pulumi.com/oidc` 5. For the Audience, enter the name of your Pulumi organization prefixed with `aws:` (e.g. `aws:{org}`) -6. Click **Add provider** -{{< notes type="info" >}} -For legacy ESC Environments in the `default` project, the audience will use just the Pulumi organization name. -{{< /notes >}} + - **Note:** For legacy ESC environments in the `default` project, use just the Pulumi organization name without the `aws:` prefix +6. Select **Add provider** ### Create the IAM role -1. After creating the provider, click **Assign role** in the notification prompt +1. After creating the provider, select **Assign role** in the notification prompt 2. Select **Create a new role** 3. Ensure **Web identity** is selected, and verify that - `api.pulumi.com/oidc` provider is selected - Your Pulumi organization (prefixed with `aws:`) is selected as the audience -4. Click **Next** +4. Select **Next** 5. Select the permissions your role needs (e.g. **AmazonS3FullAccess** for S3 operations) -6. Click **Next** +6. Select **Next** 7. Name your role (e.g., `pulumi-esc-s3-role`) and add an optional description -8. Click **Edit** on the Select trusted entities' section -9. Ensure the "Condition" subject claim includes `aws:` before your organization name (i.e.`"api.pulumi.com/oidc:aud": "aws:myorg"` ) -10. Review and click **Create role** +8. Select **Edit** on the Select trusted entities' section +9. Ensure the Condition checks the audience claim with `aws:` before your organization name (i.e. `"api.pulumi.com/oidc:aud": "aws:myorg"`, or just `myorg` for legacy default project environments) +10. Review and select **Create role** Example trust policy: -```yaml +```json { "Version": "2012-10-17", "Statement": [ @@ -80,9 +87,9 @@ If you want to set environment level or even granular permissions in your trust ## Create a Pulumi ESC environment 1. Navigate to the [Pulumi Cloud Console](https://app.pulumi.com/) -2. Click **Environments** and then **Create environment** +2. Select **Environments** and then **Create environment** 3. Enter a name for your environment (e.g., `aws-s3-access`) -4. Click **Create environment** +4. Select **Create environment** ## Configure the AWS Provider integration @@ -107,7 +114,7 @@ Be sure to replace `` with the ARN of the IAM role you c ![An image of the ESC environment editor role trust policy](/docs/esc/assets/esc-environment-editor.png) -Click **Save** to store your environment configuration. +Select **Save** to store your environment configuration. ## Use esc run to execute AWS commands @@ -139,15 +146,17 @@ ESC dynamic credentials and the `esc run` command can be used for various scenar - **Secure script execution**: Execute scripts that interact with AWS without embedding credentials - **Team collaboration**: Provide team members with secure, scoped access to resources -## Additional OIDC authentication configurations +## Next steps -See the following guides to set up OIDC between Pulumi ESC and your specific cloud provider: +- [Integrate with Pulumi IaC](/docs/esc/guides/integrate-with-pulumi-iac/) - Use dynamic credentials in your infrastructure code +- [Running commands with esc run](/docs/esc/guides/running-commands-with-esc/) - Learn more about injecting secrets into commands +- [Integrating external secrets](/docs/esc/guides/external-secrets/) - Pull secrets from cloud vaults + +### Provider-specific OIDC configuration + +For detailed OIDC setup instructions for other cloud providers: - [Configuring OIDC for AWS](/docs/esc/environments/configuring-oidc/aws/) - [Configuring OIDC for Azure](/docs/esc/environments/configuring-oidc/azure/) - [Configuring OIDC for Google Cloud](/docs/esc/environments/configuring-oidc/gcp/) - [Configuring OIDC for Vault](/docs/esc/environments/configuring-oidc/vault/) - -In the next section, you will learn how to retrieve secret values from external sources. - -{{< get-started-stepper >}} diff --git a/content/docs/esc/integrations/dynamic-login-credentials/gh-login.md b/content/docs/esc/integrations/dynamic-login-credentials/gh-login.md index 561c225a4676..654a4512403b 100644 --- a/content/docs/esc/integrations/dynamic-login-credentials/gh-login.md +++ b/content/docs/esc/integrations/dynamic-login-credentials/gh-login.md @@ -48,7 +48,7 @@ for instructions on how to generate a private key (in PEM format) and download t Private keys do not expire and need to be manually revoked. You must keep private keys for GitHub Apps secure. Store the private key as a secret by using the `fn::secret` function. -See "[Pulumi ESC: Store and Retrieve Secrets](/docs/esc/get-started/store-and-retrieve-secrets/#store-environment-values)". +See "[Managing Secrets](/docs/esc/guides/managing-secrets/#storing-secrets)". ```yaml appId: 123456 diff --git a/content/docs/esc/integrations/infrastructure/cloudflare.md b/content/docs/esc/integrations/infrastructure/cloudflare.md index d676a3f14f89..cf0d68247521 100644 --- a/content/docs/esc/integrations/infrastructure/cloudflare.md +++ b/content/docs/esc/integrations/infrastructure/cloudflare.md @@ -42,7 +42,7 @@ Ensure you have: ### 1. Create an ESC Environment -Use the Pulumi ESC CLI to create and configure an Environment. Alternatively, to use the Pulumi Cloud console follow the [console instructions](https://www.pulumi.com/docs/esc/get-started/create-environment/#create-via-the-console). +Use the Pulumi ESC CLI to create and configure an Environment. ```bash esc login # if needed @@ -195,7 +195,7 @@ In addition to the [prerequisites above](#prerequisites), ensure you have: ### 1. Create (or Modify) an ESC Environment -Use the Pulumi ESC CLI to create and configure an Environment. Alternatively, to use the Pulumi Cloud console follow the [console instructions](https://www.pulumi.com/docs/esc/get-started/create-environment/#create-via-the-console). +Use the Pulumi ESC CLI to create and configure an Environment. ```bash esc login # if needed diff --git a/content/docs/iac/concepts/resources/names.md b/content/docs/iac/concepts/resources/names.md index 061758de0564..eb9e790fbd1e 100644 --- a/content/docs/iac/concepts/resources/names.md +++ b/content/docs/iac/concepts/resources/names.md @@ -276,7 +276,7 @@ config: pattern: ${name}-${project}-${stack} ``` -And when configuring it in an [ESC environment](/docs/esc/get-started/integrate-with-pulumi-iac/), you can specify the configuration as such: +And when configuring it in an [ESC environment](/docs/esc/guides/integrate-with-pulumi-iac/), you can specify the configuration as such: ```yaml pulumiConfig: diff --git a/content/docs/insights/discovery/get-started/begin.md b/content/docs/insights/discovery/get-started/begin.md index ad4206eec41a..3544b29a7219 100644 --- a/content/docs/insights/discovery/get-started/begin.md +++ b/content/docs/insights/discovery/get-started/begin.md @@ -26,7 +26,7 @@ If you're new to Pulumi you can click here to [start a free trial](https://app.p ## Create an ESC environment -Pulumi Insights Account Discovery requires read-only access to your cloud accounts. This access is granted by [creating an ESC environment](/docs/esc/get-started/create-environment/) that generates valid credentials for the corresponding Pulumi provider when accessed. +Pulumi Insights Account Discovery requires read-only access to your cloud accounts. This access is granted by [creating an ESC environment](/docs/esc/get-started/) that generates valid credentials for the corresponding Pulumi provider when accessed. {{% notes "info" %}} Account Discovery leverages Pulumi ESC to securely manage the credentials required to discover and read infrastructure resources, aligning with enterprise best practices for managing application secrets. diff --git a/content/docs/support/faq/secrets-config.md b/content/docs/support/faq/secrets-config.md index 96d08a2fa849..2c17d203b90f 100644 --- a/content/docs/support/faq/secrets-config.md +++ b/content/docs/support/faq/secrets-config.md @@ -26,7 +26,7 @@ See our [pricing page](https://www.pulumi.com/pricing/) for details. ## What counts as a secret towards pricing? -Secrets include [static secrets](/docs/esc/get-started/store-and-retrieve-secrets/), [dynamic login credentials](/docs/esc/integrations/dynamic-login-credentials/) and [dynamic secrets](/docs/esc/integrations/dynamic-secrets/). +Secrets include [static secrets](/docs/esc/guides/managing-secrets/), [dynamic login credentials](/docs/esc/integrations/dynamic-login-credentials/) and [dynamic secrets](/docs/esc/integrations/dynamic-secrets/). In other words, when using Pulumi ESC's document editor, each definition of `fn::secret` and `fn::open::*` (except with the [pulumi-stacks provider](/docs/esc/integrations/infrastructure/pulumi-iac/pulumi-stacks/)) is counted as a secret. diff --git a/scripts/image-borders/Pipfile b/scripts/image-borders/Pipfile new file mode 100644 index 000000000000..0c970cee2f5c --- /dev/null +++ b/scripts/image-borders/Pipfile @@ -0,0 +1,13 @@ +[[source]] +url = "https://pypi.org/simple" +verify_ssl = true +name = "pypi" + +[packages] +pillow = "*" +click = "*" + +[dev-packages] + +[requires] +python_version = "3" diff --git a/scripts/image-borders/add_borders.py b/scripts/image-borders/add_borders.py new file mode 100755 index 000000000000..6e5709fe4f4a --- /dev/null +++ b/scripts/image-borders/add_borders.py @@ -0,0 +1,269 @@ +#!/usr/bin/env python3 +""" +Add 1px #CCCCCC borders to PNG images referenced in markdown files. + +This script parses markdown files, finds PNG image references, checks if they +already have a grey border, and adds one if needed. +""" + +import os +import re +import sys +from pathlib import Path +from typing import List, Tuple + +try: + from PIL import Image, ImageDraw, PngImagePlugin + import click +except ImportError: + print("Error: Required packages not installed.", file=sys.stderr) + print("Please run: pipenv install", file=sys.stderr) + sys.exit(1) + +# Border color (medium grey for visibility on both white and light backgrounds) +BORDER_COLOR = "#999999" +BORDER_WIDTH = 1 + +# Tolerance for exact pixel matching (accounts for compression artifacts) +# Much tighter than before - we're checking ALL pixels, not sampling +PIXEL_TOLERANCE = 2 + +# Metadata key for tracking processed images +METADATA_KEY = "X-Border-Added" + + +def rgb_from_hex(hex_color: str) -> Tuple[int, int, int]: + """Convert hex color to RGB tuple.""" + hex_color = hex_color.lstrip('#') + return tuple(int(hex_color[i:i+2], 16) for i in (0, 2, 4)) + + +def color_distance(c1: Tuple[int, int, int], c2: Tuple[int, int, int]) -> float: + """Calculate Euclidean distance between two RGB colors.""" + return sum((a - b) ** 2 for a, b in zip(c1, c2)) ** 0.5 + + +def has_border(image_path: Path, border_color: str) -> bool: + """Check if an image already has a border of the specified color. + + Uses a hybrid approach: + 1. First checks PNG metadata for X-Border-Added marker (deterministic). + 2. For images **without** that marker, examines a single-pixel-wide frame + around the image and compares its color to the target border color. + + For previously processed images that are later cropped or edited, this + still behaves conservatively: it only reports a positive border match + when the entire outer frame is within tolerance of the expected color. + """ + try: + img = Image.open(image_path) + + # Step 1: Check metadata for our marker + if hasattr(img, 'info') and METADATA_KEY in img.info: + metadata_color = img.info[METADATA_KEY] + if metadata_color == border_color: + return True + + # Step 2: Examine outer edge pixels (fallback for legacy images) + width, height = img.size + target_rgb = rgb_from_hex(border_color) + + if img.mode != 'RGB': + img = img.convert('RGB') + + # Helper to test whether all pixels along a line match the target + def edge_matches(coords): + for x, y in coords: + pixel = img.getpixel((x, y)) + if color_distance(pixel[:3], target_rgb) > PIXEL_TOLERANCE: + return False + return True + + top = [(x, 0) for x in range(width)] + bottom = [(x, height - 1) for x in range(width)] + left = [(0, y) for y in range(1, height - 1)] + right = [(width - 1, y) for y in range(1, height - 1)] + + if not edge_matches(top): + return False + if not edge_matches(bottom): + return False + if height > 2 and not edge_matches(left): + return False + if height > 2 and not edge_matches(right): + return False + + return True + + except Exception as e: + print(f"Error checking border for {image_path}: {e}", file=sys.stderr) + return False + + +def add_border(image_path: Path, border_color: str, border_width: int) -> bool: + """ + Add a border to an image and mark it with metadata. + + Returns True if border was added, False if image already had a border. + """ + try: + # Check if border already exists + if has_border(image_path, border_color): + return False + + img = Image.open(image_path) + + # Create new image with border + new_width = img.width + 2 * border_width + new_height = img.height + 2 * border_width + + # Create new image with border color background + bordered_img = Image.new('RGB', (new_width, new_height), rgb_from_hex(border_color)) + + # Convert original image to RGB if needed (preserve transparency in paste) + if img.mode == 'RGBA': + # Paste with alpha channel as mask + bordered_img.paste(img, (border_width, border_width), img) + else: + if img.mode != 'RGB': + img = img.convert('RGB') + bordered_img.paste(img, (border_width, border_width)) + + # Create PNG metadata to mark this image as processed + pnginfo = PngImagePlugin.PngInfo() + pnginfo.add_text(METADATA_KEY, border_color) + + # Save with metadata + bordered_img.save(image_path, 'PNG', optimize=True, pnginfo=pnginfo) + return True + + except Exception as e: + print(f"Error processing {image_path}: {e}", file=sys.stderr) + return False + + +def extract_png_images(markdown_path: Path, repo_root: Path) -> List[Path]: + """ + Extract PNG image references from a markdown file. + + Returns a list of absolute paths to PNG files. + """ + images = [] + + try: + with open(markdown_path, 'r', encoding='utf-8') as f: + content = f.read() + + # Match markdown image syntax: ![alt text](/path/to/image.png) + pattern = r'!\[.*?\]\((.*?\.png)\)' + matches = re.findall(pattern, content, re.IGNORECASE) + + for match in matches: + # Convert markdown path to filesystem path + # Paths can be: + # - Absolute: /docs/esc/assets/image.png -> content/docs/esc/assets/image.png + # - Relative: ./images/image.png -> relative to markdown file location + + if match.startswith('/docs/'): + # Absolute path from site root + rel_path = 'content' + match + abs_path = repo_root / rel_path + elif match.startswith('docs/'): + # Absolute path without leading slash + rel_path = 'content/' + match + abs_path = repo_root / rel_path + else: + # Relative path - resolve relative to markdown file's directory + abs_path = (markdown_path.parent / match).resolve() + + if abs_path.exists(): + images.append(abs_path) + else: + print(f"Warning: Image not found: {abs_path}", file=sys.stderr) + + except Exception as e: + print(f"Error reading markdown file: {e}", file=sys.stderr) + + return images + + +@click.command() +@click.argument('markdown_file', type=click.Path(exists=True)) +@click.option('--repo-root', type=click.Path(exists=True), + help='Repository root directory (auto-detected if not specified)') +@click.option('--dry-run', is_flag=True, help='Show what would be done without making changes') +def main(markdown_file: str, repo_root: str, dry_run: bool): + """ + Add 1px #CCCCCC borders to PNG images referenced in MARKDOWN_FILE. + + Detects existing borders by checking edge pixels. Only modifies images + that don't already have a grey border. + """ + md_path = Path(markdown_file).resolve() + + # Auto-detect repo root if not specified + if repo_root: + root = Path(repo_root).resolve() + else: + # Find repo root by looking for content/ directory + root = md_path.parent + while root != root.parent: + if (root / 'content').exists(): + break + root = root.parent + else: + print("Error: Could not find repository root. Please specify --repo-root.", + file=sys.stderr) + sys.exit(1) + + print(f"Processing: {md_path}") + print(f"Repository root: {root}") + print() + + # Extract images from markdown + images = extract_png_images(md_path, root) + + if not images: + print("No PNG images found in markdown file.") + return + + print(f"Found {len(images)} PNG image(s)") + print() + + # Process each image + modified = [] + skipped = [] + + for img_path in images: + rel_path = img_path.relative_to(root) + + if dry_run: + # In dry-run mode, still honor border detection so output + # reflects what would actually happen. + if has_border(img_path, BORDER_COLOR): + print(f"Would skip (has border): {rel_path}") + skipped.append(rel_path) + else: + print(f"Would add border: {rel_path}") + modified.append(rel_path) + else: + if add_border(img_path, BORDER_COLOR, BORDER_WIDTH): + modified.append(rel_path) + print(f"✓ Added border: {rel_path}") + else: + skipped.append(rel_path) + print(f"⊘ Skipped (has border): {rel_path}") + + # Summary + print() + print("Summary:") + print(f" Modified: {len(modified)}") + print(f" Skipped: {len(skipped)}") + + if dry_run: + print() + print("DRY RUN - No changes were made") + + +if __name__ == '__main__': + main() diff --git a/static/programs/aws-import-export-pulumi-config-csharp/Program.cs b/static/programs/aws-import-export-pulumi-config-csharp/Program.cs index c14414b1a5b3..065ebb25727f 100644 --- a/static/programs/aws-import-export-pulumi-config-csharp/Program.cs +++ b/static/programs/aws-import-export-pulumi-config-csharp/Program.cs @@ -11,15 +11,15 @@ static async Task Main(string[] args) // Import the configuration values var config = new Config(); - // Retrieve the value of "myEnvironment" and "myPassword" - var environment = config.Get("myEnvironment"); - var password = config.GetSecret("myPassword"); + // Retrieve the value of "region" and "apiKey" + var region = config.Get("region"); + var apiKey = config.GetSecret("apiKey"); // Return a dictionary of outputs return new Dictionary { - ["Environment"] = environment, - ["Password"] = password + ["Region"] = region, + ["ApiKey"] = apiKey }; }); } diff --git a/static/programs/aws-import-export-pulumi-config-go/main.go b/static/programs/aws-import-export-pulumi-config-go/main.go index 87715afe6b87..2d8011da77b3 100644 --- a/static/programs/aws-import-export-pulumi-config-go/main.go +++ b/static/programs/aws-import-export-pulumi-config-go/main.go @@ -10,13 +10,13 @@ func main() { // Create a Pulumi Config config := config.New(ctx, "") - // Retrieve the value of "myEnvironment" and "myPassword" - environment := config.Get("myEnvironment") - password := config.GetSecret("myPassword") + // Retrieve the value of "region" and "apiKey" + region := config.Get("region") + apiKey := config.GetSecret("apiKey") // Export values as outputs - ctx.Export("Environment", pulumi.String(environment)) - ctx.Export("Password", pulumi.StringOutput(password)) + ctx.Export("Region", pulumi.String(region)) + ctx.Export("ApiKey", pulumi.StringOutput(apiKey)) return nil }) } diff --git a/static/programs/aws-import-export-pulumi-config-java/src/main/java/myproject/App.java b/static/programs/aws-import-export-pulumi-config-java/src/main/java/myproject/App.java index 08d73725ace7..1dba9d1550ce 100644 --- a/static/programs/aws-import-export-pulumi-config-java/src/main/java/myproject/App.java +++ b/static/programs/aws-import-export-pulumi-config-java/src/main/java/myproject/App.java @@ -9,13 +9,13 @@ public static void main(String[] args) { // Create a Pulumi Config var config = ctx.config(); - // Retrieve the values of "myEnvironment" and "myPassword" - var environment = config.get("myEnvironment"); - var password = config.getSecret("myPassword"); + // Retrieve the values of "region" and "apiKey" + var region = config.get("region"); + var apiKey = config.getSecret("apiKey"); // Export the values as a stack outputs - ctx.export("Environment", Output.of(environment)); - ctx.export("Password", Output.of(password)); + ctx.export("Region", Output.of(region)); + ctx.export("ApiKey", Output.of(apiKey)); }); } } \ No newline at end of file diff --git a/static/programs/aws-import-export-pulumi-config-javascript/index.js b/static/programs/aws-import-export-pulumi-config-javascript/index.js index 58d2f4f53079..bd6f82be2106 100644 --- a/static/programs/aws-import-export-pulumi-config-javascript/index.js +++ b/static/programs/aws-import-export-pulumi-config-javascript/index.js @@ -4,12 +4,12 @@ const pulumi = require("@pulumi/pulumi"); // Create a new Pulumi Config const config = new pulumi.Config(); -// Retrieve the values of "myEnvironment" and "myPassword" -const environment = config.get("myEnvironment"); -const password = config.getSecret("myPassword"); +// Retrieve the values of "region" and "apiKey" +const region = config.get("region"); +const apiKey = config.getSecret("apiKey"); // Export values as stack outputs module.exports = { - Environment: environment, - Password: password, + Region: region, + ApiKey: apiKey, }; diff --git a/static/programs/aws-import-export-pulumi-config-python/__main__.py b/static/programs/aws-import-export-pulumi-config-python/__main__.py index 3dd4304d652a..4f673bbab6cf 100644 --- a/static/programs/aws-import-export-pulumi-config-python/__main__.py +++ b/static/programs/aws-import-export-pulumi-config-python/__main__.py @@ -3,10 +3,10 @@ # Import the configuration values config = pulumi.Config() -# Retrieve the values of "myEnvironment" and "myPassword" -environment = config.get("myEnvironment") -password = config.get_secret("myPassword") +# Retrieve the values of "region" and "apiKey" +region = config.get("region") +api_key = config.get_secret("apiKey") # Export the values as an output -pulumi.export('Environment', environment) -pulumi.export("Password", password) \ No newline at end of file +pulumi.export('Region', region) +pulumi.export("ApiKey", api_key) \ No newline at end of file diff --git a/static/programs/aws-import-export-pulumi-config-typescript/index.ts b/static/programs/aws-import-export-pulumi-config-typescript/index.ts index b64c56ac6ff5..bec747751f35 100644 --- a/static/programs/aws-import-export-pulumi-config-typescript/index.ts +++ b/static/programs/aws-import-export-pulumi-config-typescript/index.ts @@ -3,10 +3,10 @@ import * as pulumi from "@pulumi/pulumi"; // Create a new Pulumi Config const config = new pulumi.Config(); -// Retrieve the values of "myEnvironment" and "myPassword" -const environment = config.get("myEnvironment"); -const password = config.getSecret("myPassword"); +// Retrieve the values of "region" and "apiKey" +const region = config.get("region"); +const apiKey = config.getSecret("apiKey"); // Export values as stack output -export const Environment = environment; -export const Password = password; +export const Region = region; +export const ApiKey = apiKey;