diff --git a/docs/applications/01-overview.md b/docs/applications/01-overview.md index 2f2d7da8..3eb7e842 100644 --- a/docs/applications/01-overview.md +++ b/docs/applications/01-overview.md @@ -17,7 +17,7 @@ Additionally, applications have a few other characteristics: * IAM Roles (AWS) or Service Accounts (GCP / Azure) are automatically created for your app to manage IAM with principle of least privilege. * They are container-based ([VMs are on our roadmap](https://roadmap.massdriver.cloud/bundles/application-vm-support-cl7s8svuy3959141xipth2cwcbe)). -* Ability to generate environment variables via the application [instance's](/concepts/components-instances-deployments) parameters or [connections](/concepts/connections). +* Ability to generate environment variables via the application [instance's](/concepts/components-instances-deployments) parameters or [dependencies](/concepts/dependencies). * Ability to programmatically select IAM Policies & Permissions from infrastructure components. **Supported Runtimes**: diff --git a/docs/applications/02-create-application.md b/docs/applications/02-create-application.md index bdbb7423..711f653c 100644 --- a/docs/applications/02-create-application.md +++ b/docs/applications/02-create-application.md @@ -56,7 +56,7 @@ Application templates are cached locally the first time `mass bundle new` is run ::: -Then, [`connections`](/concepts/connections) (your application dependencies) will need to be selected. +Then, [`connections`](/concepts/dependencies) (your application dependencies) will need to be selected. For this example we'll choose [`postgresql-authentication`](https://github.com/massdriver-cloud/artifact-definitions/blob/main/definitions/artifacts/postgresql-authentication.json). diff --git a/docs/bundle-development/00-overview.md b/docs/bundle-development/00-overview.md index 84fb3dcf..8051bccb 100644 --- a/docs/bundle-development/00-overview.md +++ b/docs/bundle-development/00-overview.md @@ -11,7 +11,7 @@ This section covers everything you need to build, test, and publish Massdriver b - **[Bundle YAML Specification](./bundle-yaml-spec)** - Complete reference for the `massdriver.yaml` file format - **[Schema Design](./schema-design/overview)** - JSON Schema patterns and Massdriver annotations for building parameter forms -- **[Connections & Resources](./connections-artifacts/overview)** - How bundles consume and produce resources for type-safe infrastructure composition +- **[Dependencies & Resources](./dependencies-resources/overview)** - How bundles consume and produce resources for type-safe infrastructure composition - **[Provisioners](./provisioners/overview)** - Configure OpenTofu, Terraform, Helm, and Bicep provisioning steps - **[Publishing](./publishing/versioning)** - Version, template, and publish bundles to the registry @@ -22,7 +22,7 @@ This section covers everything you need to build, test, and publish Massdriver b mass bundle new my-bundle ``` -2. **Define your schema** in `massdriver.yaml` with parameters, connections, and resources (under the legacy `artifacts:` key) +2. **Define your schema** in `massdriver.yaml` with parameters, dependencies, and resources 3. **Write your IaC** in the provisioner directory (e.g., `src/` for OpenTofu) diff --git a/docs/bundle-development/01-bundle-yaml-spec.md b/docs/bundle-development/01-bundle-yaml-spec.md index a455bb06..a17d9a01 100644 --- a/docs/bundle-development/01-bundle-yaml-spec.md +++ b/docs/bundle-development/01-bundle-yaml-spec.md @@ -101,6 +101,10 @@ steps: # - .params. - Bundle parameters # - .dependencies. - Dependency resources # - .dependencies.. - Dependency resource fields + # - .resources. - Resources this bundle produces + # + # .connections and .artifacts are deprecated aliases for .dependencies + # and .resources. They hold the same data and still work. config: # OpenTofu/Terraform config options: # json: boolean - Enable JSON output (default: false) diff --git a/docs/bundle-development/connections-artifacts/00-overview.md b/docs/bundle-development/connections-artifacts/00-overview.md deleted file mode 100644 index 3b321494..00000000 --- a/docs/bundle-development/connections-artifacts/00-overview.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -id: connections-artifacts-overview -slug: /bundle-development/connections-artifacts/overview -title: Connections & Resources -sidebar_label: Overview ---- - -Connections and resources enable type-safe composition of infrastructure components in Massdriver. - -## Key concepts - -- **Resources** are the outputs a bundle produces (e.g., database connection details, cluster credentials). Bundles declare them under the `artifacts:` key in `massdriver.yaml` — the YAML key retains its original name for backwards compatibility. -- **Connections** are the inputs a bundle consumes from other bundles' resources. -- **Resource Types** are the schemas that define the contract between bundles. - -## How it works - -When you connect bundles on the canvas, Massdriver validates that: -1. The resource's type matches the connection's expected type -2. The resource data conforms to the resource type schema -3. Any additional constraints (version, region) are satisfied - -This validation happens at design time, preventing incompatible infrastructure from being deployed. - -## In this section - -- **[Resource Type Specification](./artifact-definition-spec)** - Complete reference for defining resource type schemas - -## Related documentation - -- [Concepts: Resources & Resource Types](/concepts/resources-and-types) - Conceptual overview -- [Bundle YAML: connections](/bundle-development/bundle-yaml-spec#connections) - Connection schema reference -- [Bundle YAML: artifacts](/bundle-development/bundle-yaml-spec#artifacts) - Resource output reference (YAML key still named `artifacts:`) -- [Resource Types Repository](https://github.com/massdriver-cloud/artifact-definitions) - Standard resource types (the GitHub repo URL retains the legacy name) diff --git a/docs/bundle-development/dependencies-resources/00-overview.md b/docs/bundle-development/dependencies-resources/00-overview.md new file mode 100644 index 00000000..7a85962e --- /dev/null +++ b/docs/bundle-development/dependencies-resources/00-overview.md @@ -0,0 +1,39 @@ +--- +id: dependencies-resources-overview +slug: /bundle-development/dependencies-resources/overview +title: Dependencies & Resources +sidebar_label: Overview +--- + +Dependencies and resources enable type-safe composition of infrastructure components in Massdriver. + +## Key concepts + +- **Resources** are what a bundle produces for other bundles to consume — database connection details, cluster credentials, network layouts. Bundles declare them under `resources:` in `massdriver.yaml`. +- **Dependencies** are what a bundle consumes from other bundles' resources. Bundles declare them under `dependencies:` in `massdriver.yaml`. +- **Resource types** are the versioned schemas that define the contract between bundles. + +Drawing a line between two components on the canvas fills one bundle's dependency with another bundle's resource. + +## How it works + +When you draw a dependency between bundles on the canvas, Massdriver validates that: + +1. The resource's type matches the type the dependency expects +2. The resource data conforms to the resource type schema +3. The bundle versions at each end fall inside the dependency's version ranges + +The first two checks happen when you draw the line, so incompatible infrastructure never reaches a deployment. The third is re-checked per environment, so one blueprint can serve environments running different bundle versions. + +## In this section + +- **[Resource Type Specification](./resource-type-spec)** - Authoring, versioning, and publishing a resource type +- **[Version Resolution](./version-resolution)** - How a version range picks a resource at deploy time + +## Related documentation + +- [Concepts: Dependencies](/concepts/dependencies) - Version ranges on a dependency +- [Concepts: Resources & Resource Types](/concepts/resources-and-types) - Conceptual overview +- [Bundle YAML: dependencies](/bundle-development/bundle-yaml-spec#dependencies) - What a bundle consumes +- [Bundle YAML: resources](/bundle-development/bundle-yaml-spec#resources) - What a bundle produces +- [Resource Types Repository](https://github.com/massdriver-cloud/artifact-definitions) - Standard resource types (the GitHub repo URL retains the legacy name) diff --git a/docs/bundle-development/connections-artifacts/01-artifact-definition-spec.md b/docs/bundle-development/dependencies-resources/01-resource-type-spec.md similarity index 73% rename from docs/bundle-development/connections-artifacts/01-artifact-definition-spec.md rename to docs/bundle-development/dependencies-resources/01-resource-type-spec.md index f90bae3b..b3273ba5 100644 --- a/docs/bundle-development/connections-artifacts/01-artifact-definition-spec.md +++ b/docs/bundle-development/dependencies-resources/01-resource-type-spec.md @@ -1,6 +1,6 @@ --- -id: artifact-definition-spec -slug: /bundle-development/connections-artifacts/artifact-definition-spec +id: resource-type-spec +slug: /bundle-development/dependencies-resources/resource-type-spec title: Resource Type Specification sidebar_label: Resource Type Spec --- @@ -10,7 +10,7 @@ sidebar_label: Resource Type Spec This document outlines the `massdriver.yaml` format for authoring resource types. This format provides a more ergonomic authoring experience compared to writing raw JSON Schema, with support for referencing external files for instructions and export templates. :::tip When to Use This Format -Use the `massdriver.yaml` format when creating new resource types. It separates concerns by keeping markdown instructions and Liquid templates in their own files, making definitions easier to read and maintain. +Use the `massdriver.yaml` format for every resource type. It is the format that supports versioning and publishing to your organization's catalog, and it keeps markdown instructions and Liquid templates in their own files. The raw JSON schema format it replaces is deprecated. ::: ## File Structure @@ -20,6 +20,9 @@ A resource type using this format consists of a directory containing: ``` my-resource-type/ ├── massdriver.yaml # Main definition file +├── README.md # Published with the artifact +├── CHANGELOG.md # Published with the artifact +├── icon.svg # Published with the artifact ├── instructions/ # Onboarding instruction markdown files │ ├── step1.md │ └── step2.md @@ -40,6 +43,12 @@ my-resource-type/ # reference path: / (e.g., "acme/aws-rds-postgres") name: my-resource-type-name +# version (required to publish) +# Semantic version of the resource type (MAJOR.MINOR.PATCH). +# The CLI publishes this as the artifact's tag in your organization's catalog. +# Publishing is immutable: a version that already exists cannot be republished. +version: 2.1.0 + # label (required) # Human-readable display name shown in the Massdriver UI. # Used in dropdowns, connection labels, and the resource type selector. @@ -69,6 +78,12 @@ ui: # only see it as an environment default. connectionOrientation: link + # environmentDefaultGroup (optional) + # Groups this resource type with others in an environment's defaults panel. + # The group named "credentials" holds cloud credential types, which the UI + # separates from the rest of an environment's defaults. + environmentDefaultGroup: credentials + # instructions (optional) # Onboarding instructions shown to users when they create resources of this # type. Each instruction becomes a step in the onboarding wizard. @@ -288,8 +303,7 @@ schema: token: title: API Token type: string - $md: - sensitive: true + $md.sensitive: true ``` ## Complete Example with All Features @@ -411,50 +425,82 @@ DATABASE_NAME={{ artifact.authentication.database }} ## Publishing -Publish your resource type using the Massdriver CLI: +A resource type publishes to your organization's catalog as an OCI artifact, the same way a bundle does. + +```bash +# Create the repository in the catalog +mass resource-type create aws-vpc + +# Publish the version in massdriver.yaml +mass resource-type publish ./aws-vpc + +# Pull a published version back down +mass resource-type pull aws-vpc@2.1.0 +``` + +`mass resource-type publish` takes a directory containing a `massdriver.yaml`, or the `massdriver.yaml` itself, and defaults to the current directory. + +The published artifact carries the `massdriver.yaml`, the readme, the changelog, the icon, and the instruction and export template files the `massdriver.yaml` references. Nothing else in the directory is included. + +Publishing is immutable. Once a version exists it cannot be overwritten, so anything pinned to it keeps resolving to what it resolved to the first time. + +### Versions and release channels + +Resource types use the same version model as bundles: semantic versions, release channels, and per-environment pinning. A bundle names the versions it accepts in its `dependencies` and `resources` blocks, and Massdriver resolves the range at deploy time. See [Version Resolution](/bundle-development/dependencies-resources/version-resolution). + +Each resource type also gets a repository in the OCI catalog, with the same access grants and attribute filters as a bundle repository. + +### Referenced files must exist + +`ui.instructions[].path` and `exports[].templatePath` point at files rather than carrying their content inline. A path that does not exist, or that resolves outside the resource type's directory, fails the publish. An incomplete artifact is never shipped. + +### Publishing a raw JSON schema is deprecated + +`mass resource-type publish` still accepts a raw JSON or YAML schema file, the format that predates `massdriver.yaml`, and prints a deprecation warning. That path will be removed in a future release. + +A raw schema carries no version of its own. It is stored as the resource type's unversioned `0.0.0` document, cannot take part in versioning, and cannot be pulled back down. + +`mass resource-type convert` migrates one: ```bash -mass definition publish ./path/to/massdriver.yaml +mass resource-type convert ./my-resource-type.json ``` -The CLI will: -1. Read and parse the `massdriver.yaml` file -2. Inline the content from instruction and export template files -3. Build the JSON Schema format expected by the API -4. Validate against the resource type meta-schema -5. Publish to your organization +It writes a `massdriver.yaml` alongside the schema and pulls inlined instruction and export content back out into referenced files. A placeholder `version` is written into the output — set a real version before you publish. + +Resource types that existed before this format were migrated in place and keep working. ## Referencing in Bundles -Once published, reference your resource type in bundle `massdriver.yaml` files: +Once published, name the resource type and the versions you accept in a bundle's `massdriver.yaml`: ```yaml -# In a bundle's massdriver.yaml -artifacts: - required: - - database - properties: - database: - # Omit org prefix for definitions in your own organization - $ref: postgres-database - # Or use fully qualified name: acme/postgres-database - -connections: - required: - - database - properties: - database: - $ref: postgres-database +# What the bundle consumes +dependencies: + database: + # Omit the org prefix for resource types in your own organization + resource_type: postgres-database@~2 + required: true + +# What the bundle produces for other bundles to consume +resources: + database: + resource_type: postgres-database@2.1.0 + required: true ``` +A dependency accepts a range. A resource pins the single version it produces. See [Version Resolution](/bundle-development/dependencies-resources/version-resolution) for the accepted range forms. + ## Field Reference | Field | Required | Description | |-------|----------|-------------| | `name` | Yes | Unique identifier (lowercase, hyphens) | +| `version` | To publish | Semantic version, published as the artifact's tag | | `label` | Yes | Display name in UI | | `icon` | No | URL to icon image | | `ui.connectionOrientation` | No | `"link"` or `"environmentDefault"` | +| `ui.environmentDefaultGroup` | No | Groups the type in an environment's defaults panel; `"credentials"` marks cloud credential types | | `ui.instructions` | No | Array of onboarding steps | | `ui.instructions[].label` | Yes | Step title | | `ui.instructions[].path` | Yes | Path to markdown file | @@ -468,5 +514,6 @@ connections: ## See Also - [Resource Types Concept](/concepts/resources-and-types) - Understanding resource types -- [Custom Resource Type Guide](/guides/custom-artifact-definition) - JSON format and advanced customization +- [Version Resolution](/bundle-development/dependencies-resources/version-resolution) - How a version range picks a resource at deploy time +- [Custom Resource Type Guide](/guides/custom-resource-type) - JSON format and advanced customization - [Massdriver Annotations](/bundle-development/schema-design/massdriver-annotations) - Special `$md` annotations diff --git a/docs/bundle-development/dependencies-resources/02-version-resolution.md b/docs/bundle-development/dependencies-resources/02-version-resolution.md new file mode 100644 index 00000000..a268be32 --- /dev/null +++ b/docs/bundle-development/dependencies-resources/02-version-resolution.md @@ -0,0 +1,79 @@ +--- +id: version-resolution +slug: /bundle-development/dependencies-resources/version-resolution +title: Version Resolution +sidebar_label: Version Resolution +--- + +# Version Resolution + +A bundle names the resource types it consumes and produces, and the versions of those resource types it accepts. Massdriver resolves each range to one specific resource when the bundle deploys, using what is available in the target environment at that moment. + +## Declaring a range + +`dependencies` are the resource types a bundle consumes. `resources` are the resource types it produces for other bundles to consume. Both take a `resource_type` in the form `name@version`: + +```yaml +dependencies: + network: + resource_type: aws-vpc@~1.2 + required: true + database: + resource_type: postgres-authentication@2.1.0 + required: false + +resources: + api: + resource_type: aws-ecs-service@~2 + required: true +``` + +`required` means different things on each side. On a dependency it means the slot must be filled before the bundle can deploy. On a resource it means the bundle always creates it. + +A resource pins the single version it produces. A dependency accepts a range: + +| Form | Accepts | +|------|---------| +| `aws-vpc@1.2.3` | that exact version | +| `aws-vpc@~1.2` | the newest `1.2.x` | +| `aws-vpc@~1` | the newest `1.x` | +| `aws-vpc@latest` | the newest stable version | +| `aws-vpc@latest+dev` | the newest version, including development releases | + +The `+dev` suffix also applies to a tilde range. `~1+dev` accepts the newest `1.x` including development releases; `~1` on its own skips them. + +## What fills a slot + +At deploy time Massdriver looks for a resource of the right type whose version the range accepts. It checks three sources and takes the first match: + + + +1. **A remote reference** set on the instance. This is a hand-picked resource, so both its resource type and its version are checked. The check runs again on every deploy, which catches a reference that was assigned before the bundle moved to a narrower range. +2. **A connection** drawn on the project blueprint. The resource type was matched when the connection was drawn, so only the version is re-checked here. +3. **An environment default**. Among the environment's defaults of that resource type, Massdriver takes the highest version the range accepts. + +A source that does not match is skipped, and the next source is tried. If no source matches, a `required` dependency blocks the deploy. + +## One default per version + +An environment holds one default per version of a resource type. Two bundles asking for different ranges each draw the version they asked for, from the same environment, with no per-instance configuration. + +This is also how a dependency picks up a newer version without the consuming bundle being republished. Add a newer `aws-vpc` default to the environment, and a bundle declaring `aws-vpc@~1` resolves to it on its next deploy as long as the new version is still in the `1.x` line. + +## Remote references are checked when you assign them + +Choosing a remote reference for a slot validates the pick against the slot's resource type and version range. An out-of-range or wrong-type resource is rejected at that point rather than during a deployment. + +## The legacy format + +`connections` and `artifacts` are the previous names for `dependencies` and `resources`. They still work and publish with a warning. + +A slot declared in the legacy format carries no version range. It matches an environment default of its resource type at any version, and it trusts a blueprint connection without re-checking the version. Moving the slot to `dependencies` or `resources` is what turns the version checks on. + +The two forms are mutually exclusive. Setting both `connections` and `dependencies`, or both `artifacts` and `resources`, is an error, so migrate one block at a time. + +## Related documentation + +- [Bundle YAML: dependencies and resources](/bundle-development/bundle-yaml-spec#dependencies) — full field reference +- [Connections](/concepts/dependencies) — version ranges on the blueprint connection itself +- [Resource Type Spec](/bundle-development/dependencies-resources/resource-type-spec) — authoring and publishing a versioned resource type diff --git a/docs/bundle-development/provisioners/01-overview.md b/docs/bundle-development/provisioners/01-overview.md index 55fbf87c..f7fcdfba 100644 --- a/docs/bundle-development/provisioners/01-overview.md +++ b/docs/bundle-development/provisioners/01-overview.md @@ -29,7 +29,7 @@ The `steps` block in the `massdriver.yaml` file specifies the steps to execute d - **`provisioner`**: **REQUIRED** Specifies the provisioner use (e.g., `terraform`, `opentofu`, `helm`, `bicep`). - **`path`**: **REQUIRED** The relative path to the IaC for this step of the bundle. - **`skip_on_delete`**: If set to `true`, the step will be skipped during the `decommission` action. This is useful for retaining resources like encryption keys. -- **`config`**: A block that allows custom configuration for the provisioner. Refer to the provisioner documention for a list of accepted values. Each field within the `config` block must be specified as `jq` formatted queries, using `params` and `connections` as inputs. +- **`config`**: A block that allows custom configuration for the provisioner. Refer to the provisioner documention for a list of accepted values. Each field within the `config` block must be specified as `jq` formatted queries, run against the deployment context described in [The jq context](#the-jq-context). ### Example @@ -50,11 +50,33 @@ steps: provisioner: bicep skip_on_delete: true config: - region: .connections.foo.specs.region + region: .dependencies.foo.specs.region resource_group: '@text "foo"' delete_resource_group: 'true' ``` +### The jq context + +Every `jq` expression in a `config` block, and in the `params.jq`, `connections.jq`, `envs.jq`, and `secrets.jq` templates, runs against the same object: + +| Key | Holds | +|-----|-------| +| `.params` | The instance's parameters, plus `md_metadata` | +| `.dependencies` | The resources filling the bundle's dependency slots, keyed by slot name | +| `.resources` | The resources the bundle produces, keyed by slot name | +| `.id` | The instance identifier | + +`.connections` and `.artifacts` are deprecated aliases for `.dependencies` and `.resources`. They hold the same data and still work, so existing bundles keep running, but new expressions should use the current names. + +```yaml +config: + # Preferred + region: .dependencies.azure_credentials.specs.region + + # Deprecated alias, same value + region: .connections.azure_credentials.specs.region +``` + --- ## Environment @@ -67,11 +89,14 @@ The following files are generated and placed at the specified path in the provis | File Path | Description | |---------------------------------|-------------------------------------------------| -| `/massdriver/params.json` | Parameters from instance configuration | -| `/massdriver/connections.json` | Connection artifacts | -| `/massdriver/envs.json` | Environment variables | -| `/massdriver/secrets.json` | Secrets (in decrypted form) | -| `/massdriver/config.json` | Provisioner configuration (from `config` block) | +| `/massdriver/params.json` | Parameters from instance configuration | +| `/massdriver/dependencies.json` | The resources filling the bundle's dependencies | +| `/massdriver/connections.json` | Deprecated. The same document as `dependencies.json` | +| `/massdriver/envs.json` | Environment variables | +| `/massdriver/secrets.json` | Secrets (in decrypted form) | +| `/massdriver/config.json` | Provisioner configuration (from `config` block) | + +`dependencies.json` and `connections.json` hold the same document. Both are written on every step, so a custom provisioner image can move to the new name whenever it is ready. For more information about how a provisioner interacts with these files, refer to the provisioner-specific documentation. diff --git a/docs/bundle-development/publishing/01-bundle-templates.md b/docs/bundle-development/publishing/01-bundle-templates.md index 801334b2..18e36588 100644 --- a/docs/bundle-development/publishing/01-bundle-templates.md +++ b/docs/bundle-development/publishing/01-bundle-templates.md @@ -98,8 +98,10 @@ connections: properties: {} {% endif %} -# Resources - declare the outputs your bundle produces. -# The YAML key remains `artifacts:` for backwards compatibility. +# Resources - declare what your bundle produces for other bundles to consume. +# `connections:` and `artifacts:` are the legacy keys, still accepted with a +# deprecation warning. New templates should emit `dependencies:` and +# `resources:`, which carry a resource type version. artifacts: properties: {} diff --git a/docs/bundle-development/publishing/02-module-patterns.md b/docs/bundle-development/publishing/02-module-patterns.md index b319ff72..bb32a1c8 100644 --- a/docs/bundle-development/publishing/02-module-patterns.md +++ b/docs/bundle-development/publishing/02-module-patterns.md @@ -7,7 +7,7 @@ sidebar_label: Module Patterns If your team has an existing library of Terraform or OpenTofu modules — naming utilities, diagnostic configurations, RBAC helpers, VM extensions, and other shared modules — this guide explains how each pattern maps to Massdriver and where your modules should live after migration. -This guide assumes you're familiar with [bundles](/concepts/bundles), [resources](/concepts/resources-and-types), and [connections](/concepts/connections). If you're new to these concepts, start with the [getting started guides](/getting-started/deploying-first-bundle). +This guide assumes you're familiar with [bundles](/concepts/bundles), [resources](/concepts/resources-and-types), and [dependencies](/concepts/dependencies). If you're new to these concepts, start with the [getting started guides](/getting-started/deploying-first-bundle). ## The Migration Map @@ -152,7 +152,7 @@ When `ssh.private_key` is set, the provisioner configures SSH authentication and ### How connections supercharge child modules -The real power of this pattern comes from combining child modules with Massdriver [connections](/concepts/connections). When a bundle declares a connection, it receives the full resource data from another bundle — including infrastructure IDs, IAM policies, authentication details, and network configuration. +The real power of this pattern comes from combining child modules with Massdriver [connections](/concepts/dependencies). When a bundle declares a connection, it receives the full resource data from another bundle — including infrastructure IDs, IAM policies, authentication details, and network configuration. Consider a bundle for a VM-based API that needs access to a PostgreSQL database. The database resource includes everything the child module needs: @@ -337,7 +337,7 @@ Now when someone needs to update SSO settings, they configure it through the bun - [Creating a Bundle from an OpenTofu Module](/guides/bundle-from-opentofu) — Step-by-step conversion of a single module - [Bootstrap Your Platform](/guides/bootstrap-platform) — Model your entire platform architecture before implementing -- [Custom Resource Types](/guides/custom-artifact-definition) — Create the contracts that connect your bundles +- [Custom Resource Types](/guides/custom-resource-type) — Create the contracts that connect your bundles - [Using Bundle Deployment Metadata](/getting-started/using-bundle-metadata) — Full reference for `md_metadata` - [Sharing Infrastructure](/guides/sharing-infrastructure) — Environment defaults and remote references diff --git a/docs/bundle-development/schema-design/02-massdriver-annotations.md b/docs/bundle-development/schema-design/02-massdriver-annotations.md index afc89428..dd94d00c 100644 --- a/docs/bundle-development/schema-design/02-massdriver-annotations.md +++ b/docs/bundle-development/schema-design/02-massdriver-annotations.md @@ -90,7 +90,8 @@ The `$md.enum` annotation expects a map with the following properties: | Property | Required | Description | |----------|----------|-------------| -| `connection` | Yes | Name of the connection resource to query | +| `dependency` | Yes | Name of the dependency to query | +| `connection` | — | The name `dependency` replaced. Still accepted; `dependency` wins when a bundle sets both | | `options` | Yes | JQ filter to extract available options from the resource data | | `value` | No | JQ formatter for option values (defaults to `.`) | | `label` | No | JQ formatter for option labels (defaults to `value` formatter) | @@ -116,12 +117,12 @@ properties: description: Select the database instance to connect to type: string $md.enum: - connection: postgres_cluster + dependency: postgres_cluster options: .data.instances[] ``` **How it works:** -- Queries the `postgres_cluster` connection resource +- Queries the resource filling the `postgres_cluster` dependency - Extracts instance names using the JQ filter `.data.instances[]` - Creates a dropdown with each instance as both the value and label @@ -146,14 +147,14 @@ properties: description: Select the subnet for resource deployment type: string $md.enum: - connection: vpc + dependency: vpc options: .data.infrastructure.subnets[] value: .id label: '"\(.name) - \(.cidr) (\(.availability_zone))"' ``` **How it works:** -- Queries the `vpc` connection resource +- Queries the resource filling the `vpc` dependency - Iterates over subnets using `.data.infrastructure.subnets[]` - Extracts the subnet ID as the value: `.id` - Creates a formatted label: `"Private Subnet 1 - 10.0.1.0/24 (us-east-1a)"` @@ -179,7 +180,7 @@ properties: description: Select the IAM policy to attach type: string $md.enum: - connection: security + dependency: security options: .data.security.iam | keys value: . label: . @@ -187,8 +188,8 @@ properties: ### Error Handling -If the connection is not found or the JQ query is invalid, Massdriver will display an error option in the dropdown: -- `"ERROR: Connection not found: "` +If the dependency is not wired up or the JQ query is invalid, Massdriver will display an error option in the dropdown: +- `"ERROR: Dependency not found: "` - `"ERROR: Invalid JQ query: "` These errors help developers identify configuration issues during bundle development. @@ -196,8 +197,8 @@ These errors help developers identify configuration issues during bundle develop ### Technical Details The `$md.enum` extension: -1. Finds the specified connection by its name on the consuming bundle -2. Executes the JQ `options` filter against the connection's resource data +1. Finds the specified dependency by its name on the consuming bundle +2. Executes the JQ `options` filter against the resource filling it 3. For each result, applies the `value` and `label` formatters 4. Generates a JSON Schema `oneOf` array with `const` (value) and `title` (label) pairs 5. Removes the `$md.enum` annotation from the final schema @@ -516,7 +517,7 @@ properties: description: Select the subnet for database deployment type: string $md.enum: - connection: vpc + dependency: vpc options: .data.infrastructure.private_subnets[] value: .id label: '"\(.name) (\(.availability_zone))"' diff --git a/docs/concepts/02-artifacts-and-definitions.md b/docs/concepts/02-artifacts-and-definitions.md index 8594c8cf..61d817e1 100644 --- a/docs/concepts/02-artifacts-and-definitions.md +++ b/docs/concepts/02-artifacts-and-definitions.md @@ -85,27 +85,45 @@ Created manually for external resources: 2. **Structural Matching**: Once provisioned, the actual data is validated against the schema 3. **Data Injection**: During deployment, resource data is injected into the consuming bundle +## Resource types are versioned + +A resource type is a published artifact, not a loose schema. It is authored as a `massdriver.yaml`, published to a repository in your organization's catalog, and pinned by version — the same model bundles use. + +Every resource type has: + +- A semantic version and release channels +- A repository in the OCI catalog, with the same access grants and attribute filters as a bundle repository +- Immutable published versions. Once a version exists it cannot be overwritten + +```bash +mass resource-type create aws-vpc +mass resource-type publish ./aws-vpc +mass resource-type pull aws-vpc@2.1.0 +``` + +Versioning the resource type versions the contract between bundles. Adding a required field to a payload that three bundles consume is a breaking change to all three, and a major version bump is how the platform is told so. + ## Usage in massdriver.yaml +A bundle names the resource types it consumes under `dependencies` and the ones it produces under `resources`, each with the versions it accepts: + ```yaml -# Bundle consumes a VPC resource -connections: - required: - - vpc - properties: - vpc: - $ref: aws-vpc - -# Bundle produces a database resource -artifacts: - required: - - database - properties: - database: - $ref: postgresql-authentication +# What the bundle consumes +dependencies: + vpc: + resource_type: aws-vpc@~1 + required: true + +# What the bundle produces +resources: + database: + resource_type: postgresql-authentication@1.0.0 + required: true ``` -> The bundle spec keys (`connections`, `artifacts`) retain their original names so existing `massdriver.yaml` files keep building. The bundle spec is moving to `resources` over time. +A dependency accepts a range and Massdriver resolves it at deploy time. A resource pins the single version it produces. See [Version Resolution](/bundle-development/dependencies-resources/version-resolution). + +> `connections` and `artifacts` are the previous names for these blocks. They still work and publish with a warning, but a slot declared that way carries no version range and takes part in no version checks. ## Best Practices @@ -117,6 +135,7 @@ artifacts: ## Related Documentation - [Bundle YAML Specification](/bundle-development/bundle-yaml-spec) - Connection and resource configuration -- [Resource Type Specification](/bundle-development/connections-artifacts/artifact-definition-spec) - Complete schema reference +- [Resource Type Specification](/bundle-development/dependencies-resources/resource-type-spec) - Complete schema reference +- [Version Resolution](/bundle-development/dependencies-resources/version-resolution) - How a version range picks a resource at deploy time - [Resource Types Repository](https://github.com/massdriver-cloud/artifact-definitions) - Standard resource types - [Massdriver Annotations](/bundle-development/schema-design/massdriver-annotations) - `$md.sensitive` and other extensions diff --git a/docs/concepts/05-connections.md b/docs/concepts/05-connections.md deleted file mode 100644 index a04ef6aa..00000000 --- a/docs/concepts/05-connections.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -id: concepts-connections -slug: /concepts/connections -title: Connections -sidebar_label: Connections ---- - -Connections are the lines between [components](/concepts/components-instances-deployments#components) in the Massdriver UI. They indicate other application and infrastructure bundles that a bundle is dependent on. They can be thought of as an "input" that is another instance's resource. - -Connections are unidirectional. They always flow from "left" to "right" and are the _edges_ of a directed acyclic graph defining the dependency hierarchy of your infrastructure and applications. - -A dotted line indicates that a [resource](/concepts/resources-and-types) _has not_ been provisioned yet for the connection. - -A solid line indicates that a [resource](/concepts/resources-and-types) _has_ been provisioned for the connection. - - - -## Dynamic Configuration from Connections - -You can use the `$md.enum` annotation in your bundle's params schema to create dynamic dropdown fields that query data from connected resources. This enables users to select from available cloud resources (like subnets, database instances, or IAM roles) that exist in their connected infrastructure. - -For example, a bundle connected to a VPC can provide a dropdown to select from available subnets, or a bundle connected to a database cluster can let users pick a specific database instance. - -See the [Massdriver Annotations Reference](/bundle-development/schema-design/massdriver-annotations#mdenuml) for complete documentation and examples. - -## Removing Connections - -To remove a connection, click the **X** on the connection line. - -:::caution -Removing a connection **without decommissioning the dependent component** may result in an inconsistent state resulting in orphaned resources. -::: diff --git a/docs/concepts/05-dependencies.md b/docs/concepts/05-dependencies.md new file mode 100644 index 00000000..b4f2d2ae --- /dev/null +++ b/docs/concepts/05-dependencies.md @@ -0,0 +1,74 @@ +--- +id: concepts-dependencies +slug: /concepts/dependencies +title: Dependencies +sidebar_label: Dependencies +--- + +Dependencies are the lines between [components](/concepts/components-instances-deployments#components) in the Massdriver UI. They indicate other application and infrastructure bundles that a bundle depends on. Drawing one fills a slot the consuming bundle declared with another instance's resource. + +Dependencies are unidirectional. They always flow from "left" to "right" and are the _edges_ of a directed acyclic graph defining the dependency hierarchy of your infrastructure and applications. + +A dotted line indicates that a [resource](/concepts/resources-and-types) _has not_ been provisioned yet. + +A solid line indicates that a [resource](/concepts/resources-and-types) _has_ been provisioned. + + + +## Version ranges on a dependency + +A dependency is drawn once on the project blueprint, but the environments under that project can run different bundle versions. Each dependency therefore records a version range for the source component and a version range for the destination component. + +The dependency is only wired up in an environment where the bundle versions deployed there satisfy both ranges. + + + +### How the ranges are set + +You do not type the ranges. When you draw a dependency, Massdriver reads the bundle version at each end and stores that version's compatibility boundary: + +| Deployed version | Stored range | Reason | +|------------------|--------------|--------| +| `1.2.3` | `~1` | For `1.0.0` and above, the major version is the compatibility boundary | +| `2.0.0` | `~2` | A new major version is a new contract | +| `0.4.1` | `~0.4` | Below `1.0.0`, the minor version is the compatibility boundary | +| `1.2.3-dev.20060102T150405Z` | `~1` | A development release shares its base version's boundary | + +A development release satisfies the same range as the version it is based on, so a dependency stays wired while you test a `-dev` build. + +:::caution Bundles below 1.0.0 narrow faster +A `0.x` bundle changes its compatibility boundary on every minor bump. A component moving from `0.4.1` to `0.5.0` leaves the `~0.4` range, and the dependency drawn at `0.4.1` no longer applies to it. Draw the dependency again at the new version, or publish `1.0.0` so the boundary widens to the major version. +::: + +### More than one dependency on the same field + +A destination field can carry several dependencies, as long as no two of them could apply at the same time. Massdriver rejects a new one when its source range **and** its destination range both overlap an existing dependency on that field, because a single pair of deployed versions would then satisfy two at once. + +Non-overlapping pairs are what make a staged rollout work. A field can hold `vpc ~1 → db ~1` and `vpc ~1 → db ~2` together. Every environment satisfies exactly one of them, so staging moves to the new major version on its own schedule and production keeps the wiring it already had. + +### Seeing what an instance is bound to + +Open an instance and select the **Dependencies** tab. Each row shows the dependency's resource type and the version range it accepts, next to the instance currently filling it. The **Resources** tab shows the same for what the instance produces. + +## Dynamic Configuration from Dependencies + +You can use the `$md.enum` annotation in your bundle's params schema to create dynamic dropdown fields that query data from connected resources. This enables users to select from available cloud resources (like subnets, database instances, or IAM roles) that exist in their connected infrastructure. + +For example, a bundle that depends on a VPC can provide a dropdown to select from available subnets, or a bundle that depends on a database cluster can let users pick a specific database instance. + +See the [Massdriver Annotations Reference](/bundle-development/schema-design/massdriver-annotations#mdenuml) for complete documentation and examples. + +## Removing a dependency + +To remove a dependency, click the **X** on the line. + +:::caution +Removing a dependency **without decommissioning the dependent component** may result in an inconsistent state resulting in orphaned resources. +::: + +## Related documentation + +- [Version resolution](/bundle-development/dependencies-resources/version-resolution) — how a dependency's version range picks a resource at deploy time +- [Bundle YAML: dependencies](/bundle-development/bundle-yaml-spec#dependencies) — declaring what a bundle consumes diff --git a/docs/concepts/07-organization-settings.md b/docs/concepts/07-organization-settings.md new file mode 100644 index 00000000..a128afcf --- /dev/null +++ b/docs/concepts/07-organization-settings.md @@ -0,0 +1,103 @@ +--- +id: concepts-organization-settings +slug: /concepts/organization-settings +title: Organization Settings +sidebar_label: Organization Settings +--- + +Organization settings hold behavior that applies across every project in the organization. Changing them requires the `organization:manageSettings` action, which the `organization:manage` umbrella includes. See [Access Control](/platform-operations/security/access-control) for how actions are granted. + +Settings are changed through the `updateOrganizationSettings` mutation. Only the settings you send are changed; the rest keep their current values. + +```graphql +mutation { + updateOrganizationSettings( + organizationId: "your-org-id" + input: { + namingConvention: "{{project.id}}-{{environment.local_id}}-{{instance.local_id}}" + defaultBundleAccess: ALL_PROJECTS + } + ) { + result { id } + } +} +``` + +## Naming convention + +A naming convention is a [Liquid](https://shopify.github.io/liquid/) template that names the cloud resources your bundles provision. It becomes the name prefix for every instance created under the organization. + +The default is: + +```liquid +{{instance.id}}-{{component.suffix}} +``` + +Sending an empty string or `null` restores that default. + +### Atoms + +A template is composed from a fixed vocabulary. The examples below are for a `db` component in the `prod` environment of the `api` project. + +| Atom | Value | Example | +|------|-------|---------| +| `{{org.id}}` | Your organization's id | `sandbox` | +| `{{project.id}}` | The project's id | `api` | +| `{{project.name}}` | The project's display name. May contain spaces | `Checkout API` | +| `{{environment.id}}` | The full environment id, including the project | `api-prod` | +| `{{environment.local_id}}` | The environment id without the project part | `prod` | +| `{{environment.name}}` | The environment's display name. May contain spaces | `Production West` | +| `{{instance.id}}` | The full instance id. Unique per instance in the organization | `api-prod-db` | +| `{{instance.local_id}}` | The component's local id within the project | `db` | +| `{{component.id}}` | The component's id, including the project. The same in every environment | `api-db` | +| `{{component.name}}` | The component's display name. May contain spaces | `Database` | +| `{{component.suffix}}` | A short random token for the component. Shared by the component across environments | `a1b2` | +| `{{attrs.}}` | Any custom attribute your organization has declared | `{{attrs.cost_center}}` | + +`{{attrs.}}` reads the attributes on the project, the environment, and the instance, merged together. The most specific level wins when the same key is set at more than one level. + +### Filters + +Built-in Liquid filters work on any atom: + +```liquid +{{project.id}}-{{environment.name | replace: " ", "-" | downcase}}-{{instance.local_id}} +``` + +Liquid **tags** — anything in `{% %}` — are rejected. A template is a composition of atoms and filters, nothing more. + +### Rules + +A template is validated when you save it, so a broken template never reaches a deployment. + +- **The name should be distinct per instance.** The template should include `{{instance.id}}`, or `{{component.id}}` together with an environment atom, or `{{project.id}}` together with an environment atom and `{{instance.local_id}}`. `{{component.suffix}}` alone is not enough, because a component shares its suffix across every environment. +- **Every atom must be in the table above.** An atom outside the vocabulary is rejected, and the error names it. +- **Attributes must be declared.** `{{attrs.}}` is rejected when your organization has not declared `` as a custom attribute. +- **255 characters maximum.** +- **No control characters.** Tabs, newlines, and carriage returns are rejected. Spaces are allowed. + +### When the name is applied + +The name is resolved when an instance is first created and does not change afterward. Adopting a convention, or changing one, affects the instances you create next. Instances that already exist keep the names their infrastructure is running under. + +## Default bundle access + +`defaultBundleAccess` controls what access a bundle repository gets at the moment it is created. + +| Value | Behavior | +|-------|----------| +| `NONE` (default) | A new repository stays restricted until you author a grant for it | +| `ALL_PROJECTS` | A new repository is created with an org-wide `repo:pull` grant, so every project can use its bundles | + +The grant `ALL_PROJECTS` creates is an ordinary grant row. It is listed on the repository and you can revoke it with `deleteGrant`, the same as a grant you author by hand. + +Two limits are worth knowing: + +- The setting applies to **bundle** repositories. Resource type repositories are catalog metadata and are not grant-gated, so they are unaffected. +- The setting applies **at creation**. Turning it on does not grant access to repositories that already exist, and turning it off does not revoke grants it created earlier. + +## Related documentation + +- [Organizations](/concepts/organizations) — members, groups, and finding your organization id +- [Access Control](/platform-operations/security/access-control) — actions, attributes, and grants +- [Identifier constraints](/reference/identifier-constraints) — the shape of the ids the atoms produce diff --git a/docs/concepts/10-deployments.md b/docs/concepts/10-deployments.md index 6d66ed50..0068d5eb 100644 --- a/docs/concepts/10-deployments.md +++ b/docs/concepts/10-deployments.md @@ -56,6 +56,24 @@ Use this when a change requires review before applying — for example, producti > `PLAN` deployments are not part of the propose-and-approve flow. A plan is a non-destructive preview, so it does not require approval. +### Separation of duty + +An environment can require that a proposal is approved by somebody other than the person who proposed it. Enable **Separation of duty** on the environment, in the environment form or the environments table. + + + +With the setting on, `approveDeployment` is refused when the subject calling it is the same account or service account that called `proposeDeployment`. Approval has to come from a second reviewer. + +Rejection is not gated. A proposer can always withdraw their own proposal with `rejectDeployment`, so the control adds a second pair of eyes without stranding a change nobody wants. + +The setting is read from the environment at the moment of approval, so turning it on takes effect on proposals that are already open. + +The field is `separationOfDuty` on the V2 API, and `separation_of_duty` on the `massdriver_environment` Terraform resource. + +:::caution Terraform sends this on every apply +`separation_of_duty` and `decommission_protection` default to `false` and are sent on every apply. An environment that was protected outside of Terraform is unprotected on the next apply unless the setting is in your configuration. +::: + ## The lifecycle A deployment walks through a state machine after creation. Direct pushes enter at `PENDING`; proposals enter at `PROPOSED`. @@ -117,7 +135,7 @@ flowchart TB When a deployment is created, the following are frozen into the deployment record: - `params` — the instance's configuration values -- `connection_params` — the resolved [connections](/concepts/connections) +- `connection_params` — the resolved [connections](/concepts/dependencies) - `version` — the bundle release to run - `md_metadata` — system metadata (instance name, tags, deployment id) @@ -188,4 +206,4 @@ The trade-off compared to a merge queue: there is no automatic, clean rollback o ## Related Documentation - [Components, Instances & Deployments](/concepts/components-instances-deployments) — how deployments fit into the overall lifecycle. -- [Connections](/concepts/connections) — what `connection_params` resolves from. +- [Connections](/concepts/dependencies) — what `connection_params` resolves from. diff --git a/docs/getting_started/02-connecting-bundles.md b/docs/getting_started/02-connecting-bundles.md index d632ce1a..5345919e 100644 --- a/docs/getting_started/02-connecting-bundles.md +++ b/docs/getting_started/02-connecting-bundles.md @@ -108,9 +108,16 @@ This schema defines exactly what JSON structure bundles must produce and consume 1. From your getting-started repository root, run: ```bash - mass definition publish -f artifact-definitions/getting-started.json + mass resource-type publish artifact-definitions/getting-started.json ``` + :::note + Publishing a raw JSON schema is deprecated and prints a warning. A raw schema has no + version of its own, so it cannot take part in resource type versioning. For your own + resource types, author a `massdriver.yaml` instead — see the + [Resource Type Spec](/bundle-development/dependencies-resources/resource-type-spec). + ::: + 2. You should see output like: ``` diff --git a/docs/guides/bootstrap-platform.md b/docs/guides/bootstrap-platform.md index 26af99be..9e695dfd 100644 --- a/docs/guides/bootstrap-platform.md +++ b/docs/guides/bootstrap-platform.md @@ -27,7 +27,7 @@ This is useful for answering questions like: Should you have separate `postgres` 1. **Model** - Publish the catalog to your instance, add bundles to canvases, connect them, configure parameters 2. **Implement** - Replace placeholder code with your OpenTofu/Terraform -3. **Iterate** - Add more bundles, create [custom resource types](/guides/custom-artifact-definition), use [release channels](/bundle-development/publishing/versioning#release-channels) +3. **Iterate** - Add more bundles, create [custom resource types](/guides/custom-resource-type), use [release channels](/bundle-development/publishing/versioning#release-channels) ## Prerequisites @@ -42,7 +42,7 @@ The repository contains setup instructions, customization guides, and examples. ## Related -- [Custom Resource Types](/guides/custom-artifact-definition) +- [Custom Resource Types](/guides/custom-resource-type) - [Core Resource Types](https://github.com/massdriver-cloud/artifact-definitions) - [Massdriver Slack](https://massdriver.cloud/slack) diff --git a/docs/guides/bundle-from-opentofu.md b/docs/guides/bundle-from-opentofu.md index 6d5d5f7d..0fcca701 100644 --- a/docs/guides/bundle-from-opentofu.md +++ b/docs/guides/bundle-from-opentofu.md @@ -46,7 +46,7 @@ Be sure to complete the [prerequisites](https://docs.massdriver.cloud/getting-st ✔ Path to an existing opentofu-module to generate params from, leave blank to skip: path/to/your/module█ ``` -4. Next you will be prompted to specify any [connections](https://docs.massdriver.cloud/concepts/connections). [Connections](https://docs.massdriver.cloud/concepts/connections) are dependencies your module has on external bundles or inputs. If your module needs to authenticate to AWS, GCP or Azure, be sure to specify the appropriate credential: +4. Next you will be prompted to specify any [connections](https://docs.massdriver.cloud/concepts/dependencies). [Connections](https://docs.massdriver.cloud/concepts/dependencies) are dependencies your module has on external bundles or inputs. If your module needs to authenticate to AWS, GCP or Azure, be sure to specify the appropriate credential: | Cloud | Connection | Name | |-------|--------------------------------------|---------------------------| diff --git a/docs/guides/custom_artifact_definition.md b/docs/guides/custom_artifact_definition.md deleted file mode 100644 index 085ad87e..00000000 --- a/docs/guides/custom_artifact_definition.md +++ /dev/null @@ -1,292 +0,0 @@ ---- -id: custom-artifact-definition -slug: /guides/custom-artifact-definition -title: Crafting Custom Resource Types -sidebar_label: Custom Resource Type ---- - - - -# Crafting Custom Resource Types - -In this guide, we're going to walk through the steps to create your own custom resource types in Massdriver. This is for those moments when the existing definitions just don't cut it for your unique needs. Let's demystify the process and make it as straightforward as possible. And, just in case you're looking for a primer on what resources and resource types actually are, make sure to check out the [Resources & Resource Types](/concepts/resources-and-types) concepts page. - -## How to Create Your Own Custom Resource Type - -### Step 1: Spotting the Need - -Check out the Massdriver [resource types GitHub repo](https://github.com/massdriver-cloud/artifact-definitions/tree/main/definitions/artifacts) first. If what you need is nowhere to be found, that's your green light to craft something custom. - -:::tip Bootstrap Your Resource Types - -If you're setting up a self-hosted Massdriver instance, check out the **[Massdriver Catalog](https://github.com/massdriver-cloud/massdriver-catalog)**. It includes example resource types for common infrastructure patterns (networks, databases, storage) that you can customize for your organization. This is a great starting point for designing your platform's resource type contracts before implementing infrastructure code. - -::: - -### Step 2: Getting Started - -With the [Massdriver CLI](/reference/cli/overview), you've got the toolkit you need to forge your own definitions. It's usually easier to tweak an existing one than to start from scratch: - -1. **Pick a Starting Point**: Hunt down an existing resource type that's close to what you need, or use this starting template: - -```json artifact-definition-name.json -{ - "$md": { - "name": "artifact-definition-name" - }, - "type": "object", - "title": "Resource Type Name", - "description": "", - "additionalProperties": false, - "properties": { - "authentication": { - "title": "Authentication", - "type": "object", - "properties": {} - }, - "infrastructure": { - "title": "Infrastructure", - "type": "object", - "properties": {} - } - } -} -``` - -2. **Make It Your Own**: Copy its content into your favorite editor (like VS Code) and start tweaking it to suit your requirements. - -### Step 3: Key Components of a Resource Type - -Structure your resource type to match your infrastructure abstraction. Group related properties logically and use `$md.sensitive: true` to protect sensitive fields like passwords and tokens. - -### Step 4: Tailoring Your Definition - -1. **Define Your Structure**: Add properties that match your infrastructure needs. -2. **Mark Sensitive Fields**: Use `$md.sensitive: true` for passwords, tokens, and other secrets. -3. **Prune What You Don't Need**: If the copied definition includes irrelevant bits, cut them out or alter them. - -By the end of this step, your definition should look something like this: -```json -{ - "$md": { - "name": "artifact-definition-name" - }, - "type": "object", - "title": "Resource Type Name", - "description": "", - "additionalProperties": false, - "required": [ - "infrastructure", - "authentication" - ], - "properties": { - "infrastructure": { - "title": "Infrastructure configuration", - "type": "object", - "required": [ - "foo", - "bar" - ], - "properties": { - "foo": { - "type": "string", - "title": "Foo", - "description": "Foo description", - "examples": [], - "pattern": "^.*+$", - "message": { - "pattern": "Must be a valid format for foo." - } - }, - "bar": { - "title": "Bar", - "description": "Bar description", - "additionalProperties": false, - "examples": [], - "type": "string" - } - } - }, - "authentication": { - "title": "Authentication configuration", - "type": "object", - "required": [ - "token" - ], - "properties": { - "token": { - "title": "Token", - "type": "string", - "$md": { - "sensitive": true - } - } - } - }, - "iam": { - "title": "IAM", - "description": "IAM Roles And Scopes", - "additionalProperties": false, - "patternProperties": { - "^[a-z]+[a-z_]*[a-z]$": { - "type": "object", - "required": [ - "role", - "scope" - ], - "properties": { - "role": { - "title": "Role", - "description": "Cloud Role", - "pattern": "^[a-zA-Z ]+$", - "message": { - "pattern": "Must be a valid Cloud Role (uppercase, lowercase letters and spaces)" - }, - "examples": [ - "Data Reader" - ] - }, - "scope": { - "title": "Scope", - "description": "Cloud IAM Scope (cloud resource identifier)", - "type": "string" - } - } - } - } - }, - "cloud": { - "type": "object", - "properties": { - "region": { - "type": "string", - "title": "Cloud Region", - "description": "Select the cloud region you'd like to provision your resources in." - } - } - } - } -} -``` - -### Step 5: Publishing to Massdriver - -Got your definition looking sharp? Use the `mass definition publish /path/to/definition.json` command in the CLI to send it out into the world. - -### Step 6: Fetching Your Masterpiece - -Once published, snag your resource type with the `mass definition get org/definition-name` command to confirm it's ready for action in your bundles. - -### Step 7: Using Your Custom Resource Type - -Now that your custom resource type is published, you can use it in your bundles. Just reference it in your bundle's `artifacts:` field (the YAML key remains `artifacts:` for backwards compatibility) and structure your `_artifacts.tf` file, and you're good to go. - -:::tip Recommended: Omit Organization Prefix -When referencing resource types from your own organization, you can omit the organization prefix. Massdriver will automatically use your organization's definitions. This keeps your bundle configuration cleaner and more portable. -::: - -``` yaml massdriver.yaml -artifacts: - required: - - artifact_definition_name - properties: - artifact_definition_name: - # Recommended: omit org prefix for your own resource types - $ref: artifact-definition-name - # Also valid: acme/artifact-definition-name -``` - -``` hcl src/_artifacts.tf -resource "massdriver_artifact" "artifact_definition_name" { - field = "artifact_definition_name" - provider_resource_id = artifact_dummy_resource.main.id - name = "Artifact Dummy Resource ${var.md_metadata.name_prefix}" - artifact = jsonencode( - { - infrastructure = { - foo = artifact_dummy_resource.main.foo - bar = artifact_dummy_resource.main.bar - } - authentication = { - token = artifact_dummy_resource.main.token - } - iam = { - "read" = { - role = "Data Reader" - scope = artifact_dummy_resource.main.id - } - } - cloud = { - region = artifact_dummy_resource.main.region - } - } - ) -} -``` - -To confirm that your custom resource type is working as expected for your bundle, run the `mass bundle lint` and `mass bundle build` commands to check for any issues. When you're ready to publish your bundle changes, `mass bundle publish` will publish your bundle to your Bundle Catalog. - -## Customizing Massdriver - -### Customizing Onboarding - -Massdriver lets you fully customize the onboarding experience for cloud credentials and other resource types. You can define onboarding instructions, UI labels, and icons directly in your resource type using the `$md` and `$md.ui` fields. This enables you to provide clear, step-by-step guidance for your users when they add new credentials. - -For example, the onboarding panel that appears on the right side of the **Import Resource** dialog (shown below for the `aws-iam-role` type) is rendered straight from its `ui.instructions` array. Each instruction has a `label` and `content` field, so you can walk users through CLI commands, console clicks, or a one-click flow — whichever combination makes sense for the resource. - - - -**Relevant schema fields:** -- `$md.label`: Sets the display name for your resource type in the UI. -- `$md.icon`: Sets a custom icon for your resource type. -- `$md.ui.instructions`: An array of onboarding steps, each with a `label` and `content`, shown to users during credential setup. - -See a real-world example of onboarding instructions in the [aws-iam-role resource type](https://github.com/massdriver-cloud/artifact-definitions/blob/main/definitions/artifacts/aws-iam-role.json#L20). - -### Note on Icons and Instructions - -Currently, icons (as data URLs) and `instructions.content` (as base64-encoded markdown) are packed directly into the JSON Schema. With our upcoming move to OCI for resource types (as we've already done for bundles), you'll soon be able to include these files directly in the same directory as your definition—no more packing required. Stay tuned for updates! Here is the [script](https://github.com/massdriver-cloud/artifact-definitions/blob/main/hack/pack.rb) we use for packaging resource types. - -### Customizing the Resource Types that can be defaulted in an Environment - -Massdriver environments support "environment default" resources—things like credentials, networks, or DNS zones that are commonly shared across multiple bundles. Any resource type appears as a selectable **Resource Type** in the **Environment Defaults** dialog, letting users pin a default for that type without wiring it into every bundle. The recording below shows the dialog in action — setting a Kubernetes Cluster default for an environment. - - - -**Relevant schema fields:** - -- `$md.ui.connectionOrientation`: Controls how resources of this type appear on the canvas. If set to `"link"`, users can draw lines to connect bundles to the resource. If set to `"environmentDefault"`, the resource is only shown as a default and not as a connectable box. These options are independently controllable, so you can allow both defaulting and explicit connections if desired. For example, SREs might want to draw lines to a shared Kubernetes cluster, while end developers only see it as a default and don't interact with it directly. - -**Example snippet:** -```json -{ - "$md": { - "name": "aws-iam-role", - "label": "My Cloud Credential", - "icon": "https://example.com/my-icon.svg", - "ui": { - "connectionOrientation": "environmentDefault", - "instructions": [ - { - "label": "Step 1: Create a Service Account", - "content": "Go to your cloud provider and create a new service account..." - } - ] - } - } -} -``` - -**References:** -- [Resource type JSON Schema](https://api.massdriver.cloud/json-schemas/artifact-definition.json) (the JSON Schema URL retains the legacy filename) -- [Open source resource types](https://github.com/massdriver-cloud/artifact-definitions) - -By leveraging these schema fields, you can tailor both the onboarding experience and environment default behavior for your custom resource types, ensuring a seamless and intuitive experience for your users. - -## Wrapping Up - -And there you have it! Creating your own resource types in Massdriver opens up a world of customization for your cloud infrastructure projects. By following these steps, you're well on your way to tailoring Massdriver to your project's unique requirements. If you've got any questions or need a hand, don't hesitate to reach out to our team. We're here to help you make the most of Massdriver's powerful features. Happy crafting! diff --git a/docs/guides/custom_resource_type.md b/docs/guides/custom_resource_type.md new file mode 100644 index 00000000..b4429763 --- /dev/null +++ b/docs/guides/custom_resource_type.md @@ -0,0 +1,323 @@ +--- +id: custom-resource-type +slug: /guides/custom-resource-type +title: Crafting Custom Resource Types +sidebar_label: Custom Resource Type +--- + + + +# Crafting Custom Resource Types + +This guide walks through creating your own resource type in Massdriver, for when the existing types do not cover what you need. For a primer on what resources and resource types are, see the [Resources & Resource Types](/concepts/resources-and-types) concepts page. + +A resource type is authored as a `massdriver.yaml` in its own directory, published to your organization's catalog as a versioned OCI artifact, and referenced from bundles by name and version. + +## Step 1: Check whether one already exists + +Look through the Massdriver [resource types GitHub repo](https://github.com/massdriver-cloud/artifact-definitions/tree/main/definitions/artifacts) first. If nothing there fits, write your own. + +:::tip Bootstrap Your Resource Types + +If you're setting up a self-hosted Massdriver instance, check out the **[Massdriver Catalog](https://github.com/massdriver-cloud/massdriver-catalog)**. It includes example resource types for common infrastructure patterns (networks, databases, storage) that you can customize for your organization. This is a great starting point for designing your platform's resource type contracts before implementing infrastructure code. + +::: + +## Step 2: Create the resource type + +Use the [Massdriver CLI](/reference/cli/overview) to create the repository in your catalog: + +```bash +mass resource-type create my-resource-type +``` + +A resource type lives in a directory: + +``` +my-resource-type/ +├── massdriver.yaml +├── instructions/ +│ └── console.md +└── exports/ + └── config.yaml.liquid +``` + +Start from this template: + +```yaml massdriver.yaml +name: my-resource-type +version: 0.1.0 +label: My Resource Type + +schema: + type: object + title: My Resource Type + additionalProperties: false + properties: + authentication: + title: Authentication + type: object + properties: {} + infrastructure: + title: Infrastructure + type: object + properties: {} +``` + +If you already have a resource type as a raw JSON schema, convert it rather than rewriting it by hand: + +```bash +mass resource-type convert ./my-resource-type.json +``` + +`convert` writes a `massdriver.yaml` alongside the schema and pulls any inlined instruction and export content back out into referenced files. It writes a placeholder `version` — set a real one before you publish. + +## Step 3: Shape the schema + +Structure the schema to match your infrastructure abstraction. Group related properties, and mark secrets with `$md.sensitive: true` so their values are masked in API responses and the UI. + +```yaml massdriver.yaml +name: my-resource-type +version: 1.0.0 +label: My Resource Type +icon: https://example.com/my-icon.svg + +schema: + type: object + title: My Resource Type + additionalProperties: false + required: + - infrastructure + - authentication + properties: + infrastructure: + title: Infrastructure configuration + type: object + required: + - foo + - bar + properties: + foo: + type: string + title: Foo + description: Foo description + pattern: "^.*+$" + message: + pattern: Must be a valid format for foo. + bar: + type: string + title: Bar + description: Bar description + + authentication: + title: Authentication configuration + type: object + required: + - token + properties: + token: + title: Token + type: string + $md.sensitive: true + + iam: + title: IAM + description: IAM Roles And Scopes + additionalProperties: false + patternProperties: + "^[a-z]+[a-z_]*[a-z]$": + type: object + required: + - role + - scope + properties: + role: + title: Role + description: Cloud Role + pattern: "^[a-zA-Z ]+$" + message: + pattern: Must be a valid Cloud Role (uppercase, lowercase letters and spaces) + examples: + - Data Reader + scope: + title: Scope + description: Cloud IAM Scope (cloud resource identifier) + type: string + + cloud: + type: object + properties: + region: + type: string + title: Cloud Region + description: Select the cloud region you'd like to provision your resources in. +``` + +:::note +`$md.sensitive` is a single key with a dot in it, written at the same level as `type` and `title`. It is not a nested `$md` object. +::: + +## Step 4: Publish + +```bash +mass resource-type publish ./my-resource-type +``` + +Publishing is immutable. Once `1.0.0` exists it cannot be overwritten, so anything pinned to it keeps resolving to what it resolved to the first time. Bump `version` in `massdriver.yaml` for each change, following [semantic versioning](/bundle-development/publishing/versioning) — adding a required field to the payload is a breaking change for every bundle that consumes it, so it needs a major bump. + +The published artifact carries the `massdriver.yaml`, the readme, the changelog, the icon, and the instruction and export files the `massdriver.yaml` references. A referenced file that does not exist, or that resolves outside the directory, fails the publish. + +## Step 5: Confirm it published + +```bash +mass resource-type get my-resource-type +mass resource-type pull my-resource-type@1.0.0 +``` + +## Step 6: Use it in a bundle + +Name the resource type and the versions you accept in the bundle's `massdriver.yaml`. `resources` is what the bundle produces; `dependencies` is what it consumes. + +:::tip Recommended: Omit Organization Prefix +When referencing resource types from your own organization, you can omit the organization prefix. Massdriver will automatically use your organization's resource types. This keeps your bundle configuration cleaner and more portable. +::: + +```yaml massdriver.yaml +resources: + my_resource: + # Recommended: omit the org prefix for your own resource types + # Also valid: acme/my-resource-type@1.0.0 + resource_type: my-resource-type@1.0.0 + required: true +``` + +A bundle that consumes it declares a range instead of a pinned version: + +```yaml massdriver.yaml +dependencies: + my_resource: + resource_type: my-resource-type@~1 + required: true +``` + +See [Version Resolution](/bundle-development/dependencies-resources/version-resolution) for the accepted range forms and how a range is matched at deploy time. + +Produce the resource from your OpenTofu module: + +```hcl src/_artifacts.tf +resource "massdriver_artifact" "my_resource" { + field = "my_resource" + provider_resource_id = my_dummy_resource.main.id + name = "My Resource ${var.md_metadata.name_prefix}" + artifact = jsonencode( + { + infrastructure = { + foo = my_dummy_resource.main.foo + bar = my_dummy_resource.main.bar + } + authentication = { + token = my_dummy_resource.main.token + } + iam = { + "read" = { + role = "Data Reader" + scope = my_dummy_resource.main.id + } + } + cloud = { + region = my_dummy_resource.main.region + } + } + ) +} +``` + +> The OpenTofu provider resource is named `massdriver_artifact` for backwards compatibility. It produces a Massdriver resource. + +Run `mass bundle lint` and `mass bundle build` to check the bundle, then `mass bundle publish` to publish it. + +## Customizing Massdriver + +### Customizing Onboarding + +Massdriver lets you customize the onboarding experience for cloud credentials and other resource types. Onboarding instructions, UI labels, and icons are declared in the `massdriver.yaml`, so you can give users step-by-step guidance when they add a new credential. + +The onboarding panel on the right side of the **Import Resource** dialog — shown below for the `aws-iam-role` type — is rendered from the `ui.instructions` array. Each instruction has a `label` and a `path` to a markdown file, so you can walk users through CLI commands, console clicks, or a one-click flow. + + + +**Relevant fields:** + +- `label`: The display name for your resource type in the UI. +- `icon`: A custom icon for your resource type. +- `ui.instructions`: Onboarding steps, each with a `label` and a `path` to a markdown file. + +```yaml massdriver.yaml +name: my-cloud-credential +version: 1.0.0 +label: My Cloud Credential +icon: https://example.com/my-icon.svg + +ui: + connectionOrientation: environmentDefault + instructions: + - label: "Step 1: Create a Service Account" + path: ./instructions/create-service-account.md + - label: "Step 2: Grant Access" + path: ./instructions/grant-access.md + +schema: + type: object + title: My Cloud Credential + properties: {} +``` + +Instruction content and icons are files in the resource type's directory. They are packaged into the published artifact by `mass resource-type publish`, so nothing has to be base64-encoded or inlined into the schema. + +See a real-world example of onboarding instructions in the [aws-iam-role resource type](https://github.com/massdriver-cloud/artifact-definitions/blob/main/definitions/artifacts/aws-iam-role.json#L20). + +### Offering a downloadable file + +`exports` gives users a download button on the resource, rendering a [Liquid](https://shopify.github.io/liquid/) template against the resource's data: + +```yaml massdriver.yaml +exports: + - downloadButtonText: Download .env + fileFormat: env + templatePath: ./exports/dotenv.liquid + templateLang: liquid +``` + +```liquid exports/dotenv.liquid +DATABASE_HOST={{ artifact.authentication.hostname }} +DATABASE_PORT={{ artifact.authentication.port }} +DATABASE_USER={{ artifact.authentication.username }} +DATABASE_PASSWORD={{ artifact.authentication.password }} +``` + +### Customizing the resource types that can be defaulted in an environment + +Massdriver environments support environment default resources — credentials, networks, or DNS zones that are shared across multiple bundles. Any resource type appears as a selectable **Resource Type** in the **Environment Defaults** dialog, so users can pin a default for that type without wiring it into every bundle. The recording below shows a Kubernetes Cluster default being set for an environment. + + + +**Relevant fields:** + +- `ui.connectionOrientation`: How resources of this type appear on the canvas. `link` lets users draw lines to connect bundles to the resource. `environmentDefault` shows the resource only as a default, not as a connectable box. For example, SREs might draw lines to a shared Kubernetes cluster while developers only see it as a default. +- `ui.environmentDefaultGroup`: Groups the type with others in the environment defaults panel. The group named `credentials` holds cloud credential types, which the UI separates from the rest. + +**References:** + +- [Resource Type Spec](/bundle-development/dependencies-resources/resource-type-spec) — the complete `massdriver.yaml` field reference +- [Version Resolution](/bundle-development/dependencies-resources/version-resolution) — how a version range picks a resource at deploy time +- [Open source resource types](https://github.com/massdriver-cloud/artifact-definitions) + +## Publishing a raw JSON schema + +`mass resource-type publish` still accepts a raw JSON or YAML schema file, the format that predates `massdriver.yaml`, and prints a deprecation warning. That path will be removed in a future release. + +A raw schema carries no version of its own. It is stored as the resource type's unversioned `0.0.0` document, cannot be pinned by version, and cannot be pulled back down. Run `mass resource-type convert` to move one to `massdriver.yaml`. diff --git a/docs/guides/import-artifact.md b/docs/guides/import-artifact.md index 0577dc71..a73c77e8 100644 --- a/docs/guides/import-artifact.md +++ b/docs/guides/import-artifact.md @@ -14,7 +14,7 @@ Why import a resource? A few common scenarios: ## Prerequisites -To import a resource, you need a [resource type](/concepts/resources-and-types) (its schema). You can create a [custom resource type](/guides/custom-artifact-definition) or use an [official Massdriver resource type](https://github.com/massdriver-cloud/artifact-definitions/tree/main/definitions/artifacts). +To import a resource, you need a [resource type](/concepts/resources-and-types) (its schema). You can create a [custom resource type](/guides/custom-resource-type) or use an [official Massdriver resource type](https://github.com/massdriver-cloud/artifact-definitions/tree/main/definitions/artifacts). You also need a payload that conforms to that resource type. Example schema and payload: diff --git a/docs/introduction.md b/docs/introduction.md index 4dc07267..d125d6f2 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -13,7 +13,7 @@ Massdriver is an internal developer platform that turns infrastructure-as-code i **Resources** are the structured outputs each bundle produces — credentials, ARNs, hostnames, etc. — described by **resource types** (JSON Schema contracts). Resource types pass state between infrastructure modules, even across different IaC tools, and enable automatic configuration: binding IAM policies, injecting credentials, connecting services. -**The canvas** lets developers drag bundles, connect them visually, and deploy. Massdriver validates connections using resource type schemas, so developers don't need deep cloud knowledge to assemble working infrastructure. +**The canvas** lets developers drag bundles, connect them visually, and deploy. Massdriver validates dependencies using resource type schemas, so developers don't need deep cloud knowledge to assemble working infrastructure. **Ephemeral pipelines** run IaC automatically when developers deploy. No pipeline code to maintain. Massdriver orchestrates the workflow, manages state, and runs compliance checks on every deployment. diff --git a/docs/platform-operations/security/03-graphql-permissions.md b/docs/platform-operations/security/03-graphql-permissions.md index 8779aed1..ee4a8cea 100644 --- a/docs/platform-operations/security/03-graphql-permissions.md +++ b/docs/platform-operations/security/03-graphql-permissions.md @@ -202,7 +202,8 @@ There is no `updateGrant` — grants are immutable; delete and re-create to chan | Operation | Type | Required permission(s) | Notes | |---|---|---|---| | `accessTokens` | Query | *no explicit gate* | Open to every org member; only returns your own tokens — admins cannot list other principals' tokens. | -| `createAccessToken` | Mutation | *no explicit gate* | Open to every org member; issues a token for the calling subject. | +| `createPersonalAccessToken` | Mutation | *no explicit gate* | Open to every org member; issues a token for the calling account. Human accounts only. Expiry capped at 1 year. | +| `createServiceAccountAccessToken` | Mutation | *no explicit gate* | Issues a token for the calling service account. Service accounts only. Expiry capped at 10 years. | | `revokeAccessToken` | Mutation | *owner-only* | Owner-scoped: only the token's owning subject can revoke; admins cannot revoke another user's personal tokens. | ## Integration diff --git a/docs/platform-operations/self-hosted/01-install.md b/docs/platform-operations/self-hosted/01-install.md index d0122774..c31ed440 100644 --- a/docs/platform-operations/self-hosted/01-install.md +++ b/docs/platform-operations/self-hosted/01-install.md @@ -107,6 +107,18 @@ Edit your `values-custom.yaml` file to provide the necessary configuration. Focu ``` +#### Optional Configuration + +**Documentation link** + +The documentation link in the sidebar points at `https://docs.massdriver.cloud`. Installations that serve their own documentation can point it somewhere else with `MD_DOCS_URL`: + +```bash +MD_DOCS_URL=https://docs.internal.example.com +``` + +Leave it unset to keep the default. + :::info Custom Release Name (Optional) If you plan to use a different release name than `massdriver`, search for `"release name"` in the values file and update the associated values accordingly. diff --git a/docs/platform-operations/self-hosted/05-dynamic-credentials.md b/docs/platform-operations/self-hosted/05-dynamic-credentials.md index 7ae9c2cc..3a982613 100644 --- a/docs/platform-operations/self-hosted/05-dynamic-credentials.md +++ b/docs/platform-operations/self-hosted/05-dynamic-credentials.md @@ -49,7 +49,7 @@ kubectl get serviceaccount -n massdriver -l app.kubernetes.io/component=provisio :::warning Republish Bundles After Publishing a Resource Type -Bundles burn in their connection schema when they are published, and Massdriver validates every connection against that schema before a deployment runs. Whenever you change the schema of any resource type, be sure to run `mass bundle build` and `mass bundle publish` on the bundles that connect to it. +Bundles burn in their dependency schema when they are published, and Massdriver validates every dependency against that schema before a deployment runs. Whenever you change the schema of any resource type, be sure to run `mass bundle build` and `mass bundle publish` on the bundles that depend on it. ::: @@ -200,7 +200,7 @@ mass resource-type publish aws-iam-role/massdriver.yaml ### Step 5: Use the Credential in Your Bundles -Declare the credential as a connection: +Declare the credential as a dependency: ```yaml massdriver.yaml connections: @@ -329,7 +329,7 @@ mass resource-type publish gcp-service-account/massdriver.yaml ### Step 5: Use the Credential in Your Bundles -Declare the credential as a connection: +Declare the credential as a dependency: ```yaml massdriver.yaml connections: @@ -465,7 +465,7 @@ mass resource-type publish azure-service-principal/massdriver.yaml ### Step 5: Use the Credential in Your Bundles -Declare the credential as a connection: +Declare the credential as a dependency: ```yaml massdriver.yaml connections: @@ -529,10 +529,10 @@ If `AZURE_FEDERATED_TOKEN_FILE` is unset inside the pod, the webhook did not mut | AWS `AccessDenied` on `sts:AssumeRole` | Either the provisioner role lacks `sts:AssumeRole` on the target role, or the target role's trust policy or external ID condition does not match. | | GCP `Permission 'iam.serviceAccounts.getAccessToken' denied` | The provisioner service account is missing `roles/iam.serviceAccountTokenCreator` on the target service account, or the IAM Credentials API is not enabled in the target project. | | Azure `AZURE_FEDERATED_TOKEN_FILE` not set | The `azure.workload.identity/use: "true"` pod label is not reaching the workflow pods. Check `kubectl get pod -n massdriver POD_NAME -o jsonpath='{.metadata.labels}'`. | -| Deployment fails validation with a missing required property on a connection | A bundle still carries the old burned-in credential schema. Re-run `mass bundle build` and `mass bundle publish`. | +| Deployment fails validation with a missing required property on a dependency | A bundle still carries the old burned-in credential schema. Re-run `mass bundle build` and `mass bundle publish`. | ## Related Configuration - Dynamic credentials cover **provisioning only**. Massdriver's blob storage access is a separate identity configured through `massdriver.blobStorage.serviceAccount.annotations` — see [Cloud Storage](/platform-operations/self-hosted/cloud-storage). - The [AWS Cost and Usage Report](/reference/integrations/aws-cost-and-usage-reports) and [Azure Cost Management](/reference/integrations/azure-cost-management-exports) integrations provision their own credentials and are unaffected. -- For background on shaping credential resource types, see [Customizing Cloud Support](/guides/customizing-cloud-support) and [Crafting Custom Resource Types](/guides/custom-artifact-definition). +- For background on shaping credential resource types, see [Customizing Cloud Support](/guides/customizing-cloud-support) and [Crafting Custom Resource Types](/guides/custom-resource-type). diff --git a/docs/reference/integrations/01-aws-cost-and-usage-reports.md b/docs/reference/integrations/01-aws-cost-and-usage-reports.md index 63ceed94..a5ef48b9 100644 --- a/docs/reference/integrations/01-aws-cost-and-usage-reports.md +++ b/docs/reference/integrations/01-aws-cost-and-usage-reports.md @@ -112,11 +112,21 @@ The IAM user has these minimal permissions: The CUR report is configured with: - **Time Granularity**: Daily -- **Format**: CSV (text/csv) -- **Compression**: ZIP +- **Format**: ZIP-compressed CSV, or Parquet - **Additional Schema Elements**: RESOURCES (resource-level details) - **Report Versioning**: OVERWRITE_REPORT +### Report format + +Massdriver reads both ZIP-compressed CSV and Parquet reports. Set the integration's **Report Format** field to match what AWS delivers: + +| Value | AWS report format | +|-------|-------------------| +| `zip` (default) | ZIP-compressed CSV | +| `parquet` | Parquet | + +Parquet reports name their columns in snake_case with native types. Massdriver maps them to the same fields it reads from a CSV report, so cost attribution behaves identically either way. + :::note Cost and Usage Reports can only be created in `us-east-1`, but the S3 bucket can be created in any AWS region. Specify the region where your bucket is located when configuring the integration. ::: diff --git a/sidebars.js b/sidebars.js index 667a2861..53044d22 100644 --- a/sidebars.js +++ b/sidebars.js @@ -23,10 +23,11 @@ module.exports = { }, { type: "category", - label: "Connections & Resources", - link: { type: "doc", id: "bundle-development/connections-artifacts/connections-artifacts-overview" }, + label: "Dependencies & Resources", + link: { type: "doc", id: "bundle-development/dependencies-resources/dependencies-resources-overview" }, items: [ - "bundle-development/connections-artifacts/artifact-definition-spec", + "bundle-development/dependencies-resources/resource-type-spec", + "bundle-development/dependencies-resources/version-resolution", ], }, { @@ -122,8 +123,9 @@ module.exports = { "concepts/concepts-projects-and-environments", "concepts/concepts-components-instances-deployments", "concepts/concepts-deployments", - "concepts/concepts-connections", + "concepts/concepts-dependencies", "concepts/concepts-organizations", + "concepts/concepts-organization-settings", ], }, { diff --git a/src/components/Diagrams/SeparationOfDuty.js b/src/components/Diagrams/SeparationOfDuty.js new file mode 100644 index 00000000..9f42be67 --- /dev/null +++ b/src/components/Diagrams/SeparationOfDuty.js @@ -0,0 +1,46 @@ +import React from "react"; +import s from "./styles.module.css"; + +export default function SeparationOfDuty() { + return ( +
+ + + PROPOSED + proposed by dana + params live on the deployment + + + + + DANA APPROVES + refused — same subject proposed it + deployment stays PROPOSED + + + + + + SAM APPROVES + params written to the instance + + APPROVED + + + + dana can still reject the proposal + withdrawing is not gated by the setting + + + the setting is read from the environment at approval time, for user and service account subjects alike + + +
+ ); +} diff --git a/src/components/Diagrams/SlotResolution.js b/src/components/Diagrams/SlotResolution.js new file mode 100644 index 00000000..87056c5d --- /dev/null +++ b/src/components/Diagrams/SlotResolution.js @@ -0,0 +1,56 @@ +import React from "react"; +import s from "./styles.module.css"; + +export default function SlotResolution() { + return ( +
+ + + DEPENDENCY SLOT + network + aws-vpc@~1 + declared in the bundle’s + massdriver.yaml under + dependencies + + filled at deploy time + by the first match + + + + + + + + 1 · REMOTE REFERENCE + shared-vpc, set on this instance + checked for resource type and version on every deploy + + + + + 2 · BLUEPRINT CONNECTION + the link drawn from vpc on the canvas + resource type matched when drawn, version re-checked here + + + + + 3 · ENVIRONMENT DEFAULT + the highest aws-vpc version this range accepts + an environment can hold one default per version + + + + a source that does not match the resource type, or falls outside the range, is skipped for the next one + + +
+ ); +} diff --git a/src/components/Diagrams/VersionedConnections.js b/src/components/Diagrams/VersionedConnections.js new file mode 100644 index 00000000..93d24f42 --- /dev/null +++ b/src/components/Diagrams/VersionedConnections.js @@ -0,0 +1,64 @@ +import React from "react"; +import s from "./styles.module.css"; + +export default function VersionedConnections() { + return ( +
+ + + PROJECT BLUEPRINT + two links on db.network, one per version pair + + + vpc ~1 → db ~1 + db.network reads vpc.network + + + vpc ~1 → db ~2 + db.network reads vpc.private_network + + both environments run aws-vpc 1.4.0 + + + + + + + + DEVELOPMENT + db 2.0.0 + wired to vpc.private_network + + + STAGING + + db 1.2.0 + wired to vpc.network + + + db 2.0.0 + wired to vpc.private_network + + + + PRODUCTION + not upgraded + db 1.2.0 + wired to vpc.network + + + each environment materializes the link whose version ranges its deployed bundles satisfy + + + staging upgraded to db 2.0.0 and picked up the vpc ~1 → db ~2 link with no rewiring + + +
+ ); +} diff --git a/src/components/Diagrams/styles.module.css b/src/components/Diagrams/styles.module.css new file mode 100644 index 00000000..712c317e --- /dev/null +++ b/src/components/Diagrams/styles.module.css @@ -0,0 +1,161 @@ +.figure { + margin: 1.5rem 0 2rem; +} + +.svg { + width: 100%; + height: auto; + display: block; +} + +.caption { + margin-top: 0.5rem; + font-size: 0.85rem; + color: var(--ifm-color-emphasis-600); + text-align: center; +} + +.surface { + fill: var(--ifm-background-surface-color); + stroke: var(--ifm-color-emphasis-300); + stroke-width: 1.5; +} + +.surfaceAccent { + fill: var(--ifm-background-surface-color); + stroke: var(--ifm-color-primary); + stroke-width: 1.5; +} + +.well { + fill: var(--ifm-color-emphasis-100); + stroke: var(--ifm-color-emphasis-300); + stroke-width: 1.5; +} + +.heading { + fill: var(--ifm-color-emphasis-700); + font-family: var(--ifm-font-family-monospace); + font-size: 11px; + font-weight: 700; + letter-spacing: 0.06em; +} + +.name { + fill: var(--ifm-font-color-base); + font-family: var(--ifm-font-family-monospace); + font-size: 13px; + font-weight: 600; +} + +.body { + fill: var(--ifm-color-emphasis-700); + font-family: var(--ifm-font-family-monospace); + font-size: 11px; +} + +.muted { + fill: var(--ifm-color-emphasis-600); + font-family: var(--ifm-font-family-monospace); + font-size: 10px; +} + +.accent { + fill: var(--ifm-color-primary); + font-family: var(--ifm-font-family-monospace); + font-size: 11px; + font-weight: 600; +} + +.accentSmall { + fill: var(--ifm-color-primary); + font-family: var(--ifm-font-family-monospace); + font-size: 10px; +} + +.danger { + fill: #d64545; + font-family: var(--ifm-font-family-monospace); + font-size: 10px; + font-weight: 600; +} + +.barMuted { + fill: var(--ifm-color-emphasis-500); +} + +.barAccent { + fill: var(--ifm-color-primary); +} + +.route { + fill: none; + stroke: var(--ifm-color-emphasis-400); + stroke-width: 1.5; +} + +.routeAccent { + fill: none; + stroke: var(--ifm-color-primary); + stroke-width: 2; +} + +.routeDanger { + fill: none; + stroke: #d64545; + stroke-width: 2; + stroke-dasharray: 4 3; +} + +/* Versioned connections: staging moves to a new major and reads the other route. */ +.stgOld { animation: stgOff 14s linear infinite; } +.stgNew { opacity: 0; animation: stgOn 14s linear infinite; } +.routeOld { animation: stgOff 14s linear infinite; } +.routeNew { opacity: 0; animation: stgOn 14s linear infinite; } +.stgBox { animation: stgBorder 14s linear infinite; } +.capBefore { animation: stgOff 14s linear infinite; } +.capAfter { opacity: 0; animation: stgOn 14s linear infinite; } + +@keyframes stgOff { 0%, 36% { opacity: 1 } 40%, 86% { opacity: 0 } 90%, 100% { opacity: 1 } } +@keyframes stgOn { 0%, 36% { opacity: 0 } 40%, 86% { opacity: 1 } 90%, 100% { opacity: 0 } } +@keyframes stgBorder { + 0%, 36% { stroke: var(--ifm-color-emphasis-300) } + 40%, 86% { stroke: var(--ifm-color-primary) } + 90%, 100% { stroke: var(--ifm-color-emphasis-300) } +} + +/* Slot resolution: each tier takes its turn, highest priority first. */ +.tier1 { animation: tier 18s linear infinite; } +.tier2 { animation: tier 18s linear infinite -6s; } +.tier3 { animation: tier 18s linear infinite -12s; } + +@keyframes tier { + 0% { opacity: 0.35 } + 4%, 29% { opacity: 1 } + 33%, 100% { opacity: 0.35 } +} + +.tierRoute1 { opacity: 0; animation: tierRoute 18s linear infinite; } +.tierRoute2 { opacity: 0; animation: tierRoute 18s linear infinite -6s; } +.tierRoute3 { opacity: 0; animation: tierRoute 18s linear infinite -12s; } + +@keyframes tierRoute { + 0% { opacity: 0 } + 4%, 29% { opacity: 1 } + 33%, 100% { opacity: 0 } +} + +/* Separation of duty: the proposer is turned away, a second reviewer approves. */ +.sodDenied { opacity: 0; animation: sodDenied 16s linear infinite; } +.sodAllowed { opacity: 0; animation: sodAllowed 16s linear infinite; } + +@keyframes sodDenied { 0%, 6% { opacity: 0 } 12%, 44% { opacity: 1 } 50%, 100% { opacity: 0 } } +@keyframes sodAllowed { 0%, 56% { opacity: 0 } 62%, 94% { opacity: 1 } 100% { opacity: 0 } } + +@media (prefers-reduced-motion: reduce) { + .stgOld, .routeOld, .capBefore, .sodDenied { opacity: 0; animation: none; } + .stgNew, .routeNew, .capAfter, .sodAllowed { opacity: 1; animation: none; } + .stgBox { stroke: var(--ifm-color-primary); animation: none; } + .tier1, .tier2, .tier3 { opacity: 1; animation: none; } + .tierRoute1, .tierRoute2, .tierRoute3 { opacity: 1; animation: none; } +} diff --git a/src/theme/MDXComponents.js b/src/theme/MDXComponents.js index 7d78bc34..8f01c8d9 100644 --- a/src/theme/MDXComponents.js +++ b/src/theme/MDXComponents.js @@ -2,10 +2,16 @@ import React from "react"; // Import the original mapper import MDXComponents from "@theme-original/MDXComponents"; import SchemaForm from "@site/src/components/SchemaForm"; +import VersionedConnections from "@site/src/components/Diagrams/VersionedConnections"; +import SlotResolution from "@site/src/components/Diagrams/SlotResolution"; +import SeparationOfDuty from "@site/src/components/Diagrams/SeparationOfDuty"; export default { // Re-use the default mapping ...MDXComponents, // Add custom components SchemaForm, + VersionedConnections, + SlotResolution, + SeparationOfDuty, }; diff --git a/vercel.json b/vercel.json index 58c63c01..b2b87b98 100644 --- a/vercel.json +++ b/vercel.json @@ -74,6 +74,36 @@ "source": "/integrations/scim", "destination": "/reference/integrations/scim", "permanent": true + }, + { + "source": "/guides/custom-artifact-definition", + "destination": "/guides/custom-resource-type", + "permanent": true + }, + { + "source": "/bundle-development/connections-artifacts/artifact-definition-spec", + "destination": "/bundle-development/dependencies-resources/resource-type-spec", + "permanent": true + }, + { + "source": "/concepts/connections", + "destination": "/concepts/dependencies", + "permanent": true + }, + { + "source": "/bundle-development/connections-artifacts/overview", + "destination": "/bundle-development/dependencies-resources/overview", + "permanent": true + }, + { + "source": "/bundle-development/connections-artifacts/resource-type-spec", + "destination": "/bundle-development/dependencies-resources/resource-type-spec", + "permanent": true + }, + { + "source": "/bundle-development/connections-artifacts/version-resolution", + "destination": "/bundle-development/dependencies-resources/version-resolution", + "permanent": true } ] }