Add Radius.Core/terraformConfigs and bicepConfigs resources#11780
Open
Add Radius.Core/terraformConfigs and bicepConfigs resources#11780
Conversation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Unit Tests 2 files ± 0 421 suites +6 6m 29s ⏱️ -32s Results for commit fb85b57. ± Comparison against base commit f1bcb28. This pull request removes 9 and adds 122 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #11780 +/- ##
==========================================
- Coverage 51.45% 50.90% -0.55%
==========================================
Files 699 724 +25
Lines 44130 45373 +1243
==========================================
+ Hits 22705 23096 +391
- Misses 19259 20094 +835
- Partials 2166 2183 +17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Draft
12 tasks
sylvainsf
force-pushed
the
add-terraform-bicep-config
branch
2 times, most recently
from
d38468e to
6543a67
Compare
Define TerraformConfigResource and BicepConfigResource in Radius.Core TypeSpec namespace. Add terraformConfig/bicepConfig reference fields to EnvironmentProperties. Regenerate OpenAPI spec and Go SDK (v20250801preview). Add design doc at docs/architecture/terraform-bicep-config.md.
Add TerraformConfig and BicepConfig datamodel structs reusing existing recipe_types.go sub-types. Add bidirectional API-to-datamodel converters for v20250801preview. Add datamodel/converter bridge files for version routing. Update environment converter to handle terraformConfig/bicepConfig reference fields.
Add resource registrations in SetupRadiusCoreNamespace using default sync CRUD controllers. Add route test entries for Put and Patch operations.
Add FetchTerraformConfig and FetchBicepConfig utilities in pkg/rp/util/. Update getConfigurationV20250801 in the recipe config loader to resolve referenced config resources and populate RecipeConfig. Add PUT-time validation in the environment controller to verify referenced config resource IDs exist.
Test_TerraformConfig_Redis: deploys a Terraform recipe (Redis on K8s) via a Radius.Core/terraformConfigs resource referenced by a Radius.Core/environments resource. Validates env var injection via the new config path. Test_BicepConfig_CRUD: validates CRUD and environment reference wiring for Radius.Core/bicepConfigs without requiring a private registry. Both tests follow existing dynamicrp test patterns alongside recipepacks_test.go.
- TerraformConfig: replace authentication/providers/envSecrets with terraformrc (providerInstallation: networkMirror/direct, credentials) + plain env map. Per spec, provider secrets and envSecrets are handled by recipe parameters, not config. - BicepConfig: replace inline authentication map with structured registryAuthentication (authenticationMethod enum: BasicAuth/AzureWI/ AwsIrsa, plus identity fields and basicAuthSecretId). - Loader bridges new shape into the legacy RecipeConfig consumed by the shared driver: terraformrc.credentials -> Authentication.Git.PAT, env -> EnvironmentVariables, registryAuthentication.basicAuthSecretId -> Bicep.Authentication["default"]. - Updates architecture doc and bicepconfig functional test fixture.
Add ProviderInstallation field to the shared TerraformConfigProperties datamodel and wire it through to the Terraform driver. The driver writes a .terraformrc file in the working directory and points Terraform at it via TF_CLI_CONFIG_FILE during both Deploy and Delete (so terraform init can resolve providers from a network mirror on destroy). The new field is optional and only populated by the Radius.Core path. Applications.Core leaves it nil, so the driver behavior is unchanged for the legacy code path consumed by existing recipeConfig users. Tests: - pkg/recipes/terraform/cliconfig_test.go: HCL rendering, file mode, edge cases, error handling. - TestSetEnvironmentVariables: new case for providerInstallation. - TestApplyTerraformCLIConfig: Delete-path helper. - Fixes pre-existing test compile failure in environment_test.go where getConfigurationV20250801 had a stale call signature.
Test_TerraformConfig_BicepConfig_Combined deploys a single Radius.Core/environments referencing both a terraformConfig (with terraformrc.providerInstallation and env vars) and a bicepConfig, then runs a Terraform recipe end-to-end. This exercises the new .terraformrc rendering path without requiring a network mirror in the test cluster by using providerInstallation.direct (default registry behavior).
- Remove the Implementation plan section (superseded by actual code). - Replace the aspirational Delete protection section with a Status and known limitations section that lists what is wired vs. follow-ups. - Expand the Runtime flow to call out the .terraformrc rendering path added in pkg/recipes/terraform/cliconfig.go. - Add a Code map pointing at the relevant files.
TerraformCredentialConfig and SecretConfig both have a single Secret string field; staticcheck (S1016) prefers a type conversion over a struct literal.
- Regenerate Bicep type definitions for radius.core 2025-08-01-preview to pick up the terraformConfigs/bicepConfigs schema refactor. - gofmt fix on terraformconfig_conversion.go indentation. Generated with mockgen v0.4.0 (matches CI version pinned in .github/workflows/lint.yaml).
…on failure (EOF on 127.0.0.1:45533) affecting 16 unrelated tests
Functional tests were failing with 'resource type not found' because the new types were defined in TypeSpec/codegen and wired into the corerp REST router, but never registered in the UCP manifest files that get baked into the ucpd image at build time. Adds the resource types to both the self-hosted (production) and dev manifests, plus the corresponding RBAC operation entries in pkg/corerp/setup/operations.go (mirroring the recipePacks pattern). Fixes Test_TerraformConfig_Redis, Test_BicepConfig_CRUD, and Test_TerraformConfig_BicepConfig_Combined.
The dependabot bump in #11784 wrote the JSON without a trailing newline, which fails prettier's format check on every PR that gets merged with main. Reformatted via 'make format-write'.
sylvainsf
force-pushed
the
add-terraform-bicep-config
branch
from
a99b027 to
371a010
Compare
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds new Radius.Core/terraformConfigs and Radius.Core/bicepConfigs resources, wires them into Radius.Core/environments, and threads their data into recipe execution and generated API artifacts.
Changes:
- Adds new TypeSpec/OpenAPI/resource-provider models, clients, converters, and manifests for
terraformConfigsandbicepConfigs. - Extends
Radius.Core/environmentsand the config loader/terraform executor to resolve and consume the new config resources. - Adds functional/unit tests and a new architecture document describing the feature.
Reviewed changes
Copilot reviewed 43 out of 46 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
typespec/Radius.Core/terraformConfigs.tsp |
Defines new Terraform config resource schema. |
typespec/Radius.Core/main.tsp |
Imports new Radius.Core resource specs. |
typespec/Radius.Core/environments.tsp |
Adds environment references to shared config resources. |
typespec/Radius.Core/bicepConfigs.tsp |
Defines new Bicep config resource schema. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/tfbicep-combined-test.bicep |
Combined functional scenario for both config resources. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/terraformconfig-redis-test.bicep |
Functional Terraform config scenario. |
test/functional-portable/dynamicrp/noncloud/resources/testdata/bicepconfig-test.bicep |
Functional Bicep config scenario. |
test/functional-portable/dynamicrp/noncloud/resources/terraformconfig_bicepconfig_test.go |
Functional tests for new resources and combined flow. |
swagger/specification/radius/resource-manager/Radius.Core/preview/2025-08-01-preview/openapi.json |
Regenerated OpenAPI for new resources and environment fields. |
pkg/rp/util/config.go |
Adds fetch helpers for new config resources. |
pkg/recipes/terraform/execute.go |
Wires Terraform CLI config generation into execution paths. |
pkg/recipes/terraform/execute_test.go |
Adds tests around Terraform CLI config/env handling. |
pkg/recipes/terraform/cliconfig.go |
Renders .terraformrc from provider installation settings. |
pkg/recipes/terraform/cliconfig_test.go |
Tests .terraformrc rendering/writing. |
pkg/recipes/configloader/environment.go |
Resolves environment config references into recipe config. |
pkg/recipes/configloader/environment_test.go |
Updates loader tests for new function signature. |
pkg/corerp/setup/setup.go |
Registers new Radius.Core resources. |
pkg/corerp/setup/setup_test.go |
Extends setup routing tests for new resources. |
pkg/corerp/setup/operations.go |
Adds RBAC operation entries for new resources. |
pkg/corerp/frontend/controller/environments/v20250801preview/createorupdateenvironment.go |
Validates referenced config IDs during environment PUT/PATCH. |
pkg/corerp/datamodel/terraformconfig.go |
Adds Terraform config datamodel. |
pkg/corerp/datamodel/recipe_types.go |
Extends shared recipe config with provider installation support. |
pkg/corerp/datamodel/environment_v20250801preview.go |
Adds new environment reference fields. |
pkg/corerp/datamodel/converter/terraformconfig_converter.go |
Adds Terraform config versioned/datamodel converter. |
pkg/corerp/datamodel/converter/bicepconfig_converter.go |
Adds Bicep config versioned/datamodel converter. |
pkg/corerp/datamodel/bicepconfig.go |
Adds Bicep config datamodel. |
pkg/corerp/api/v20250801preview/zz_generated_terraformconfigs_client.go |
Generated Terraform config client. |
pkg/corerp/api/v20250801preview/zz_generated_responses.go |
Generated response types for new clients. |
pkg/corerp/api/v20250801preview/zz_generated_options.go |
Generated option types for new clients. |
pkg/corerp/api/v20250801preview/zz_generated_models.go |
Generated API models for new resources and env fields. |
pkg/corerp/api/v20250801preview/zz_generated_models_serde.go |
Generated serde for new models. |
pkg/corerp/api/v20250801preview/zz_generated_constants.go |
Generated enum/constants for Bicep auth methods. |
pkg/corerp/api/v20250801preview/zz_generated_client_factory.go |
Exposes new generated clients from client factory. |
pkg/corerp/api/v20250801preview/zz_generated_bicepconfigs_client.go |
Generated Bicep config client. |
pkg/corerp/api/v20250801preview/terraformconfig_conversion.go |
Converts Terraform config API models to/from datamodel. |
pkg/corerp/api/v20250801preview/fake/zz_generated_terraformconfigs_server.go |
Generated fake Terraform config server. |
pkg/corerp/api/v20250801preview/fake/zz_generated_server_factory.go |
Wires new fake servers into factory. |
pkg/corerp/api/v20250801preview/fake/zz_generated_bicepconfigs_server.go |
Generated fake Bicep config server. |
pkg/corerp/api/v20250801preview/environment_conversion.go |
Converts new environment reference fields. |
pkg/corerp/api/v20250801preview/bicepconfig_conversion.go |
Converts Bicep config API models to/from datamodel. |
hack/bicep-types-radius/generated/radius/radius.core/2025-08-01-preview/types.json |
Regenerated Bicep type metadata. |
hack/bicep-types-radius/generated/index.json |
Adds new Bicep type index entries. |
docs/architecture/terraform-bicep-config.md |
New architecture doc for the feature. |
deploy/manifest/built-in-providers/self-hosted/radius_core.yaml |
Registers new resource types in self-hosted manifest. |
deploy/manifest/built-in-providers/dev/radius_core.yaml |
Registers new resource types in dev manifest. |
.devcontainer/devcontainer-lock.json |
Formatting-only lockfile change. |
Comment on lines
+107
to
+115
| if newResource.Properties.TerraformConfig != "" { | ||
| tfID, parseErr := resources.Parse(newResource.Properties.TerraformConfig) | ||
| if parseErr != nil { | ||
| return rest.NewBadRequestResponse(fmt.Sprintf("Invalid terraformConfig resource ID: %s", newResource.Properties.TerraformConfig)), nil | ||
| } | ||
| _, _, err = e.GetResource(ctx, tfID) | ||
| if err != nil { | ||
| return rest.NewBadRequestResponse(fmt.Sprintf("Referenced terraformConfig resource %q does not exist.", newResource.Properties.TerraformConfig)), nil | ||
| } |
Comment on lines
+117
to
+125
| if newResource.Properties.BicepConfig != "" { | ||
| bcID, parseErr := resources.Parse(newResource.Properties.BicepConfig) | ||
| if parseErr != nil { | ||
| return rest.NewBadRequestResponse(fmt.Sprintf("Invalid bicepConfig resource ID: %s", newResource.Properties.BicepConfig)), nil | ||
| } | ||
| _, _, err = e.GetResource(ctx, bcID) | ||
| if err != nil { | ||
| return rest.NewBadRequestResponse(fmt.Sprintf("Referenced bicepConfig resource %q does not exist.", newResource.Properties.BicepConfig)), nil | ||
| } |
Comment on lines
+248
to
+255
| // Map the new structured auth into the legacy BicepConfigProperties shape. | ||
| if bcProps.RegistryAuthentication != nil && bcProps.RegistryAuthentication.BasicAuthSecretId != "" { | ||
| config.RecipeConfig.Bicep = datamodel.BicepConfigProperties{ | ||
| Authentication: map[string]datamodel.RegistrySecretConfig{ | ||
| "default": { | ||
| Secret: bcProps.RegistryAuthentication.BasicAuthSecretId, | ||
| }, | ||
| }, |
Comment on lines
+207
to
+216
| // Map terraformrc.credentials into the legacy Authentication.Git.PAT shape | ||
| // that the existing Terraform driver expects for private module source auth. | ||
| if len(tfProps.Terraformrc.Credentials) > 0 { | ||
| config.RecipeConfig.Terraform.Authentication = datamodel.AuthConfig{ | ||
| Git: datamodel.GitAuthConfig{ | ||
| PAT: make(map[string]datamodel.SecretConfig), | ||
| }, | ||
| } | ||
| for host, cred := range tfProps.Terraformrc.Credentials { | ||
| config.RecipeConfig.Terraform.Authentication.Git.PAT[host] = datamodel.SecretConfig(cred) |
Comment on lines
+1
to
+3
| # Reusable Terraform and Bicep Config for Radius.Core/environments | ||
|
|
||
| ## Summary |
Comment on lines
+67
to
+68
| @doc("The ID of an Applications.Core/SecretStore resource containing username and password for BasicAuth. Required when authenticationMethod is 'BasicAuth'.") | ||
| basicAuthSecretId?: string; |
|
|
||
| Follows the schema from the [feature spec](https://github.com/radius-project/design-notes/pull/107). | ||
| The `terraformrc` property is a like-for-like representation of the Terraform CLI | ||
| configuration file (`.terraformrc`). The `env` property holds non-sensitive |
Member
There was a problem hiding this comment.
Let's make sure terraform.rc is supported as well.
Contributor
Author
There was a problem hiding this comment.
Same structure just a different filename?
| shared Terraform and Bicep drivers consume it through the same path the | ||
| legacy `Applications.Core` `recipeConfig` flow uses. | ||
| 5. When `terraformrc.providerInstallation` is set, the Terraform driver | ||
| renders a `.terraformrc` in the working directory and points Terraform at |
Member
There was a problem hiding this comment.
What if .terraformrc / terraform.rc already exists (provided by user)?
willdavsmith
reviewed
May 4, 2026
| properties: { | ||
| terraformrc: { | ||
| providerInstallation: { | ||
| networkMirror: { |
Contributor
There was a problem hiding this comment.
do we support manual binary upload or only network download
Contributor
Author
There was a problem hiding this comment.
Out of scope for this change, this is the network mirror for terraform providers not the binary.
willdavsmith
reviewed
May 4, 2026
| networkMirror: { | ||
| url: 'https://mirror.corp.example.com/terraform/providers' | ||
| include: ['*'] | ||
| exclude: ['hashicorp/azurerm'] |
Contributor
There was a problem hiding this comment.
What is the exclude key?
Contributor
Author
There was a problem hiding this comment.
"don't use local mirror for these providers"
willdavsmith
reviewed
May 4, 2026
| TF_REGISTRY_CLIENT_TIMEOUT: '15' | ||
| } | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
if a user doesn't provide this, do we have an auto-configured terraform install (like we do today)?
willdavsmith
reviewed
May 4, 2026
| registryAuthentication: { | ||
| authenticationMethod: 'BasicAuth' | ||
| basicAuthSecretId: acrSecret.id | ||
| } |
Contributor
There was a problem hiding this comment.
do we need to handle mirrored bicep download here?
6 tasks
Resolves several review comments on PR #11780. Schema changes: - BicepConfigProperties.registryAuthentication (single value, no host) is replaced with registryAuthentications: Record<...> keyed by registry hostname. The Bicep driver looks up credentials by the host parsed from the recipe template path; the previous shape gave the user no way to declare which registry the credentials applied to and the loader was hard-coding 'default' as the key, so credentials were silently dropped. - TerraformrcConfig.credentials documentation clarifies it is for HTTP Terraform CLI registry auth (rendered as native credentials "host" {} blocks with a 'token' key), not Git module-source PAT auth. Wiring changes: - Terraform driver now renders credentials "host" { token = ... } blocks in the generated .terraformrc, resolving the token value from the fetched-secret map at execution time. Drops the broken bridge that forwarded credentials into the legacy Git PAT path (which expects the 'pat' key, not 'token'). - terraform driver FindSecretIDs reports the new credentials secret stores with the 'token' key so the engine fetches them before the driver runs. - Loader uses the new host-keyed Bicep map to populate the legacy Authentication map (BasicAuth only; AzureWI/AwsIrsa stay schema-only follow-ups). Validation: - Environment controller's referenced-config-id checks now also verify the resource type matches the expected Radius.Core/{terraformConfigs, bicepConfigs}, instead of accepting any resource ID that happens to exist (Copilot review #1, #2). - New bicepconfigs.ValidateRequest hook (UpdateFilter) enforces that authenticationMethod=BasicAuth requires basicAuthSecretId, AzureWI requires both Wi ids, and AwsIrsa requires the role ARN. TypeSpec cannot express these conditional-required-field rules without a discriminated- union restructure (Copilot review #6). Tests: - New TestWriteTerraformCLIConfig_Credentials exercises native credentials rendering: success, deterministic ordering, missing secret reference, missing secret data, missing 'token' key, and embedded-quote escaping. - TestWriteTerraformCLIConfig_BothBlocks asserts provider_installation and credentials co-render in one .terraformrc. - New bicepconfigs.TestValidateRequest covers the controller-level conditional validation matrix. - bicepconfig-test.bicep and tfbicep-combined-test.bicep updated to the new schema shape with a real SecretStore reference (was previously a no-op BasicAuth without a secret, which would now be rejected). Docs: - docs/architecture/terraform-bicep-config.md updated for the new schema, the credentials/token rendering, and the architecture README index now links the page (Copilot review #5).
Radius functional test overviewClick here to see the test run details
Test Status |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds two new resources in the
Radius.Corenamespace —terraformConfigsandbicepConfigs— that environments reference by resource ID. Platform teams can now define private registry authentication, Terraform CLI configuration (provider_installation,credentials, env vars), and Bicep registry authentication once and share it across multiple environments.This implements a scoped subset of design-notes#107. Out of scope: Terraform binary lifecycle (
rad terraform install), backend state stores, installer pipeline. See docs/architecture/terraform-bicep-config.md for the full design.What's included
Radius.Core/terraformConfigsandRadius.Core/bicepConfigs(regenerated SDK).terraformConfig?andbicepConfig?properties onRadius.Core/environments, validated for existence at PUT time.pkg/corerp/setup/setup.go.pkg/recipes/configloader/environment.gothat resolves the new config resources at recipe execution time and populatesRecipeConfigconsumed by the shared driver.ProviderInstallationfield on the sharedTerraformConfigPropertiesdatamodel; the Terraform driver writes a.terraformrcto the working dir and points Terraform at it viaTF_CLI_CONFIG_FILEon both Deploy and Delete. Field is optional and only populated by the Radius.Core path — the legacyApplications.CorerecipeConfigflow is unchanged..terraformrcwriter, the env-var setter, and the Delete-path helper.Test_TerraformConfig_BicepConfig_Combined) deploying both configs in a single environment and running a Terraform recipe end-to-end.Known follow-ups (filed separately)
bicepConfig.registryAuthentication.{AzureWI,AwsIrsa}are accepted by the schema but not yet wired in the Bicep driver (onlyBasicAuthflows through today).referencedByon the config resources is exposed in the schema but not yet populated.Type of change
Fixes: #issue_number
Contributor checklist
Please verify that the PR meets the following requirements, where applicable: