Adding design for automatically registering core RRTs#11610
Conversation
kachawla
force-pushed
the
rrt-auto-registration
branch
from
f165ae2 to
4177f2d
Compare
There was a problem hiding this comment.
Pull request overview
Adds a design note proposing automated default registration of Radius resource types by embedding selected manifests from resource-types-contrib into the Radius binary and registering them during UCP initialization.
Changes:
- Introduces a design for using
resource-types-contribas a Go module dependency and embedding default manifests viago:embed. - Proposes a centralized
defaults.yaml+go generateworkflow to control which manifests ship as defaults. - Describes initializer/runtime behavior, error handling, and a test/CI validation plan for generated embed lists.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11610 +/- ##
=======================================
Coverage 51.38% 51.38%
=======================================
Files 699 699
Lines 44114 44114
=======================================
Hits 22666 22666
Misses 19279 19279
Partials 2169 2169 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
kachawla
force-pushed
the
rrt-auto-registration
branch
2 times, most recently
from
06da8d4 to
369e713
Compare
kachawla
force-pushed
the
rrt-auto-registration
branch
from
369e713 to
9262bfb
Compare
|
|
||
| ### High Level Design | ||
|
|
||
| The design introduces `resource-types-contrib` as a Go module dependency of `radius`. Resource type manifests are embedded into the Radius binary using Go's `embed.FS` mechanism. A central `defaults.yaml` file in `resource-types-contrib` lists which manifests should be embedded and registered by default. |
There was a problem hiding this comment.
Naming nit: default-types.yaml might be more appropriate if the design is to only store default type definitions in the file.
However, if additional default items (e.g. default recipe packs @nithyatsu 👀) may be stored in the same config file, then defaults.yaml makes sense.
There was a problem hiding this comment.
I think we would include recipes corresponding to each type in this file as well. What do you think @nithyatsu?
There was a problem hiding this comment.
yes, it makes sense to store bicep recipe corresponding to each type as well to help with creation of correct default recipe pack. Would the memory increase that comes along be ok?
There was a problem hiding this comment.
Would the memory increase that comes along be ok?
I wasn't thinking of changing how recipes stored, but mainly when they are published. Since we want to make sure any time a default type is added, corresponding recipe is published and default recipe pack is updated to include it, we should point the workflow that does this today to look at defaults.yaml.
brooke-hamilton
left a comment
brooke-hamilton
left a comment
There was a problem hiding this comment.
🚀 I really like how this proposal sets up resource-types-contrib as a clear dependency of Radius via a Golang module dependecy.
|
|
||
| ### 3. Tagged releases and automated dependency updates for `resource-types-contrib` | ||
|
|
||
| `resource-types-contrib` does not have a formal release or tagging process today. Without tagged releases, Radius depends on Go pseudo-versions (e.g., `v0.0.0-20260408153021-abc123def456`), and dependency updates require a maintainer to manually run `go get -u`. This limits automation and makes it harder to track what changed between versions. |
There was a problem hiding this comment.
+1 on creating GitHub releases for the generated go code. Having releases would allow for idiomatic consumption of the releases in the Radius repo, and trigger dependabot updates when a new version is available.
If there is a need for more granular releases for specific resource types, then each resource type could be separated into its own go module and released separately. I don't think this would be required, but it's an option if we find ourselves updating the individual types in different cycles.
nellshamrell
left a comment
nellshamrell
left a comment
There was a problem hiding this comment.
Excellent work, @kachawla!
|
|
||
| Today, resource type manifests for default registration in Radius are manually duplicated from the `resource-types-contrib` repository into the `radius` repository under `deploy/manifest/built-in-providers/`. This creates a maintenance burden - when a resource type schema is updated in `resource-types-contrib`, the corresponding file in `radius` must be manually updated, leading to schema drift, stale definitions, and duplicated effort. | ||
|
|
||
| This design introduces a mechanism to automatically embed resource type manifests from `resource-types-contrib` as a Go module dependency of `radius`. A central configuration file (`defaults.yaml`) in `resource-types-contrib` declares which resource types should be default-registered. At build time, only those manifests are embedded into the Radius binary via `go:embed`. At startup, the UCP initializer reads the embedded manifests and registers them alongside any existing directory-based manifests. |
There was a problem hiding this comment.
I like this - it lets us define recipes that require extra scrutiny when reviewing (we can also use CI to flag when recipes on the defaults.yml are changed and require additional review). This makes the threat of malicious contributions easier to manage (malicious contributions are a risk in any OSS project).
There was a problem hiding this comment.
that's a good point to use CI for flagging changes. I'll add that. Thanks for reviewing
| - **Runtime fetching of manifests**: Manifests are embedded at build time, not downloaded at runtime. This avoids network dependencies during startup. | ||
| - **Migrating non-dynamic-rp providers**: Resource types served by `applications-rp` or the deployment engine (e.g., `Applications.Core`, `Microsoft.Resources`) require explicit `location` addresses and remain as directory-based manifests in `radius`. Migrating them is out of scope. | ||
| - **Recipe registration**: This design covers resource type schema registration only, not recipe registration or recipe pack management. | ||
| - **Release process for `resource-types-contrib`**: This design assumes a Radius maintainer manually bumps the `resource-types-contrib` dependency in `go.mod` to pick up changes. Establishing a formal release/tagging process for `resource-types-contrib` is out of scope. |
There was a problem hiding this comment.
This was going to be my next question - whether there is a release/tagging process which would prevent a bug that was accidentally merged into a default resource type from manifesting in Radius. I agree it is out of scope for this proposal, but would be good to consider in the future.
| - Networking/loadBalancers/loadBalancers.yaml | ||
| ``` | ||
| 3. They run `go generate` and commit `defaults.yaml` along with the auto-generated `manifests_gen.go` (which contains the `//go:embed` directives that tell the Go compiler which files to embed in the binary). | ||
| 4. A Radius maintainer manually bumps the dependency by running `go get -u github.com/radius-project/resource-types-contrib` in the `radius` repository and merging the resulting `go.mod` change. Since `resource-types-contrib` does not have tagged releases today, Go resolves a pseudo-version based on the latest commit (e.g., `v0.0.0-20260408153021-abc123def456`). |
There was a problem hiding this comment.
ah - excellent use of the commit hash! This should not be used as a long-term solution as I believe git by default uses SHA-1 for its hashes, which is not very strong and can be broken. More to consider when we think about tagged releases in the future, but not a blocker to this proposal.
| - **Discoverability**: A single file shows all defaults at a glance. | ||
| - **Reviewability**: PR diffs for `defaults.yaml` clearly show what's being added or removed. | ||
| - **No parser coupling**: `resource-types-contrib` metadata stays out of the Radius manifest parser. | ||
| - **Extensible**: Works for any directory structure; new top-level directories (e.g., `Networking/`) work without changing Go code. |
|
|
||
| ## Security | ||
|
|
||
| No changes to the security model. The embedded manifests are static YAML files compiled into the binary at build time, so there is no new attack surface for injection or tampering beyond what exists for any compiled-in resource. The `defaults.yaml` file is validated at startup, and invalid entries cause a clear startup failure. |
There was a problem hiding this comment.
Agreed - to me this adds no more significant risks than manually copying and pasting resource types does. Like all OSS projects, we need to consider ways to do more extensive security filtering/testing of pull requests, but this design does not add additional risk.
|
|
||
| 1. **`go generate` enforcement**: Should `resource-types-contrib` CI block merges if `manifests_gen.go` is out of date, or should CI auto-regenerate and commit? | ||
|
|
||
| - **Option A: CI blocks merges (proposed).** CI runs `go generate` and `git diff --exit-code manifests_gen.go`. If the file is stale, the PR fails. Contributors must run `go generate` locally before pushing. This keeps generated files explicitly reviewed in PRs and avoids hidden auto-commits. |
kachawla
force-pushed
the
rrt-auto-registration
branch
2 times, most recently
from
ce6d3d4 to
133e092
Compare
Radius functional test overviewClick here to see the test run details
Test Status⌛ Building Radius and pushing container images for functional tests... |
kachawla
force-pushed
the
rrt-auto-registration
branch
2 times, most recently
from
8044493 to
c834f40
Compare
| 2. **`go.mod` bump requires PR review in `radius`.** Changes in `resource-types-contrib` only reach Radius when a maintainer runs `go get -u` and merges the resulting `go.mod`/`go.sum` change. This is a second review gate. | ||
|
|
||
| 3. **Manifest parsing and schema validation.** Manifests are parsed using a strict YAML decoder that rejects unknown top-level fields, duplicate keys, and any data that does not conform to the expected `ResourceProvider` structure. Schemas within each manifest are further validated against OpenAPI format; malformed or structurally invalid schemas are rejected at startup. There is no risk of code execution through YAML parsing, as Go YAML parsers do not support executable YAML tags. | ||
|
|
There was a problem hiding this comment.
It would be to describe what happens when ,say, schema validation flags issues for one resource-type and how we can proceed with a release when that happens.
There was a problem hiding this comment.
Startup would fail. Added a note here on how to handle it https://github.com/kachawla/radius/blob/c4446f7a45072d295d3c334d36692a6d578d7ef3/eng/design-notes/extensibility/2026-04-automated-default-resource-type-registration.md#error-handling
| 3. **Manifest parsing and schema validation.** Manifests are parsed using a strict YAML decoder that rejects unknown top-level fields, duplicate keys, and any data that does not conform to the expected `ResourceProvider` structure. Schemas within each manifest are further validated against OpenAPI format; malformed or structurally invalid schemas are rejected at startup. There is no risk of code execution through YAML parsing, as Go YAML parsers do not support executable YAML tags. | ||
|
|
||
| 4. **Schema runtime behavior.** The `schema` field within each API version accepts arbitrary JSON Schema content (including `additionalProperties: true`), so its contents are not structurally restricted beyond OpenAPI validity. After registration, dynamic-rp reads stored schemas at runtime for request validation and sensitive field identification (encryption). The schema is never passed to Terraform or Bicep recipes, and users always provide resource property values explicitly, so a crafted schema cannot inject values into recipe execution. The residual risks are limited to weakened request validation (overly permissive properties), unnecessary encryption overhead (incorrectly marking fields as sensitive), or performance degradation (very large or deeply nested schemas). These risks are mitigated by the requirement for manifest changes to go through code review, where reviewers can inspect the schema content. | ||
|
|
There was a problem hiding this comment.
I agree reviews are essential to mitigate risk. We had initially disallowed additionalProperties: true for security reasons, but ACI had a use case which required us to support it.
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
kachawla
force-pushed
the
rrt-auto-registration
branch
from
c834f40 to
735706a
Compare
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
kachawla
force-pushed
the
rrt-auto-registration
branch
from
c4446f7 to
63e928e
Compare
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
- Add doc.go and pkg/resourcetypescontrib/import.go to implementation details
- Remove postgreSqlDatabases from defaults.yaml examples
- Add radius_data.yaml to removed files list
- Update file paths to show {dev,self-hosted}/ subdirectories
- Replace go get -u with go get MODULE@latest
- Fix Security section to describe copy-based approach (not embedded/compiled)
- Update sync-resource-types example to show stale file cleanup
- Update Test plan with actual test name and mark upgrade test as follow-up
- Update Development plan with all implementation details
Signed-off-by: Karishma Chawla <kachawla@microsoft.com>
Description
Certain resource types defined in the resource-types-contrib repo are expected to be registered with Radius by default for every installation. The way we enable it today is by manually copying over the manifests into Radius repo and maintaining duplicate files. This design automates propagating the changes to the Radius repo.
Type of change
Fixes: #11108
Contributor checklist
Please verify that the PR meets the following requirements, where applicable: