Add env/file interpolation for config.toml and APIP_CP_ env prefix#2616
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughPlatform API configuration now uses Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Validation ResultsDependency name: github.com/knadh/koanf/providers/confmap |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
distribution/all-in-one/docker-compose.yaml (1)
54-77: 🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick winAdd the Platform API secrets to
platform-api
distribution/all-in-one/docker-compose.yamldoesn’t passAPIP_CP_ENCRYPTION_KEY/APIP_CP_AUTH_JWT_SECRET_KEY, and it doesn’t mount aconfig.tomleither, so the container falls back to defaults and startup validation will reject the missing keys. Add those vars here or inject them viaenv_file, matching the test compose files.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@distribution/all-in-one/docker-compose.yaml` around lines 54 - 77, Add the required Platform API secrets to the platform-api service configuration, using APIP_CP_ENCRYPTION_KEY and APIP_CP_AUTH_JWT_SECRET_KEY with the same injection approach and values or env_file pattern used by the test compose files. If those values are defined through config.toml there, mount the corresponding config.toml into platform-api as well so startup validation receives both keys.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/ai-workspace/features/secrets-management.md`:
- Around line 384-389: Update the secrets setup instructions around the `.env`
example to separate shell key generation from `.env` contents: show `openssl
rand -hex 32` commands whose outputs users paste as `AUTH_JWT_SECRET_KEY` and
`ENCRYPTION_KEY`, rather than documenting command substitution inside the env
file.
In `@portals/ai-workspace/.env.example`:
- Around line 8-11: Update the platform-api service in docker-compose.yaml to
pass AUTH_JWT_SECRET_KEY and ENCRYPTION_KEY into the container via env_file or
explicit environment entries, so the mounted config-platform-api.toml can
resolve both env tokens; do not rely on the host .env file alone.
---
Outside diff comments:
In `@distribution/all-in-one/docker-compose.yaml`:
- Around line 54-77: Add the required Platform API secrets to the platform-api
service configuration, using APIP_CP_ENCRYPTION_KEY and
APIP_CP_AUTH_JWT_SECRET_KEY with the same injection approach and values or
env_file pattern used by the test compose files. If those values are defined
through config.toml there, mount the corresponding config.toml into platform-api
as well so startup validation receives both keys.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 7115bf82-4ec6-4206-ac10-9de0e95a3b55
⛔ Files ignored due to path filters (1)
platform-api/go.sumis excluded by!**/*.sum
📒 Files selected for processing (25)
distribution/all-in-one/.env.exampledistribution/all-in-one/docker-compose.yamldocs/ai-workspace/configuration.mddocs/ai-workspace/features/secrets-management.mdplatform-api/README.mdplatform-api/config/config.goplatform-api/config/config.tomlplatform-api/config/config_test.goplatform-api/go.modplatform-api/internal/server/server.goportals/ai-workspace/.env.exampleportals/ai-workspace/configs/config-platform-api-template.tomlportals/ai-workspace/configs/config-platform-api.tomlportals/ai-workspace/docker-compose.yamlportals/developer-portal/.env.exampleportals/developer-portal/configs/config-platform-api.toml.exampleportals/developer-portal/distribution/.env.exampleportals/developer-portal/distribution/docker-compose.yamlportals/developer-portal/docker-compose.platform-api.yamlportals/developer-portal/docker-compose.yamlportals/developer-portal/it/docker-compose.test.postgres.yamlportals/developer-portal/it/docker-compose.test.yamltests/integration-e2e/docker-compose.sqlite.yamltests/integration-e2e/docker-compose.sqlserver.yamltests/integration-e2e/docker-compose.yaml
💤 Files with no reviewable changes (3)
- portals/developer-portal/docker-compose.platform-api.yaml
- portals/developer-portal/distribution/docker-compose.yaml
- portals/developer-portal/docker-compose.yaml
Dependency Validation ResultsDependency name: github.com/knadh/koanf/providers/confmap |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/ai-workspace/configuration.md`:
- Around line 100-101: Update the inline code formatting in the configuration
instructions so the `docker compose --env-file keys.env up` command has a
closing backtick before the alternative sentence. Keep the `{{ file }}` inline
code span separate and preserve the surrounding wording.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0bb7ab30-345a-4971-af12-cb01ffc92529
📒 Files selected for processing (12)
distribution/all-in-one/docker-compose.yamldocs/ai-workspace/configuration.mddocs/ai-workspace/features/secrets-management.mdplatform-api/README.mdplatform-api/config/config.tomlportals/ai-workspace/configs/config-platform-api-template.tomlportals/ai-workspace/configs/config-platform-api.tomlportals/ai-workspace/docker-compose.yamlportals/developer-portal/configs/config-platform-api.toml.exampleportals/developer-portal/distribution/docker-compose.yamlportals/developer-portal/docker-compose.platform-api.yamlportals/developer-portal/docker-compose.yaml
🚧 Files skipped from review as they are similar to previous changes (6)
- portals/ai-workspace/configs/config-platform-api.toml
- portals/developer-portal/docker-compose.yaml
- portals/ai-workspace/configs/config-platform-api-template.toml
- distribution/all-in-one/docker-compose.yaml
- platform-api/README.md
- portals/ai-workspace/docker-compose.yaml
5747829 to
7a57fd3
Compare
Port the shared config-interpolation mechanism drafted for the gateway (PR wso2#2596) to the platform-api config loader. Any config field may now pull its value from an environment variable ({{ env "NAME" }}) or an allowlisted file ({{ file "/path" }}), resolved after the env+file merge and before unmarshal. String leaves without a "{{" token pass through unchanged, so existing token-free configs are unaffected. - Vendor common/configinterpolate from wso2#2596 (branch conf-template). This is a temporary copy to be reconciled when wso2#2596 merges to main. - Wire an interpolate() step into config.LoadConfig, mirroring the gateway-controller. File reads are restricted to a per-component allowlist (/etc/platform-api, /secrets/platform-api), overridable via the shared APIP_CONFIG_FILE_SOURCE_ALLOWLIST env var. Fails closed on a missing required env var or a disallowed/missing/oversize file; resolved values are never logged (only reference counts at info level). - Add github.com/knadh/koanf/providers/confmap for the reload step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dependency Validation ResultsDependency name: github.com/knadh/koanf/providers/confmap |
1 similar comment
Dependency Validation ResultsDependency name: github.com/knadh/koanf/providers/confmap |
|
@thivindu let's fix the minor comments in a separate PR |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
platform-api/internal/integration/harness_test.go (1)
51-63: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse a wrapped function to ensure cleanup with
defer.If
m.Run()or file writing panics,os.Remove(f.Name())will not execute. Using an anonymous function allowsdeferto run reliably beforeos.Exit()is called, which aligns with best practices for temporary file cleanup.♻️ Proposed refactor
- f, err := os.CreateTemp("", "it-config-*.toml") - if err != nil { - panic(fmt.Sprintf("integration harness: create temp config: %v", err)) - } - if _, err := fmt.Fprintf(f, "encryption_key = %q\n\n[auth.jwt]\nenabled = true\nsecret_key = %q\n", testKey, testKey); err != nil { - panic(fmt.Sprintf("integration harness: write temp config: %v", err)) - } - _ = f.Close() - config.SetConfigPath(f.Name()) - - code := m.Run() - _ = os.Remove(f.Name()) - os.Exit(code) + code := func() int { + f, err := os.CreateTemp("", "it-config-*.toml") + if err != nil { + panic(fmt.Sprintf("integration harness: create temp config: %v", err)) + } + defer os.Remove(f.Name()) + + if _, err := fmt.Fprintf(f, "encryption_key = %q\n\n[auth.jwt]\nenabled = true\nsecret_key = %q\n", testKey, testKey); err != nil { + f.Close() + panic(fmt.Sprintf("integration harness: write temp config: %v", err)) + } + _ = f.Close() + config.SetConfigPath(f.Name()) + + return m.Run() + }() + os.Exit(code)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@platform-api/internal/integration/harness_test.go` around lines 51 - 63, Wrap the temporary-config setup, test execution, and cleanup in an anonymous function so cleanup is registered with defer and runs when file writing or m.Run panics. Update the flow around m.Run and os.Remove, preserve the returned exit code, and call os.Exit only after the function completes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@platform-api/internal/integration/harness_test.go`:
- Around line 51-63: Wrap the temporary-config setup, test execution, and
cleanup in an anonymous function so cleanup is registered with defer and runs
when file writing or m.Run panics. Update the flow around m.Run and os.Remove,
preserve the returned exit code, and call os.Exit only after the function
completes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f7e46f96-1430-4339-b549-8c7ea7003ba3
⛔ Files ignored due to path filters (1)
platform-api/go.sumis excluded by!**/*.sum
📒 Files selected for processing (34)
.github/workflows/ai-workspace-pr-check.ymldistribution/all-in-one/docker-compose.yamldocs/ai-workspace/configuration.mddocs/ai-workspace/features/secrets-management.mdgateway/examples/llm-provider.yamlplatform-api/README.mdplatform-api/config/config.goplatform-api/config/config.tomlplatform-api/config/config_test.goplatform-api/go.modplatform-api/internal/integration/harness_test.goplatform-api/internal/repository/api_deployments_test.goplatform-api/internal/repository/subscription_repository.goplatform-api/internal/server/server.goplatform-api/plugins/eventgateway/service/websub_api_hmac_secret.goportals/ai-workspace/README.mdportals/ai-workspace/configs/config-platform-api-template.tomlportals/ai-workspace/configs/config-platform-api.tomlportals/ai-workspace/docker-compose.yamlportals/ai-workspace/production/README.mdportals/developer-portal/README.mdportals/developer-portal/configs/config-platform-api.toml.exampleportals/developer-portal/distribution/docker-compose.yamlportals/developer-portal/docker-compose.platform-api.yamlportals/developer-portal/docker-compose.yamlportals/developer-portal/docs/administer/manage-organizations.mdportals/developer-portal/it/configs/config-platform-api-it.tomlportals/developer-portal/it/docker-compose.test.postgres.yamlportals/developer-portal/it/docker-compose.test.yamltests/integration-e2e/README.mdtests/integration-e2e/docker-compose.sqlite.yamltests/integration-e2e/docker-compose.sqlserver.yamltests/integration-e2e/docker-compose.yamltests/integration-e2e/platform-api-config.toml
🚧 Files skipped from review as they are similar to previous changes (24)
- .github/workflows/ai-workspace-pr-check.yml
- portals/ai-workspace/production/README.md
- gateway/examples/llm-provider.yaml
- platform-api/plugins/eventgateway/service/websub_api_hmac_secret.go
- tests/integration-e2e/docker-compose.sqlite.yaml
- platform-api/config/config.toml
- tests/integration-e2e/README.md
- portals/developer-portal/it/docker-compose.test.yaml
- portals/developer-portal/it/docker-compose.test.postgres.yaml
- portals/developer-portal/docs/administer/manage-organizations.md
- docs/ai-workspace/configuration.md
- platform-api/internal/repository/subscription_repository.go
- portals/developer-portal/README.md
- tests/integration-e2e/docker-compose.sqlserver.yaml
- distribution/all-in-one/docker-compose.yaml
- portals/developer-portal/configs/config-platform-api.toml.example
- portals/ai-workspace/configs/config-platform-api-template.toml
- portals/developer-portal/docker-compose.platform-api.yaml
- portals/developer-portal/distribution/docker-compose.yaml
- platform-api/config/config_test.go
- portals/ai-workspace/configs/config-platform-api.toml
- platform-api/internal/repository/api_deployments_test.go
- portals/ai-workspace/docker-compose.yaml
- platform-api/config/config.go
57c8101
7b4bd7c to
57c8101
Compare
Dependency Validation ResultsDependency name: github.com/knadh/koanf/providers/confmap |
Summary
Brings the config-injection model agreed in discussion #1983 to the platform-api, and tightens how secrets are provided cross the sample deployments. Three logical changes:
variable (
{{ env "NAME" }}) or an allowlisted file ({{ file "/path" }}), resolved atstartup, fail-closed.
APIP_CP_env-var prefix - config-override environment variables are now namespacedAPIP_CP_*, matching the gateway'sAPIP_GW_and the Developer Portal'sAPIP_DP_.them via
{{ env }}/{{ file }}tokens with values supplied from a.envfile or a mountedsecret file, and docs are updated to teach the preferred pattern.
Related to
{{ env }}/{{ file }}interpolation in gatewayconfig.toml#25911. Config interpolation
common/configinterpolatepackage (from feat(config): add env/file interpolation for config.toml #2596) inplatform-api/config/config.go, wired as aninterpolate(k)step after the env+file merge and before unmarshal, mirroring the gateway-controller./etc/platform-api,/secrets/platform-api), overridable via the sharedAPIP_CONFIG_FILE_SOURCE_ALLOWLIST.{{ … }}token passes through untouched.github.com/knadh/koanf/providers/confmapfor the post-interpolation reload.2.
APIP_CP_environment-variable prefixConfig-override env vars must now be prefixed with
APIP_CP_. The prefix is stripped and theremainder mapped to a config key:
LOG_LEVEL=debugAPIP_CP_LOG_LEVEL=debugDATABASE_HOST=…APIP_CP_DATABASE_HOST=…AUTH_JWT_SECRET_KEY=…(direct override)APIP_CP_AUTH_JWT_SECRET_KEY=…Two variables are intentionally not prefixed:
APIP_DEMO_MODE— a standalone runtime flag (read directly, not via the config loader).APIP_CONFIG_FILE_SOURCE_ALLOWLIST— the shared interpolation allowlist.Note on the two naming contexts: the
APIP_CP_prefix applies only to the direct config-override path. The{{ env "NAME" }}interpolation tokens read bare names viaos.LookupEnv, so{{ env "ENCRYPTION_KEY" }}readsENCRYPTION_KEY, while the direct override for the same key isAPIP_CP_ENCRYPTION_KEY.Action required for operators: rename any platform-api config env vars to
APIP_CP_*. All first-party compose/CI stacks in this repo have been updated.3. Secret handling
Secrets should never be hardcoded - not as raw values in
config.toml, nor as literals in a compose file. Instead they are referenced in the config with a token and resolved at startup:Files changed
Code
Docs
Sample deployments
Test/CI stacks (prefix rename only)
Related