Add shared/multi-tenant cluster note (oidc-reviewer collision)#2870
Add shared/multi-tenant cluster note (oidc-reviewer collision)#2870pnalubandhu-a11y wants to merge 5 commits into
Conversation
On a shared cluster, two W&B installs with the same release name collide on the cluster-scoped <release>-oidc-reviewer ClusterRoleBinding. Add a short note to use a unique release name (plus internalJWTMap and namespaceIsolation).
|
Narrowing the ping. cc @wandb/delivery-tooling-team 🙂 Context: docs PR in the self-managed OpenShift series. Adds a shared/multi-tenant cluster note — two installs sharing a cluster collide on the cluster-scoped |
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
📚 Mintlify Preview Links📝 Changed (1 total)📄 Pages (1)
🤖 Generated automatically when Mintlify deployment succeeds |
🔗 Link Checker Results✅ All links are valid! No broken links were detected. Checked against: https://wb-21fd5541-sa-openshift-shared-cluster-oidc-reviewer.mintlify.app |
📋 Validation of technical assertionsEvery claim below was checked against the truth on the ground using Glean (Slack, Jira, Zendesk, GitHub) and Sourcegraph (the implementing Helm chart plus the vendored Helm ownership-validation code), then put through an adversarial verification pass.
Open items — recommended next steps
Scope notes
🤖 Generated by the CoreWeave Docs Team's Validator ( |
johndmulhausen
left a comment
There was a problem hiding this comment.
Automated documentation fact-check — see the summary comment for the full validation table.
|
|
||
| - Use a unique release/CR name for each install (for example `wandb-team-a`, not `wandb`). | ||
| - Set `app.internalJWTMap: []` unless you need internal JWT mapping. This skips rendering the `oidc-reviewer` ClusterRoleBinding. | ||
| - Install the operator with `namespaceIsolation.enabled=true`. |
There was a problem hiding this comment.
❌ Refuted — the source says this is wrong. namespaceIsolation.enabled=true does not avoid the oidc-reviewer collision; it's the cause of cluster-scoped RBAC failures. That key only gates the operator's --isolation-namespaces arg and the operator's namespaced Role/RoleBinding (in the separate operator chart) — it never touches the cluster-scoped oidc-reviewer ClusterRoleBinding. With isolation on, the operator's service account is scoped to one namespace and can no longer manage that cluster-scoped object (customer report: "namespace isolation on so it can't access the cluster based roles by design").
Removing this bullet is the safe fix — the two working remedies (unique release name, app.internalJWTMap: []) are already listed above.
| - Install the operator with `namespaceIsolation.enabled=true`. |
Source says: namespace isolation gates only the operator's namespaced RBAC, unrelated to the cluster-scoped oidc-reviewer binding · Source: helm-charts:operator/deployment.yaml#L37-L38, operator/values.yaml#L248-L250
🤖 CoreWeave Docs Team's Validator — refuted
Review feedback: namespaceIsolation doesn't avoid the oidc-reviewer collision — with it on, the operator's namespaced SA can't manage the cluster-scoped binding at all. The working remedies (unique release name, app.internalJWTMap: []) stay.
|
Thanks @johndmulhausen / validator — good catch. Removed the |
mdlinville
left a comment
There was a problem hiding this comment.
Removing my approval until the technical assertion flagged by John is resolved one way or the other. Thanks. :)
|
@mdlinville thanks — both of John's flagged items are now resolved: #5 (the blocking one — #4 (advisory — Net: the ❌ refuted assertion is gone and the remaining item is advisory + now caveated in-text. Good to re-review when you have a moment — thanks! 🙏 |
|
@mdlinville when you get a moment — both items you and John flagged are now resolved:
CI is green. Would you mind re-reviewing to clear the changes-requested? Thanks! 🙏 |
What
Adds a "Shared or multi-tenant clusters" note to the self-managed operator page.
Why
The chart creates cluster-scoped objects named after the release (e.g.
<release>-oidc-reviewer). If two installs on the same cluster use the same release name, the second one fails with a Helm ownership error and it isn't obvious why. The docs currently assume a dedicated cluster.Change
Short note under "Environment-specific considerations": use a unique release name per install, set
app.internalJWTMap: []when not needed, and install the operator withnamespaceIsolation.enabled=true.Testing
Reproduced the collision on a shared OCP 4.21 cluster — a second release named
wandbfailed with:ClusterRoleBinding "wandb-oidc-reviewer" ... cannot be imported ... release-namespace must equal ...A unique release name avoids it.