Skip to content

Fix OpenShift securityContext example (podSecurityContext + fsGroup)#2869

Open
pnalubandhu-a11y wants to merge 3 commits into
mainfrom
sa/openshift-podsecuritycontext-fsgroup
Open

Fix OpenShift securityContext example (podSecurityContext + fsGroup)#2869
pnalubandhu-a11y wants to merge 3 commits into
mainfrom
sa/openshift-podsecuritycontext-fsgroup

Conversation

@pnalubandhu-a11y

Copy link
Copy Markdown

What

Fixes the OpenShift security context example on the self-managed operator page.

Why

If you copy the example as-is, the pods don't start on OpenShift:

  1. The settings are under pod.securityContext, but the chart reads podSecurityContext for the app/api components, so runAsUser/fsGroup are ignored and the pods stay at the default UID 999.
  2. Setting only the UID isn't enough either — the leftover fsGroup: 0 is rejected by restricted-v2:
    fsGroup: Invalid value: [0]: 0 is not an allowed group

Change

  • Use podSecurityContext instead of pod.securityContext
  • Set both runAsUser and fsGroup, in the project's UID range
  • Add the oc get namespace ... command so people know which value to use
  • Add a short note about the fsGroup requirement

Testing

Tried it on OCP 4.21. With the old pod.securityContext the app and api pods stayed at UID 999; with podSecurityContext they came up in the namespace range and ran fine on restricted-v2 — no custom SCC needed.

The example puts the settings under `pod.securityContext`, but the chart
reads `podSecurityContext` for the app/api pods, so runAsUser/fsGroup get
ignored and the pods stay at UID 999 — which restricted-v2 rejects.

Switched to `podSecurityContext`, set fsGroup too (not just the UID), and
added how to find your project's UID range. Tested on OCP 4.21.
@pnalubandhu-a11y pnalubandhu-a11y requested a review from a team as a code owner July 6, 2026 20:51
@mintlify

mintlify Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
wandb 🟢 Ready View Preview Jul 6, 2026, 9:23 PM

@pnalubandhu-a11y

pnalubandhu-a11y commented Jul 6, 2026

Copy link
Copy Markdown
Author

Narrowing the ping. cc @wandb/delivery-tooling-team 🙂

Context: one of four small docs PRs fixing the W&B self-managed on OpenShift install guide (from a real OCP 4.21 deploy). This one fixes the securityContext example — use the top-level podSecurityContext key (the nested pod.securityContext is ignored for runAsUser/fsGroup) and set both runAsUser and fsGroup in the project UID range. Companions: #2870, #2872, #2874. @wandb/docs-team is the reviewer.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

📚 Mintlify Preview Links

🔗 View Full Preview

📝 Changed (1 total)

📄 Pages (1)

File Preview
platform/hosting/self-managed/operator.mdx Operator

🤖 Generated automatically when Mintlify deployment succeeds
📍 Deployment: e942926 at 2026-07-08 22:45:14 UTC

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🔗 Link Checker Results

All links are valid!

No broken links were detected.

Checked against: https://wb-21fd5541-sa-openshift-podsecuritycontext-fsgroup.mintlify.app

@johndmulhausen

Copy link
Copy Markdown
Contributor

📋 Validation of technical assertions

Every claim below was checked against the truth on the ground using Glean (Slack, Jira, Confluence, GitHub) and Sourcegraph (the implementing Helm chart code), then put through an adversarial verification pass.
7 assertions · ✅ 6 confirmed · ✅ 1 confirmed (qualified) · 🟡 0 partial · ⚠️ 0 reframed · ⛔ 0 could not validate · ❌ 0 refuted.
Nothing was refuted. The central fix — using the top-level podSecurityContext key instead of pod.securityContext, and setting both runAsUser and fsGroup inside the project's UID range — is corroborated directly by the chart source and by the internal Slack thread where a W&B engineer hit and fixed this exact error on OpenShift. One claim is qualified only because the doc slightly simplifies the error-string value representation.

# Assertion in the docs Verdict Evidence (truth on the ground) Source
1 By default, W&B containers run at $UID 999 · operator.mdx:66,88 ✅ Confirmed Chart default podSecurityContext.runAsUser: 999. Internal deploy error also shows runAsUser: Invalid value: 999 under restricted-v2. helm-charts:wandb-base/values.yaml#L101-L104; #platform-delivery-tooling-team
2 Set runAsUser and fsGroup within the project's assigned UID range, with $GID of 0 · operator.mdx:66 ✅ Confirmed Engineer's working OpenShift config: runAsUser: 1000730000, fsGroup: 1000730000, runAsGroup: 0 — all inside the project range [1000730000, 1000739999]. #platform-delivery-tooling-team
3 Find the range via oc get namespace <ns> -o jsonpath='{.metadata.annotations.openshift\.io/sa\.scc\.uid-range}', e.g. 1000810000/10000 · operator.mdx:75-76 ✅ Confirmed The annotation is the official OpenShift API constant openshift.io/sa.scc.uid-range; its value is a {start}/{length} block. Real range seen internally: 1000730000/10000. openshift/api security/v1/consts.go#L4-L6 (vendored)
4 These components read the top-level podSecurityContext key; the nested pod.securityContext is NOT applied to runAsUser/fsGroup · operator.mdx:87-90 ✅ Confirmed The wandb-base pod template renders pod securityContext from merge (default dict .podData.podSecurityContext) $.root.Values.podSecurityContext — it reads podSecurityContext. The chart README: "Pod Security Context … Defined in podSecurityContext." Engineer confirmed the component "didn't seem to pick the setup" until keyed correctly through wandb-base. helm-charts:wandb-base/templates/_pods.tpl#L59-L60; helm-charts:wandb-base/README.md#L147-L165
5 Without the override, pods stay at UID 999 and are rejected by the restricted-v2 SCC · operator.mdx:88-89 ✅ Confirmed Internal deploy error, verbatim: provider restricted-v2: .containers[0].runAsUser: Invalid value: 999: must be in the ranges: [1000730000, 1000739999]. #platform-delivery-tooling-team
6 Leaving fsGroup: 0 is rejected by restricted-v2 with fsGroup: Invalid value: [0]: 0 is not an allowed group · operator.mdx:109 ✅ Confirmed (qualified) Internal error, verbatim: provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{0}: 0 is not an allowed group. Qualifier: the message text is exact, but the real value representation is []int64{0}; the doc simplifies it to [0]. Harmless, but a copy-paste grep for the literal string won't match real output. #platform-delivery-tooling-team
7 Setting only runAsUser leaves fsGroup: 0 · operator.mdx:109 ✅ Confirmed The chart never defaults fsGroup to an in-range value, and the pre-existing example carried fsGroup: 0; the internal error is triggered by exactly fsGroup: 0. Overriding only runAsUser does not change fsGroup. helm-charts:wandb-base/templates/_pods.tpl#L59-L60; #platform-delivery-tooling-team

Every source above is a clickable link to where you can verify it. Full quotes and commit pins are in the Sources and decision log comment below.

Open items — recommended next steps

  • Revert "Added photos to GitHub repo" #6 (the only non-clean verdict): Optional polish — the note quotes fsGroup: Invalid value: [0]: 0 is not an allowed group, but OpenShift emits .spec.securityContext.fsGroup: Invalid value: []int64{0}: 0 is not an allowed group. Consider matching the real output so readers can grep it. Not blocking. SME: Flamarion Jorge (reported and fixed this exact error on OpenShift in #platform-delivery-tooling-team) — ask: "OK to quote the abbreviated [0] form, or should the doc use the exact []int64{0} string?"

Scope notes

  • The unchanged container: / securityContext: block (lines 98-105) was not re-verified — it is pre-existing context, not added/modified by this diff. Note that wandb-base consumes container security context under containers.<name>.securityContext or the default securityContext; the singular container.securityContext shown is outside this diff's scope.
  • All 7 in-scope assertions are ✅, so there are no inline flags on this PR — inline noise is reserved for ❌ / ⛔ / 🟡.

🤖 Generated by the CoreWeave Docs Team's Validator (beta-validator v1.0.0). Sources are linked inline; this is an accuracy aid, not a substitute for SME review.

@johndmulhausen

Copy link
Copy Markdown
Contributor

📚 Sources and decision log

Full quotes and pins behind the validation table above.

Chart source (Sourcegraph, wandb/helm-charts @ de864305)

  • charts/wandb-base/templates/_pods.tpl L59-L60 — the pod-level securityContext is rendered from the podSecurityContext key (per-component .podData.podSecurityContext merged over global .root.Values.podSecurityContext):
    securityContext:
     {{- tpl (toYaml (merge (default dict .podData.podSecurityContext) $.root.Values.podSecurityContext) | nindent 4) $.root }}
    
    The app, api, console, weave, and similar components in operator-wandb are built on wandb-base, so this is the key they consume — confirming that values placed under pod.securityContext are not applied to runAsUser/fsGroup for these components.
  • charts/wandb-base/values.yaml L101-L104 — the default: podSecurityContext:\n runAsNonRoot: true\n runAsUser: 999\n runAsGroup: 0. This is the UID 999 default.
  • charts/wandb-base/README.md L147-L165 — "Security Contexts … 1. Pod Security Context: Applied to all containers in the pod — Defined in podSecurityContext," with the example podSecurityContext:\n runAsNonRoot: true\n runAsUser: 999\n fsGroup: 0.
  • Rendered chart output confirms the app deployment (# Source: operator-wandb/charts/app/templates/deployment.yaml) ships runAsUser: 999 by default — see test-configs/operator-wandb/__snapshots__/default.snap.
  • Contrast: the auxiliary subcharts (yace, mysql, stackdriver, otel, prometheus exporters) call include "wandb.podSecurityContext" .Values.pod.securityContext — those do use pod.securityContext. The PR's scope is correctly limited to the wandb-base-derived app/api components, which use podSecurityContext.

OpenShift API (Sourcegraph, vendored copy)

  • github.com/coreweave/cw-ceph-csivendor/github.com/openshift/api/security/v1/consts.go L4-L6:
    UIDRangeAnnotation = "openshift.io/sa.scc.uid-range"
    // ... Groups are in the form of a Block which supports {start}/{length} or {start}-{end}
    
    Confirms both the annotation name and the {start}/{length} value format (e.g. 1000810000/10000).

Internal Slack (Glean) — #platform-delivery-tooling-team, "Thread between Flamarion and Venky" (2025-06-04)

  • Permalink: https://coreweave.enterprise.slack.com/archives/C06CKRTPKDF/p1749042317713539
  • Flamarion Jorge, deploying W&B on OpenShift, hit the exact error the PR documents:

    Error creating: pods "wandb-smj-" is forbidden: unable to validate against any security context constraint: [… provider restricted-v2: .spec.securityContext.fsGroup: Invalid value: []int64{0}: 0 is not an allowed group, provider restricted-v2: .containers[0].runAsUser: Invalid value: 999: must be in the ranges: [1000730000, 1000739999] …]

  • Working fix (same shape as the PR's documented example), permalink https://coreweave.enterprise.slack.com/archives/C06CKRTPKDF/p1749219978879589?thread_ts=1749042317.713539&cid=C06CKRTPKDF :
    podSecurityContext:
      runAsNonRoot: true
      runAsUser: 1000730000 # Your OpenShift user ID
      runAsGroup: 0
      fsGroup: 1000730000 # Your OpenShift user ID
      fsGroupChangePolicy: "Always"
      seccompProfile:
        type: "RuntimeDefault"
    
  • The engineer also noted the component "didn't seem to pick the setup from the wandb-base" until keyed correctly — supporting the podSecurityContext vs pod.securityContext distinction.

Note on the qualifier (#6): the doc's [0] is a simplification of the real []int64{0} in the OpenShift error; the human-readable message ("0 is not an allowed group") is verbatim.

🤖 Generated by the CoreWeave Docs Team's Validator (beta-validator v1.0.0).

Review feedback: OpenShift emits Invalid value: []int64{0}, not [0], so quote
the real string that readers can grep.
@pnalubandhu-a11y

Copy link
Copy Markdown
Author

Applied the #6 qualifier — updated the quoted error to the exact Invalid value: []int64{0} form (a7150a7) so it's greppable against real output.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the OpenShift securityContext example in the self-managed operator documentation to ensure the Helm values actually apply to the api/app components and to prevent restricted-v2 SCC rejections.

Changes:

  • Switch the example from pod.securityContext to the chart-consumed podSecurityContext.
  • Add an OpenShift command for discovering the namespace UID range and guidance for selecting values.
  • Add a note explaining why both runAsUser and fsGroup must be set for OpenShift restricted-v2.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

W&B must start as the root group (`$GID=0`) for file system permissions to function properly.
</Note>

Find your OpenShift project's assigned UID range and use the start of that range for both `runAsUser` and `fsGroup`:
Comment on lines +75 to +76
oc get namespace <your-namespace> -o jsonpath='{.metadata.annotations.openshift\.io/sa\.scc\.uid-range}'
# e.g. 1000810000/10000 → use 1000810000
#### Run the container as an un-privileged user

OpenShift and similar orchestrators often reject containers that run as root, so W&B containers must be configured to run as a non-root user that still belongs to the root group. By default, containers use a `$UID` of 999. Specify `$UID` >= 100000 and a `$GID` of 0 if your orchestrator requires the container run with a non-root user.
OpenShift and similar orchestrators often reject containers that run as root, so W&B containers must be configured to run as a non-root user that still belongs to the root group. By default, containers use a `$UID` of 999. Specify a `runAsUser` and `fsGroup` **within your project's assigned UID range** and a `$GID` of 0 if your orchestrator requires the container run with a non-root user.
@pnalubandhu-a11y

Copy link
Copy Markdown
Author

@wandb/docs-team gentle nudge — CI-green and ready for review whenever someone has a cycle. Part of the self-managed OpenShift docs series (companions #2870 / #2872 / #2874). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants