diff --git a/deploy/rustfs-operator/README.md b/deploy/rustfs-operator/README.md index b8eb386..47c128d 100755 --- a/deploy/rustfs-operator/README.md +++ b/deploy/rustfs-operator/README.md @@ -21,6 +21,57 @@ To install in a specific namespace: helm install rustfs-operator deploy/rustfs-operator/ --namespace rustfs-system --create-namespace ``` +### OpenShift Installation + +Enable the OpenShift profile so the chart omits the fixed Pod and container +security contexts from the Operator, Console, and optional Console frontend +Deployments. OpenShift SecurityContextConstraints (SCC) then assigns values +valid for the installation namespace, matching the MinIO Operator installation +contract: + +```bash +helm upgrade --install rustfs-operator deploy/rustfs-operator/ \ + --namespace rustfs-system \ + --create-namespace \ + --set openshift.enabled=true +``` + +For Tenant workloads, use explicit empty Pool security contexts as shown in +`examples/openshift-tenant.yaml`: + +```yaml +spec: + pools: + - name: pool-0 + securityContext: {} + containerSecurityContext: {} +``` + +The empty objects delegate UID, GID, FSGroup, and container security settings +to the namespace SCC. They are an OpenShift-specific contract; generic +Kubernetes Pod Security admission validates fields but does not assign an +allowed runtime identity. Keep `openshift.enabled=false` and omit the Tenant +fields on generic Kubernetes so the RustFS defaults remain in effect. + +This profile provides SCC-compatible manifests but does not by itself imply +OpenShift certification or OperatorHub distribution. Support is currently +limited to `restricted-v2`; the `restricted-v3` requirement to set +`spec.hostUsers: false` is not implemented. + +The RustFS server image is an independent prerequisite. It must support an +arbitrary SCC-assigned UID: writable image-layer directories, including +`/data` and `/logs`, must be owned by group `0` and grant the group the same +permissions as the owner. Images that keep those directories as +`10001:10001` with mode `0750` are not compatible even after fixed IDs are +removed from the Pod spec. Use a rebuilt or fixed image before applying the +OpenShift Tenant example; the chart cannot repair image filesystem ownership. + +The optional split frontend is disabled by default. Its image must also be +verified for arbitrary-UID execution, writable nginx runtime paths, and +unprivileged port binding before setting `console.frontend.enabled=true` on +OpenShift. Omitting its `securityContext` does not make an incompatible nginx +image OpenShift-ready. + ## Uninstalling the Chart To uninstall/delete the `rustfs-operator` deployment: @@ -176,6 +227,7 @@ The generated ClusterRole grants only `get`, `list`, and `watch` for Secrets and | Parameter | Description | Default | |-----------|-------------|---------| +| `openshift.enabled` | Omit chart-managed Deployment security contexts and delegate runtime identity to OpenShift SCC | `false` | | `namespace` | Namespace to deploy to | `""` (uses release namespace) | | `commonLabels` | Labels to add to all resources | `{}` | | `commonAnnotations` | Annotations to add to all resources | `{}` | @@ -352,6 +404,37 @@ cluster-scoped CRDs first so the API server accepts fields introduced by the new Operator version. The dedicated field manager deliberately takes ownership of the chart-managed CRD fields, including CRDs originally created by Helm. +When adopting OpenShift mode on an existing installation, apply the CRDs first, +then upgrade the chart with `openshift.enabled=true`, and wait for the Operator +and Console rollouts before changing Tenant manifests. The chart upgrade rolls +only those Deployments; it does not rewrite Tenant or PVC API objects. The two +empty objects form one explicit delegation signal; a lone empty object retains +the Operator defaults for compatibility with legacy field-based clients. + +Inventory existing paired empty objects with the `jq` preflight in the Operator +user guide before upgrading. This release changes such a pair from inheriting +Operator defaults to SCC delegation, so every match is a breaking migration +decision. Changing an existing Pool to `securityContext: {}` and +`containerSecurityContext: {}` changes its StatefulSet Pod template and causes +a Tenant Pod rollout. A changed SCC-assigned FSGroup can also trigger volume +ownership work during first mount; large volumes can start slowly, and CSI or +root-squash permission incompatibilities can prevent mount or write. Verify the +namespace SCC, arbitrary-UID image, and StorageClass with existing data, keep a +recoverable backup, and schedule a maintenance window. A single-replica Tenant +can be unavailable during restart, while a multi-replica Tenant temporarily +runs with reduced capacity. + +Do not roll back to an Operator version that interprets explicit empty objects +as a request for the fixed RustFS UID/GID defaults. Such a controller can put +the fixed identity back into the StatefulSet template and OpenShift may reject +the resulting Pods. Recover by rolling forward or restore a complete security +context that is valid for the namespace SCC before downgrading. + +Likewise, disabling `openshift.enabled` or rolling the chart back to a version +without this profile reintroduces the chart's fixed Operator/Console identities +and rolls those Deployments. Confirm that the namespace SCC permits those +identities before doing so; otherwise keep the profile enabled and roll forward. + This release adds secure defaults to generated RustFS Pods and containers. Existing compatible Tenants whose StatefulSet templates do not already contain those values will roll on their next reconciliation. Schedule the upgrade in a diff --git a/deploy/rustfs-operator/templates/console-deployment.yaml b/deploy/rustfs-operator/templates/console-deployment.yaml index 2903e59..d03a337 100755 --- a/deploy/rustfs-operator/templates/console-deployment.yaml +++ b/deploy/rustfs-operator/templates/console-deployment.yaml @@ -1,4 +1,6 @@ {{- $consoleLoginAdmission := default dict .Values.console.loginAdmission -}} +{{- $openShift := default dict .Values.openshift -}} +{{- $openShiftEnabled := default false $openShift.enabled -}} {{- $reservedConsoleEnv := dict "CONSOLE_LOGIN_ADMISSION_REQUESTS_PER_SECOND" "console.loginAdmission.requestsPerSecond" "CONSOLE_LOGIN_ADMISSION_BURST" "console.loginAdmission.burst" @@ -44,10 +46,12 @@ spec: imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} + {{- if not $openShiftEnabled }} {{- with .Values.console.podSecurityContext }} securityContext: {{- toYaml . | nindent 8 }} {{- end }} + {{- end }} containers: - name: console image: "{{ .Values.console.image.repository }}:{{ .Values.console.image.tag | default .Values.operator.image.tag }}" @@ -100,10 +104,12 @@ spec: resources: {{- toYaml . | nindent 12 }} {{- end }} + {{- if not $openShiftEnabled }} {{- with .Values.console.securityContext }} securityContext: {{- toYaml . | nindent 12 }} {{- end }} + {{- end }} {{- with .Values.console.nodeSelector }} nodeSelector: {{- toYaml . | nindent 8 }} diff --git a/deploy/rustfs-operator/templates/console-frontend-deployment.yaml b/deploy/rustfs-operator/templates/console-frontend-deployment.yaml index 61a748b..9022229 100755 --- a/deploy/rustfs-operator/templates/console-frontend-deployment.yaml +++ b/deploy/rustfs-operator/templates/console-frontend-deployment.yaml @@ -1,3 +1,5 @@ +{{- $openShift := default dict .Values.openshift -}} +{{- $openShiftEnabled := default false $openShift.enabled -}} {{- if and .Values.console.enabled .Values.console.frontend.enabled -}} apiVersion: apps/v1 kind: Deployment @@ -47,8 +49,10 @@ spec: resources: {{- toYaml . | nindent 12 }} {{- end }} + {{- if not $openShiftEnabled }} {{- with .Values.console.frontend.securityContext }} securityContext: {{- toYaml . | nindent 12 }} {{- end }} + {{- end }} {{- end }} diff --git a/deploy/rustfs-operator/templates/deployment.yaml b/deploy/rustfs-operator/templates/deployment.yaml index 425da12..5ba47d4 100755 --- a/deploy/rustfs-operator/templates/deployment.yaml +++ b/deploy/rustfs-operator/templates/deployment.yaml @@ -1,6 +1,8 @@ {{- $livenessProbe := default dict .Values.operator.livenessProbe -}} {{- $readinessProbe := default dict .Values.operator.readinessProbe -}} {{- $stsAdmission := default dict .Values.sts.admission -}} +{{- $openShift := default dict .Values.openshift -}} +{{- $openShiftEnabled := default false $openShift.enabled -}} {{- if and (not .Values.operator.metrics.enabled) (or (hasKey $livenessProbe "httpGet") (hasKey $readinessProbe "httpGet")) -}} {{- fail "operator.metrics.enabled=false requires overriding operator.livenessProbe and operator.readinessProbe because the chart defaults use the metrics port" -}} {{- end -}} @@ -58,10 +60,12 @@ spec: imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} + {{- if not $openShiftEnabled }} {{- with .Values.operator.podSecurityContext }} securityContext: {{- toYaml . | nindent 8 }} {{- end }} + {{- end }} containers: - name: operator image: "{{ .Values.operator.image.repository }}:{{ .Values.operator.image.tag }}" @@ -150,10 +154,12 @@ spec: resources: {{- toYaml . | nindent 12 }} {{- end }} + {{- if not $openShiftEnabled }} {{- with .Values.operator.securityContext }} securityContext: {{- toYaml . | nindent 12 }} {{- end }} + {{- end }} {{- with .Values.operator.nodeSelector }} nodeSelector: {{- toYaml . | nindent 8 }} diff --git a/deploy/rustfs-operator/values.schema.json b/deploy/rustfs-operator/values.schema.json index d223c16..fde5723 100644 --- a/deploy/rustfs-operator/values.schema.json +++ b/deploy/rustfs-operator/values.schema.json @@ -17,6 +17,18 @@ ], "description": "Kubernetes cluster DNS domain used for Tenant peer URLs, generated TLS SANs, and operator STS auto TLS." }, + "openshift": { + "type": "object", + "required": [ + "enabled" + ], + "properties": { + "enabled": { + "type": "boolean", + "description": "Omit chart-managed Deployment security contexts so OpenShift SCC can assign the runtime identity." + } + } + }, "sts": { "type": "object", "properties": { diff --git a/deploy/rustfs-operator/values.yaml b/deploy/rustfs-operator/values.yaml index 6dfc20e..c7d122b 100755 --- a/deploy/rustfs-operator/values.yaml +++ b/deploy/rustfs-operator/values.yaml @@ -3,6 +3,12 @@ # Kubernetes cluster DNS domain used for Tenant peer URLs and generated TLS SANs. clusterDomain: cluster.local +# OpenShift installation compatibility. When enabled, the chart omits Pod and +# container securityContext fields from its own Deployments so the namespace +# SecurityContextConstraints (SCC) can assign an allowed UID and FSGroup. +openshift: + enabled: false + # Operator deployment configuration operator: # Number of operator replicas diff --git a/docs/operator-user-guide.md b/docs/operator-user-guide.md index 734d314..8829943 100644 --- a/docs/operator-user-guide.md +++ b/docs/operator-user-guide.md @@ -79,6 +79,35 @@ helm install rustfs-operator deploy/rustfs-operator/ \ --create-namespace ``` +On OpenShift, enable SCC-managed runtime identities for the Operator, Console, +and optional split frontend: + +```bash +helm upgrade --install rustfs-operator deploy/rustfs-operator/ \ + --namespace rustfs-system \ + --create-namespace \ + --set openshift.enabled=true +``` + +This follows the MinIO Operator installation behavior: chart-managed +Deployments omit their Pod and container `securityContext`, allowing the +namespace SecurityContextConstraints (SCC) to assign an allowed UID and +FSGroup. This is manifest compatibility, not an OpenShift certification claim. +Generic Kubernetes installations must keep the default +`openshift.enabled=false` behavior. The current target is `restricted-v2`; +`restricted-v3` also requires `spec.hostUsers: false`, which is not yet covered. + +SCC-compatible manifests are insufficient when the server image assumes UID +`10001`. Before deploying a Tenant, use an arbitrary-UID-compatible image whose +writable image-layer directories, including `/data` and `/logs`, are owned by +group `0` and give the group the same permissions as the owner. An image with +those directories owned by `10001:10001` and mode `0750` remains incompatible; +the Operator cannot repair image filesystem ownership. + +Keep the optional split frontend disabled unless its image is independently +verified for arbitrary-UID nginx runtime paths and unprivileged port binding. +SCC-assigned identity alone cannot repair an incompatible frontend image. + Verify the operator and Console pods: ```bash @@ -108,6 +137,56 @@ are cluster-scoped and shared by all Tenant namespaces. The dedicated field manager deliberately takes ownership of the chart-managed CRD fields so this upgrade also works for CRDs originally created by Helm. +When enabling OpenShift support on an existing installation, keep this order: +apply both CRDs, upgrade the chart with `openshift.enabled=true`, wait for the +Operator and Console rollouts, and only then update Tenant security contexts. +Before upgrading the controller, inventory every explicit empty pair at Tenant +and Pool scope: + +```bash +kubectl get tenants.rustfs.com -A -o json | jq -r ' + def empty_object: type == "object" and length == 0; + .items[] as $tenant | + ([ + (select(($tenant.spec | has("securityContext")) and + ($tenant.spec | has("containerSecurityContext")) and + ($tenant.spec.securityContext | empty_object) and + ($tenant.spec.containerSecurityContext | empty_object)) | "spec"), + ($tenant.spec.pools[]? | + select((has("securityContext")) and + (has("containerSecurityContext")) and + (.securityContext | empty_object) and + (.containerSecurityContext | empty_object)) | + "pool:" + .name) + ]) as $locations | + select($locations | length > 0) | + [$tenant.metadata.namespace, $tenant.metadata.name, ($locations | join(","))] | + @tsv' +``` + +This release deliberately changes a paired `{}`/`{}` from "inherit Operator +defaults" to "delegate to platform admission". A lone empty object keeps the +legacy behavior. Treat every reported pair as a breaking migration decision: +remove both fields before upgrading when Operator defaults should remain, or +keep both only after validating the target SCC and image. + +The chart upgrade does not rewrite Tenant or PVC API objects. Converting an +existing Pool to the paired empty security contexts changes its StatefulSet Pod +template and rolls that Pool. A changed SCC-assigned FSGroup can also make +kubelet or the CSI driver update volume ownership on first mount; large volumes +can start slowly, and storage with incompatible `fsGroupPolicy`, root-squash, or +permission behavior can fail to mount or write. Test the StorageClass with +existing data, keep a recoverable backup, and schedule a maintenance window. A +single-replica Tenant can be unavailable during restart and a multi-replica +Tenant temporarily runs with reduced capacity. + +Do not downgrade to a controller that restores fixed UID/GID defaults for the +empty pair, because OpenShift SCC may reject that rollback. Roll forward, or +restore an SCC-valid complete security context before downgrading. Disabling +`openshift.enabled` or rolling back to a chart without the profile also +reintroduces fixed Operator/Console identities and rolls those Deployments; do +that only if the namespace SCC allows the fixed IDs. + Existing manifests that omit `users[].credsSecret` remain compatible. Wait for the new Operator rollout to complete before relying on an explicit user Secret reference; older binaries continue using the same-name Secret convention. @@ -170,6 +249,7 @@ Common chart sections: | Section | Purpose | |---------|---------| +| `openshift` | SCC-compatible rendering for the Operator, Console, and optional frontend Deployments. Disabled by default. | | `operator` | Operator Deployment replicas, image, resources, probes, metrics, scheduling, leader election, and tenant monitoring. | | `sts` | Operator STS endpoint, service port, TokenReview audience, and TLS handling. | | `serviceAccount` / `rbac` | Operator ServiceAccount and RBAC creation. | @@ -467,6 +547,27 @@ be rejected by cluster admission policy. For legacy compatibility, an explicit `runAsUser: 0` without an explicit `runAsNonRoot` derives `runAsNonRoot: false`; that configuration cannot run in a `restricted` namespace. +On OpenShift, use explicit empty objects at Pool level to delegate the runtime +identity and container security settings to the namespace SCC, following the +MinIO Operator contract: + +```yaml +spec: + pools: + - name: pool-0 + securityContext: {} + containerSecurityContext: {} +``` + +See `examples/openshift-tenant.yaml`. The distinction between omission and an +explicit empty pair is intentional: omission requests the RustFS defaults; +the paired `{}`/`{}` requests SCC ownership. A lone empty object retains the +defaults for compatibility. Do not use the pair on generic Kubernetes unless +another admission controller supplies equivalent settings. Updating an +existing Pool to this form rolls its StatefulSet Pods. The example uses a +placeholder image deliberately; replace it only with a verified +arbitrary-UID-compatible RustFS image. + `RuntimeDefault` also requires a RustFS image that can run under the runtime's default seccomp profile. `rustfs/rustfs:1.0.0-beta.8` is not compatible because its Tokio runtime enables io_uring; use a build containing diff --git a/docs/operator-user-guide.zh-CN.md b/docs/operator-user-guide.zh-CN.md index 9902395..36280b8 100644 --- a/docs/operator-user-guide.zh-CN.md +++ b/docs/operator-user-guide.zh-CN.md @@ -81,6 +81,29 @@ helm install rustfs-operator deploy/rustfs-operator/ \ --create-namespace ``` +在 OpenShift 上，应让 SCC 管理 Operator、Console 和可选独立前端的运行身份： + +```bash +helm upgrade --install rustfs-operator deploy/rustfs-operator/ \ + --namespace rustfs-system \ + --create-namespace \ + --set openshift.enabled=true +``` + +该行为与 MinIO Operator 的安装方式一致：Chart 管理的 Deployment 不渲染 Pod +和容器 `securityContext`，由安装 namespace 的 SecurityContextConstraints（SCC） +分配合法 UID 和 FSGroup。这仅表示 manifest 与 SCC 兼容，不代表已获得 OpenShift +认证。普通 Kubernetes 安装必须保留默认的 `openshift.enabled=false`。当前支持目标 +限定为 `restricted-v2`；`restricted-v3` 还要求 `spec.hostUsers: false`，目前尚未覆盖。 + +仅有 SCC 兼容 manifest 还不够，RustFS server 镜像也必须支持 SCC 分配的任意 UID。 +部署 Tenant 前，应确认 `/data`、`/logs` 等镜像层可写目录属于 group `0`，并且 group +权限与 owner 权限相同。如果这些目录仍为 `10001:10001`、权限 `0750`，即使 Pod spec +不再固定 UID 也无法兼容；Operator 不能修复镜像内部的文件所有权。 + +可选独立前端应保持关闭，除非其镜像已验证 nginx 运行目录支持任意 UID，并能绑定 +非特权端口。仅让 SCC 分配身份不能修复不兼容的前端镜像。 + 验证 Operator 和 Console Pod： ```bash @@ -109,6 +132,48 @@ helm upgrade rustfs-operator deploy/rustfs-operator/ \ 专用 field manager 会显式接管 Chart 管理的 CRD 字段，因此也适用于最初由 Helm 创建的 CRD。 +已有安装启用 OpenShift 支持时，应严格按以下顺序执行：先应用两个 CRD，再使用 +`openshift.enabled=true` 升级 Chart，等待 Operator 和 Console rollout 完成，最后 +更新 Tenant 安全上下文。升级 Controller 前，应先盘点 Tenant 和 Pool 两个层级已有的 +成对空对象： + +```bash +kubectl get tenants.rustfs.com -A -o json | jq -r ' + def empty_object: type == "object" and length == 0; + .items[] as $tenant | + ([ + (select(($tenant.spec | has("securityContext")) and + ($tenant.spec | has("containerSecurityContext")) and + ($tenant.spec.securityContext | empty_object) and + ($tenant.spec.containerSecurityContext | empty_object)) | "spec"), + ($tenant.spec.pools[]? | + select((has("securityContext")) and + (has("containerSecurityContext")) and + (.securityContext | empty_object) and + (.containerSecurityContext | empty_object)) | + "pool:" + .name) + ]) as $locations | + select($locations | length > 0) | + [$tenant.metadata.namespace, $tenant.metadata.name, ($locations | join(","))] | + @tsv' +``` + +此版本会有意把成对的 `{}`/`{}` 从“继承 Operator 默认值”改为“委托给平台 +准入控制器”；单独出现的空对象仍保持旧行为。每一条扫描结果都必须作为破坏性迁移 +决定处理：需要保留 Operator 默认值时，应在升级前删除这两个字段；只有验证目标 SCC +和镜像后，才能保留这一对空对象。 + +Chart 升级不会改写 Tenant 或 PVC API 对象。把已有 Pool 改成成对空安全上下文会改变 +StatefulSet Pod template 并滚动该 Pool。SCC 分配的 FSGroup 发生变化时，kubelet 或 +CSI driver 在首次挂载时还可能修改卷内权限；大容量卷启动会变慢，`fsGroupPolicy`、 +root-squash 或权限行为不兼容的存储甚至可能挂载或写入失败。应先用已有数据验证 +StorageClass，保留可恢复备份，并安排维护窗口。单副本 Tenant 在重启期间可能不可用， +多副本 Tenant 会暂时以较低容量运行。不要降级到会把空对象对重新解释为固定 UID/GID +的旧 Controller，否则 OpenShift SCC 可能拒绝回滚后的 Pod。应向前升级；或者先恢复 +一套符合 SCC 的完整安全上下文，再执行降级。 +关闭 `openshift.enabled` 或回滚到没有该配置的 Chart 也会重新引入 Operator/Console +固定身份并滚动这些 Deployment；只有 namespace SCC 允许这些固定 ID 时才能执行。 + 未配置 `users[].credsSecret` 的已有 manifest 保持兼容。只有在新 Operator rollout 全部完成后，才能依赖显式的 user Secret 引用；旧 binary 仍会按 user 同名规则读取 Secret。 @@ -163,6 +228,7 @@ helm upgrade --install rustfs-operator deploy/rustfs-operator/ \ | 配置段 | 用途 | |--------|------| +| `openshift` | Operator、Console 和可选前端 Deployment 的 SCC 兼容渲染；默认关闭。 | | `operator` | Operator Deployment 副本数、镜像、资源、探针、metrics、调度、leader election 和 Tenant monitor。 | | `sts` | Operator STS 端点、Service 端口、TokenReview audience 和 TLS。 | | `serviceAccount` / `rbac` | Operator ServiceAccount 和 RBAC 创建策略。 | @@ -453,6 +519,23 @@ capabilities，满足 Kubernetes Pod Security `restricted` 对应要求。显式 `runAsUser: 0`、但没有显式配置 `runAsNonRoot`，Operator 会推导 `runAsNonRoot: false`；该配置不能用于 `restricted` namespace。 +在 OpenShift 上，应在 Pool 级使用显式空对象，把运行身份和容器安全设置交给 +namespace SCC；该契约与 MinIO Operator 保持一致： + +```yaml +spec: + pools: + - name: pool-0 + securityContext: {} + containerSecurityContext: {} +``` + +完整示例见 `examples/openshift-tenant.yaml`。省略字段与显式空对象对的含义不同：省略 +表示使用 RustFS 默认值，成对的 `{}`/`{}` 表示由 SCC 管理；单独空对象为兼容旧配置 +仍保留默认值。普通 Kubernetes 集群若没有其他准入控制器补充等价设置，不应使用这 +一空对象对。把已有 Pool 更新成该形式会滚动 StatefulSet Pod。示例有意使用占位镜像， +必须替换为经过验证、支持任意 UID 的 RustFS 镜像。 + `RuntimeDefault` 还要求 RustFS 镜像能够在容器运行时默认 seccomp 下启动。 `rustfs/rustfs:1.0.0-beta.8` 的 Tokio runtime 启用了 io_uring，因此不兼容； 请使用包含 [rustfs/rustfs#4364](https://github.com/rustfs/rustfs/pull/4364) diff --git a/e2e/tests/openshift_manifest.rs b/e2e/tests/openshift_manifest.rs new file mode 100644 index 0000000..06365c4 --- /dev/null +++ b/e2e/tests/openshift_manifest.rs @@ -0,0 +1,179 @@ +// Copyright 2025 RustFS Team +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use serde_yaml_ng::Value; +use std::{path::PathBuf, process::Command}; + +#[test] +fn openshift_values_schema_declares_boolean_switch() { + let schema = std::fs::read_to_string( + repository_root().join("deploy/rustfs-operator/values.schema.json"), + ) + .expect("Helm values schema exists"); + let schema: serde_json::Value = + serde_json::from_str(&schema).expect("Helm values schema is valid JSON"); + + assert_eq!( + schema["properties"]["openshift"]["properties"]["enabled"]["type"].as_str(), + Some("boolean") + ); +} + +#[test] +fn openshift_chart_mode_delegates_deployment_security_contexts_to_scc() { + let Some(default_render) = helm_template(&["--set", "console.frontend.enabled=true"]) else { + return; + }; + assert!( + default_render.status.success(), + "default chart render failed: {}", + String::from_utf8_lossy(&default_render.stderr) + ); + let default_output = + String::from_utf8(default_render.stdout).expect("default helm output is UTF-8"); + let default_documents = yaml_documents(&default_output, "default chart"); + assert_eq!( + deployment(&default_documents, "rustfs-operator")["spec"]["template"]["spec"] + ["securityContext"]["fsGroup"] + .as_i64(), + Some(65534) + ); + assert_eq!( + deployment_container(&default_documents, "rustfs-operator")["securityContext"]["runAsUser"] + .as_i64(), + Some(65534) + ); + assert_eq!( + deployment_container(&default_documents, "rustfs-operator-console")["securityContext"] + ["runAsUser"] + .as_i64(), + Some(65534) + ); + assert_eq!( + deployment_container(&default_documents, "rustfs-operator-console-frontend") + ["securityContext"]["runAsUser"] + .as_i64(), + Some(101) + ); + + let openshift_render = helm_template(&[ + "--set", + "openshift.enabled=true", + "--set", + "console.frontend.enabled=true", + ]) + .expect("helm was available for the default render"); + assert!( + openshift_render.status.success(), + "OpenShift chart render failed: {}", + String::from_utf8_lossy(&openshift_render.stderr) + ); + let openshift_output = + String::from_utf8(openshift_render.stdout).expect("OpenShift helm output is UTF-8"); + let openshift_documents = yaml_documents(&openshift_output, "OpenShift chart"); + + for name in [ + "rustfs-operator", + "rustfs-operator-console", + "rustfs-operator-console-frontend", + ] { + let deployment = deployment(&openshift_documents, name); + assert!( + deployment["spec"]["template"]["spec"]["securityContext"].is_null(), + "OpenShift mode must omit the {name} Pod securityContext" + ); + for container in deployment["spec"]["template"]["spec"]["containers"] + .as_sequence() + .expect("Deployment containers are a sequence") + { + assert!( + container["securityContext"].is_null(), + "OpenShift mode must omit the {name} container securityContext" + ); + } + } +} + +#[test] +fn openshift_tenant_example_uses_explicit_empty_pool_security_contexts() { + let manifest = + std::fs::read_to_string(repository_root().join("examples/openshift-tenant.yaml")) + .expect("OpenShift Tenant example exists"); + let documents = yaml_documents(&manifest, "OpenShift Tenant example"); + let tenant = documents + .iter() + .find(|document| document["kind"].as_str() == Some("Tenant")) + .expect("example contains a Tenant"); + let pool = &tenant["spec"]["pools"][0]; + + for field in ["securityContext", "containerSecurityContext"] { + let value = pool[field] + .as_mapping() + .unwrap_or_else(|| panic!("{field} must be an object")); + assert!(value.is_empty(), "{field} must be an explicit empty object"); + } +} + +fn repository_root() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .parent() + .expect("e2e crate has a repository parent") + .to_path_buf() +} + +fn helm_template(arguments: &[&str]) -> Option { + if Command::new("helm").arg("version").output().is_err() { + assert!( + std::env::var_os("CI").is_none(), + "helm must be installed in CI" + ); + eprintln!("skipping helm template assertions: helm is not installed"); + return None; + } + + Some( + Command::new("helm") + .arg("template") + .arg("rustfs-operator") + .arg(repository_root().join("deploy/rustfs-operator")) + .args(arguments) + .output() + .expect("helm template runs"), + ) +} + +fn yaml_documents(input: &str, description: &str) -> Vec { + input + .split("---") + .filter(|document| !document.trim().is_empty()) + .map(|document| { + serde_yaml_ng::from_str(document) + .unwrap_or_else(|error| panic!("{description} contains invalid YAML: {error}")) + }) + .collect() +} + +fn deployment<'a>(documents: &'a [Value], name: &str) -> &'a Value { + documents + .iter() + .find(|document| { + document["kind"].as_str() == Some("Deployment") + && document["metadata"]["name"].as_str() == Some(name) + }) + .unwrap_or_else(|| panic!("missing Deployment {name}")) +} + +fn deployment_container<'a>(documents: &'a [Value], name: &str) -> &'a Value { + &deployment(documents, name)["spec"]["template"]["spec"]["containers"][0] +} diff --git a/examples/README.md b/examples/README.md index 156b096..66027d0 100755 --- a/examples/README.md +++ b/examples/README.md @@ -9,6 +9,7 @@ This directory contains example Tenant configurations for the RustFS Kubernetes | Example | Use Case | Complexity | Storage | Best For | |---------|----------|------------|---------|----------| | [minimal-dev-tenant.yaml](./minimal-dev-tenant.yaml) | Development/Learning | ⭐ Simple | 10Gi | **Start here** if new | +| [openshift-tenant.yaml](./openshift-tenant.yaml) | OpenShift SCC | ⭐ Simple | 80Gi | OpenShift `restricted-v2` namespaces | | [simple-tenant.yaml](./simple-tenant.yaml) | Documentation Reference | ⭐⭐ Moderate | Configurable | Learning all options | | [secret-credentials-tenant.yaml](./secret-credentials-tenant.yaml) | Secret-based Credentials | ⭐ Simple | Configurable | **Production credential security** | | [provisioning-tenant.yaml](./provisioning-tenant.yaml) | Policy/User/Bucket Provisioning | ⭐⭐ Moderate | 40Gi | Tenant bootstrap automation | @@ -26,6 +27,13 @@ This directory contains example Tenant configurations for the RustFS Kubernetes 2. Read **simple-tenant.yaml** to understand all options 3. Explore other examples based on your use case +On OpenShift, use **openshift-tenant.yaml** only with an arbitrary-UID-compatible +RustFS image. Its two explicit empty Pool security contexts form one delegation +signal for UID, GID, FSGroup, and container security settings. Both objects are +required; a lone empty object retains Operator defaults for compatibility. Do +not copy the pair to a generic Kubernetes namespace unless its admission policy +supplies equivalent security settings. + **Important Notes:** - RustFS S3 API runs on port **9000** - RustFS Console UI (per Tenant Service `{tenant}-console`) runs on port **9001** diff --git a/examples/openshift-tenant.yaml b/examples/openshift-tenant.yaml new file mode 100644 index 0000000..5e870c7 --- /dev/null +++ b/examples/openshift-tenant.yaml @@ -0,0 +1,42 @@ +# Copyright 2025 RustFS Team +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: rustfs.com/v1alpha1 +kind: Tenant +metadata: + name: rustfs-openshift + namespace: rustfs-tenant +spec: + # This must be an arbitrary-UID-compatible image. In particular, writable + # image-layer directories such as /data and /logs must be owned by group 0 + # and grant the group the same permissions as the owner. Do not deploy this + # example until such a RustFS image is available in your registry. + image: registry.example.com/rustfs/rustfs:openshift-compatible + pools: + - name: pool-0 + servers: 4 + persistence: + volumesPerServer: 2 + volumeClaimTemplate: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + + # Match the MinIO Operator OpenShift contract: explicit empty objects + # delegate UID, GID, FSGroup, and container security settings to the + # namespace SecurityContextConstraints (SCC). + securityContext: {} + containerSecurityContext: {}