Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add filesystem group change policy for large minio deployments #14528

Merged
merged 2 commits into from
Mar 11, 2022
Merged

add filesystem group change policy for large minio deployments #14528

merged 2 commits into from
Mar 11, 2022

Conversation

dharapvj
Copy link
Contributor

Description

This change adds fsGroupChangePolicy: onRootMismatch for helm based minio deployments. This basically allows minio to skip checking the fsgroup changes - which can be rather slow operation

Motivation and Context

When using minio helm chart to deploy minio in Kubernetes cluster, if we have a large number of files in minio - e.g. greater than 300Gb, minio pod fails to come up due to PVC failing to get mounted.

We see below warning in kubelet logs for that node where pod is assigned:

W1220 05:32:58.485516    8703 volume_linux.go:52] Setting volume ownership for /var/lib/kubelet/pods/408ef833-2757-4807-a213-596706ddb0c5/volumes/kubernetes.io~vsphere-volume/pvc-573193ec-093a-4cd0-9f15-eed87faa3079 and fsGroup set. If the volume has a lot of files then setting volume ownership could be slow, see https://github.com/kubernetes/kubernetes/issues/69699

Going through kubernetes/kubernetes#69699 as suggested and linked documentation - it is suggested that we should set below setting for minio container.

  securityContext:
    fsGroupChangePolicy: "OnRootMismatch"

How to test this PR?

Deploy minio with a LOT of files in storage in kubernetes and delete the worker node so that minio is forced to move to different node.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation updated
  • Unit tests added/updated

@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-gateway-nas.sh ✔️
mint-compress-encrypt-dist-erasure.sh ✔️
mint-pools.sh ✔️
Deleting image on docker hub
Deleting image locally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants