Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add KB for the issue volumes with lot of files take a long time to finish mounting #265

Merged
merged 1 commit into from
Mar 6, 2021

Conversation

PhanLe1010
Copy link
Contributor

Copy link
Member

@yasker yasker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @catherineluse can you help to take a look as well?


When starting workload pod which uses Longhorn volumes,
observe in Longhorn UI that the Longhorn volumes are attached quickly
but it takes a long time for the volumes to finish mounting and workload to be able to start.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When starting a workload pod that uses Longhorn volumes,
the Longhorn UI shows that the Longhorn volumes are attached quickly,
but it takes a long time for the volumes to finish mounting and for the workload to be able to start.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Fixed

observe in Longhorn UI that the Longhorn volumes are attached quickly
but it takes a long time for the volumes to finish mounting and workload to be able to start.

This issue only happens when Longhorn volumes that have many files/directories and the workload pod sets `fsGroup` in the `securityContext` similar to as below:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This issue only happens when the Longhorn volumes have many files/directories and securityContext.fsGroup is set in the workload pod as shown below:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Fixed


## Reason

By default, when seeing `fsGroup` field, each time a volume is mounted, Kubernetes recursively `chown()` and `chmod()` all the files and directories inside the volume.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, when seeing the fsGroup field, each time a volume is mounted, Kubernetes recursively calls chown() and chmod() on all the files and directories inside the volume.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Fixed

When `fsGroupChangePolicy` is set to `OnRootMismatch`, if the root of the volume already has the correct permissions,
the recursive permission and ownership change will be skipped.
It means that if users don't change the `pod.spec.securityContext.fsGroup` between pod's startups,
K8s will only have to check the permissions and ownership of the root and the mounting process will be much faster compared to always recursively change the volumes' ownership and permissions.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compared to always recursively changing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Fixed

…inish mounting`

longhorn/longhorn#2131

Signed-off-by: Phan Le <phan.le@rancher.com>
@PhanLe1010
Copy link
Contributor Author

PhanLe1010 commented Mar 6, 2021

@yasker

Catherine's review comments are addressed

@yasker
Copy link
Member

yasker commented Mar 6, 2021

@catherineluse can you review again? Thanks.

@yasker yasker merged commit 72686df into longhorn:master Mar 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants