Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Implement support for multiple sizes huge pages #84051
What type of PR is this?
What this PR does / why we need it:
It was tested on a local cluster with allocated huge pages of two sizes:
With this pod configuration:
Both sizes hugepages where mounted correctly in the container:
Pod level allocations for both huge pages sizes look correct as well:
Does this PR introduce a user-facing change?:
odinuge left a comment
Just some initial thoughts, but otherwise this makes much sense to me. Thanks for your work on this.
The questions about validation are open for discussion, and should probably not be a part of this PR.
Test failures look valid, so those should be addressed. We also have some e2e tests for hugepages, have you verified that they still pass?
Adding hold since this depends on support on node level: #82820
I think that overall changes in emptyDir.go follow the KEP update as well.
The KEP says that
Would it be okay to allow the above case?
This is an interesting corner case. Volumes are Pod level and right now the sizes of those volumes are calculated by sum of all containers requests/limits for hugepages. In theory, if pod has more than one container and more than one hugetlbfs mounts and not all of those volumemounts are used in all of the containers, we need to change logic of validation and logic of calculating sizes for each of hugepage volume.
Not sure if I understand what you mean. The "size" of a volume mount of type hugetlbfs is the page size used, and not the amount of memory available. By default a program can use all the pre allocated huge page memory from such a mount, but this is limited via the hugetlb cgroup.
How to handle multiple containers with multiple hugepage sizes, together with different volumes is maybe something we can incorporate into the KEP, but think it should be ok as is now too. The reason we verify that the page sizes used in the mount is in
The problem will arise when a pod use a page size in a volume without having the size in the requests/limits:
apiVersion: v1 kind: Pod metadata: name: example spec: containers: - name: container volumeMounts: - mountPath: /hugepages name: hugepage volumes: - name: hugepage emptyDir: medium: HugePages # alternatively HugePages-1Gi
This (the example above) can be valid on a given node if it supports 1GiB pages, but we cannot be sure since the scheduler doesn't take huge page support into account when finding a node. This is a case we successfully validates during volume mounting today, but the podspec is still "valid" in the sense that the apiserver will accept it.
In this pod we know that the node running the pod support
apiVersion: v1 kind: Pod metadata: name: example spec: containers: - name: container1 volumeMounts: - mountPath: /hugepages name: hugepage resources: requests: hugepages-1Gi: 2Gi limits: hugepages-1Gi: 2Gi - name: container2 volumeMounts: - mountPath: /hugepages-1Gi name: hugepage-1Gi volumes: - name: hugepage emptyDir: medium: HugePages - name: hugepage-1Gi emptyDir: medium: HugePages-1Gi
This implementation allows Pod to request multiple hugepage resources of different size and mount hugepage volumes using storage medium HugePage-<size>, e.g. spec: containers: resources: requests: hugepages-2Mi: 2Mi hugepages-1Gi: 2Gi volumeMounts: - mountPath: /hugepages-2Mi name: hugepage-2mi - mountPath: /hugepages-1Gi name: hugepage-1gi ... volumes: - name: hugepage-2mi emptyDir: medium: HugePages-2Mi - name: hugepage-1gi emptyDir: medium: HugePages-1Gi NOTE: This is an alpha feature. Feature gate HugePageStorageMediumSize must be enabled for it to work.
Extended GetMountMedium function to check if hugetlbfs volume is mounted with the page size equal to the medium size. Page size is obtained from the 'pagesize' mount option of the mounted hugetlbfs volume.
[APPROVALNOTIFIER] This PR is APPROVED
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing
As part of the telco effort we should provide possibility, to use multiple sizes huge pages under the node. Kubernetes supports this feature as alpha under the 1.18, to enable it you should enable feature gate `HugePageStorageMediumSize`, see kubernetes/kubernetes#84051. Signed-off-by: Artyom Lukianov <firstname.lastname@example.org>