title | date | weight | description |
---|---|---|---|
Run a PyTorchJob |
2023-08-09 |
6 |
Run a Kueue scheduled PyTorchJob
|
This page shows how to leverage Kueue's scheduling and resource management capabilities when running Training Operator PyTorchJobs.
This guide is for batch users that have a basic understanding of Kueue. For more information, see Kueue's overview.
Check administer cluster quotas for details on the initial cluster setup.
Check the Training Operator installation guide.
Note that the minimum requirement training-operator version is v1.7.0.
You can modify kueue configurations from installed releases to include PyTorchJobs as an allowed workload.
The target local queue should be specified in the metadata.labels
section of the PyTorchJob configuration.
metadata:
labels:
kueue.x-k8s.io/queue-name: user-queue
spec:
runPolicy:
suspend: true
By default, Kueue will set suspend
to true via webhook and unsuspend it when the PyTorchJob is admitted.
This example is based on https://github.com/kubeflow/training-operator/blob/855e0960668b34992ba4e1fd5914a08a3362cfb1/examples/pytorch/simple.yaml.
{{< include "examples/jobs/sample-pytorchjob.yaml" "yaml" >}}