-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jobparser for crd TrainingJob #14
Changes from 7 commits
5448520
1a57114
213ee4a
eec86bb
06ac7da
d1038b5
65dd456
499e1fb
39f1380
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
*~ | ||
vendor/ | ||
.glide/ | ||
.vscode/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
package v1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I noticed that in many of our Go repos, there is a directory This convention would make it hard for other Go projects to import ours. The authors have to write: import "github.com/paddlepaddle/edl/pkg/apis/paddlepaddle/v1" Please be aware that there are two To make this import path concise and clear, we might want import "github.com/paddlepaddle/edl/v1" To achive this, we should have There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please follow Go's convention and rename this file from There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
|
||
import ( | ||
"encoding/json" | ||
"fmt" | ||
) | ||
|
||
// Elastic returns true if the job can scale to more workers. | ||
func (s *TrainingJob) Elastic() bool { | ||
return s.Spec.Trainer.MinInstance < s.Spec.Trainer.MaxInstance | ||
} | ||
|
||
// GPU convert Resource Limit Quantity to int | ||
func (s *TrainingJob) GPU() int { | ||
q := s.Spec.Trainer.Resources.Limits.NvidiaGPU() | ||
gpu, ok := q.AsInt64() | ||
if !ok { | ||
// FIXME: treat errors | ||
gpu = 0 | ||
} | ||
return int(gpu) | ||
} | ||
|
||
// NeedGPU returns true if the job need GPU resource to run. | ||
func (s *TrainingJob) NeedGPU() bool { | ||
return s.GPU() > 0 | ||
} | ||
|
||
func (s *TrainingJob) String() string { | ||
b, _ := json.MarshalIndent(s, "", " ") | ||
return fmt.Sprintf("%s", b) | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -55,6 +55,7 @@ type TrainingJobSpec struct { | |
Passes int `json:"passes,omitempty"` | ||
Volumes []corev1.Volume `json:"volumes"` | ||
VolumeMounts []corev1.VolumeMount `json:"VolumeMounts"` | ||
//TODO simplify the structure of sub-resource(mengyang) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. // TODO(mengyang): Simplify the structure of sub-resource. |
||
//TrainingJob components. | ||
Master MasterSpec `json:"master"` | ||
Pserver PserverSpec `json:"pserver"` | ||
|
@@ -111,12 +112,12 @@ type TrainerJobScaleStatus struct { | |
type TrainingResourceType string | ||
|
||
const ( | ||
// MASTER is the master name of TrainingResourceType. | ||
MASTER TrainingResourceType = "MASTER" | ||
// PSERVER is the pserver name of TrainingResourceType. | ||
PSERVER TrainingResourceType = "PSERVER" | ||
// TRAINER is the trainer name of TrainingResourceType. | ||
TRAINER TrainingResourceType = "TRAINER" | ||
// Master is the master name of TrainingResourceType. | ||
Master TrainingResourceType = "MASTER" | ||
// Pserver is the pserver name of TrainingResourceType. | ||
Pserver TrainingResourceType = "PSERVER" | ||
// Trainer is the trainer name of TrainingResourceType. | ||
Trainer TrainingResourceType = "TRAINER" | ||
) | ||
|
||
// ResourceState is the state of a type of resource | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The package name here should be
instead of
package v1
Please take this file from the Go client library of Google API as a reference.