Submit job

[wangkuiyi]

elasticflow --runner=k8s \
   --num_trainer=8 \
   --num_ps=2 \
   --input=/oss/yi/mnist* \
   --output_dir=/oss/yi/experiment \
   MyModel.py \
   --embedding_size=1000

where

• elasticflow is a command-line tool,
• MyModel.py includes a class derived from torch.nn.Module, tf.keras.Model, tf.Estimator, etc.

Similarly, to start a local job

elasticflow --runner=local \
    ....

[wangkuiyi]

class MyModel(torch.nn.Module):
  ...

elasticflow.Run(
   MyModel,
   elasticflow.runner.K8S(api_server`...))

[wangkuiyi]

Nest runner example

elasticflow.runner.AWS(
   elasticflow.runner.K8S(api_server=...),
   aws_credential=...)

elasticflow.runner.AWS(
   ip_list=[.....],
   ...)