Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc refined #122

Merged
merged 4 commits into from Sep 30, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/conf.py
Expand Up @@ -39,7 +39,7 @@
"sphinx.ext.coverage",
"sphinx.ext.napoleon",
"sphinx.ext.viewcode",
"sphinx_markdown_tables"
"sphinx_markdown_tables",
]

napoleon_google_docstring = True
Expand Down
2 changes: 1 addition & 1 deletion docs/source/index.rst
Expand Up @@ -10,7 +10,7 @@ Multi-Agent Resource Optimization (MARO) platform is an instance of Reinforcemen
learning as a Service (RaaS) for real-world resource optimization. It can be
applied to many important industrial domains, such as container inventory
management in logistics, bike repositioning in transportation, virtual machine
provisioning in data centers, and asset management in finance. Besides
provisioning in data centers, and asset management in finance. Besides
`Reinforcement Learning <https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf>`_ (RL), it
also supports other planning/decision mechanisms, such as
`Operations Research <https://en.wikipedia.org/wiki/Operations_research>`_.
Expand Down
88 changes: 38 additions & 50 deletions docs/source/installation/grass_cluster_provisioning_on_azure.rst
Expand Up @@ -9,111 +9,99 @@ on Azure and run your training job in a distributed environment.
Prerequisites
-------------


* `Install the Azure CLI and login <https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest>`_
* `Install docker <https://docs.docker.com/engine/install/>`_ and
`Configure docker to make sure it can be managed as a non-root user <https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user>`_

Cluster Management
------------------


* Create a cluster with a `deployment <#grass-azure-create>`_

.. code-block:: sh

# Create a grass cluster with a grass-create deployment
maro grass create ./grass-azure-create.yml
.. code-block:: sh

# Create a grass cluster with a grass-create deployment
maro grass create ./grass-azure-create.yml

* Scale the cluster

.. code-block:: sh
.. code-block:: sh

# Scale nodes with 'Standard_D4s_v3' specification to 2
maro grass node scale my_grass_cluster Standard_D4s_v3 2

Check `VM Size <https://docs.microsoft.com/en-us/azure/virtual-machines/sizes>`_
to see more node specifications.
# Scale nodes with 'Standard_D4s_v3' specification to 2
maro grass node scale my_grass_cluster Standard_D4s_v3 2

Check `VM Size <https://docs.microsoft.com/en-us/azure/virtual-machines/sizes>`_
to see more node specifications.

* Delete the cluster

.. code-block:: sh

# Delete a grass cluster
maro grass delete my_grass_cluster
.. code-block:: sh

# Delete a grass cluster
maro grass delete my_grass_cluster

* Start/stop nodes to save costs

.. code-block:: sh
.. code-block:: sh

# Start 2 nodes with 'Standard_D4s_v3' specification
maro grass node start my_grass_cluster Standard_D4s_v3 2
# Start 2 nodes with 'Standard_D4s_v3' specification
maro grass node start my_grass_cluster Standard_D4s_v3 2

# Stop 2 nodes with 'Standard_D4s_v3' specification
maro grass node stop my_grass_cluster Standard_D4s_v3 2
# Stop 2 nodes with 'Standard_D4s_v3' specification
maro grass node stop my_grass_cluster Standard_D4s_v3 2

Run Job
-------


* Push your training image

.. code-block:: sh

# Push image 'my_image' to the cluster
maro grass image push my_grass_cluster --image-name my_image
.. code-block:: sh

# Push image 'my_image' to the cluster
maro grass image push my_grass_cluster --image-name my_image

* Push your training data

.. code-block:: sh

# Push data under './my_training_data' to a relative path '/my_training_data' in the cluster
# You can then assign your mapping location in the start-job deployment
maro grass data push my_grass_cluster ./my_training_data/* /my_training_data
.. code-block:: sh

# Push data under './my_training_data' to a relative path '/my_training_data' in the cluster
# You can then assign your mapping location in the start-job deployment
maro grass data push my_grass_cluster ./my_training_data/* /my_training_data

* Start a training job with a `deployment <#grass-start-job>`_

.. code-block:: sh

# Start a training job with a start-job deployment
maro grass job start my_grass_cluster ./grass-start-job.yml
.. code-block:: sh

# Start a training job with a start-job deployment
maro grass job start my_grass_cluster ./grass-start-job.yml

* Or, schedule batch jobs with a `deployment <#grass-start-schedule>`_

.. code-block:: sh

# Start a training schedule with a start-schedule deployment
maro grass schedule start my_grass_cluster ./grass-start-schedule.yml
.. code-block:: sh

# Start a training schedule with a start-schedule deployment
maro grass schedule start my_grass_cluster ./grass-start-schedule.yml

* Get the logs of the job

.. code-block:: sh

# Get the logs of the job
maro grass job logs my_grass_cluster my_job_1
.. code-block:: sh

# Get the logs of the job
maro grass job logs my_grass_cluster my_job_1

* List the current status of the job

.. code-block:: sh

# List the current status of the job
maro grass job list my_grass_cluster
.. code-block:: sh

# List the current status of the job
maro grass job list my_grass_cluster

* Stop a training job

.. code-block:: sh
.. code-block:: sh

# Stop a training job
maro grass job stop my_job_1
# Stop a training job
maro grass job stop my_job_1

Sample Deployments
------------------
Expand Down
77 changes: 33 additions & 44 deletions docs/source/installation/k8s_cluster_provisioning_on_azure.rst
Expand Up @@ -9,7 +9,6 @@ on Azure and run your training job in a distributed environment.
Prerequisites
-------------


* `Install the Azure CLI and login <https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest>`_
* `Install and set up kubectl <https://kubernetes.io/docs/tasks/tools/install-kubectl/>`_
* `Install docker <https://docs.docker.com/engine/install/>`_ and
Expand All @@ -18,92 +17,82 @@ Prerequisites
Cluster Management
------------------


* Create a cluster with a `deployment <#k8s-azure-create>`_

.. code-block:: sh

# Create a k8s cluster
maro k8s create ./k8s-azure-create.yml
.. code-block:: sh

# Create a k8s cluster
maro k8s create ./k8s-azure-create.yml

* Scale the cluster

.. code-block:: sh
.. code-block:: sh

# Scale nodes with 'Standard_D4s_v3' specification to 2
maro k8s node scale my_k8s_cluster Standard_D4s_v3 2

Check `VM Size <https://docs.microsoft.com/en-us/azure/virtual-machines/sizes>`_
to see more node specifications.
# Scale nodes with 'Standard_D4s_v3' specification to 2
maro k8s node scale my_k8s_cluster Standard_D4s_v3 2

Check `VM Size <https://docs.microsoft.com/en-us/azure/virtual-machines/sizes>`_
to see more node specifications.

* Delete the cluster

.. code-block:: sh
.. code-block:: sh

# Delete a k8s cluster
maro k8s delete my_k8s_cluster
# Delete a k8s cluster
maro k8s delete my_k8s_cluster

Run Job
-------


* Push your training image

.. code-block:: sh

# Push image 'my_image' to the cluster
maro k8s image push my_k8s_cluster --image-name my_image
.. code-block:: sh

# Push image 'my_image' to the cluster
maro k8s image push my_k8s_cluster --image-name my_image

* Push your training data

.. code-block:: sh

# Push data under './my_training_data' to a relative path '/my_training_data' in the cluster
# You can then assign your mapping location in the start-job deployment
maro k8s data push my_k8s_cluster ./my_training_data/* /my_training_data
.. code-block:: sh

# Push data under './my_training_data' to a relative path '/my_training_data' in the cluster
# You can then assign your mapping location in the start-job deployment
maro k8s data push my_k8s_cluster ./my_training_data/* /my_training_data

* Start a training job with a `deployment <#k8s-start-job>`_

.. code-block:: sh

# Start a training job with a start-job deployment
maro k8s job start my_k8s_cluster ./k8s-start-job.yml
.. code-block:: sh

# Start a training job with a start-job deployment
maro k8s job start my_k8s_cluster ./k8s-start-job.yml

* Or, schedule batch jobs with a `deployment <#k8s-start-schedule>`_

.. code-block:: sh

# Start a training schedule with a start-schedule deployment
maro k8s schedule start my_k8s123_cluster ./k8s-start-schedule.yml
.. code-block:: sh

# Start a training schedule with a start-schedule deployment
maro k8s schedule start my_k8s123_cluster ./k8s-start-schedule.yml

* Get the logs of the job

.. code-block:: sh

# Logs will be exported to current directory
maro k8s job logs my_k8s_cluster my_job_1
.. code-block:: sh

# Logs will be exported to current directory
maro k8s job logs my_k8s_cluster my_job_1

* List the current status of the job

.. code-block:: sh

# List current status of jobs
maro k8s job list my_k8s_cluster my_job_1
.. code-block:: sh

# List current status of jobs
maro k8s job list my_k8s_cluster my_job_1

* Stop a training job

.. code-block:: sh
.. code-block:: sh

# Stop a training job
maro k8s job stop my_k8s_cluster my_job_1
# Stop a training job
maro k8s job stop my_k8s_cluster my_job_1

Sample Deployments
------------------
Expand Down
56 changes: 21 additions & 35 deletions docs/source/installation/pip_install.rst
Expand Up @@ -5,76 +5,62 @@ Package
Install MARO from `PyPI <https://pypi.org/project/pymaro/#files>`_
----------------------------------------------------------------------


*
Max OS / Linux
* Max OS / Linux

.. code-block:: sh

pip install pymaro
pip install pymaro

*
Windows
* Windows

.. code-block:: powershell
.. code-block::

# Install torch first, if you don't have one.
pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
# Install torch first, if you don't have one.
pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install pymaro
pip install pymaro

Install MARO from Source (\ `Editable Mode <https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs>`_\ )
------------------------------------------------------------------------------------------------------------------------


*
Prerequisites

* Prerequisites

* `Python >= 3.6, < 3.8 <https://www.python.org/downloads/>`_
* C++ Compiler

* Linux or Mac OS X: ``gcc``
* Windows: `Build Tools for Visual Studio 2017 <https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15>`_

*
Enable Virtual Environment

* Enable Virtual Environment

*
Mac OS / Linux
* Mac OS / Linux

.. code-block:: sh

# If your environment is not clean, create a virtual environment firstly.
python -m venv maro_venv
source ./maro_venv/bin/activate

*
Windows
* Windows

.. code-block:: powershell

# If your environment is not clean, create a virtual environment firstly.
python -m venv maro_venv
.\maro_venv\Scripts\activate

*
Install MARO
# If your environment is not clean, create a virtual environment firstly.
python -m venv maro_venv
.\maro_venv\Scripts\activate

* Install MARO

*
Mac OS / Linux
* Mac OS / Linux

.. code-block:: sh

# Install MARO from source.
bash scripts/install_maro.sh
# Install MARO from source.
bash scripts/install_maro.sh

*
Windows
* Windows

.. code-block:: powershell

# Install MARO from source.
.\scripts\install_maro.bat
# Install MARO from source.
.\scripts\install_maro.bat