Improve docs for MLflow Project execution on Kubernetes #1566

ghost · 2019-07-10T22:42:49Z

What changes are proposed in this pull request?

This PR improves the documentation for Kubernetes-backed execution of MLflow Projects. It also depends on #1574; the diff will shrink after #1574 is merged.

How is this patch tested?

Manual generation of docs

Release Notes

This is part of the Kubernetes projects feature and should be included under the same release notes item!

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

What component(s) does this PR affect?

How should the PR be classified in the release notes? Choose one:

rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

ghost · 2019-07-12T22:00:50Z

docs/source/projects.rst

+
+You can run your MLflow Project on Kubernetes by following these steps:
+
+1. First, add a Docker environment to your MLflow Project, if one does not already exist. For


@dbczumar Could you remove the First, Then, and Finally from the steps 1, 2, and 4. They are really not needed since you are using an ordered list.

Done! Thanks

ghost · 2019-07-12T22:02:12Z

docs/source/projects.rst

-with the ``kubernetes_backend.json`` and ``kubernetes_job_template.yaml`` files.
-
+Note that the ``container.name``, ``container.image``, and ``container.command`` fields are only
+replaced for the *first* container defined in the Job Spec. All subsequent container definitions


Note that the container.name -> The container.name

ghost · 2019-07-12T22:04:34Z

docs/source/projects.rst

-
-Have in mind that mlflow overwrites the folowing attributes in the job yaml file so it can handle jobs creation and monitor its status:
+Run a Project on Kubernetes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^


Run a Project -> Run an MLflow Project

Fixed here and for Databricks (Run an MLflow Project on Databricks)

ghost · 2019-07-12T22:05:47Z

docs/source/projects.rst

@@ -349,103 +349,170 @@ useful if you quickly want to test a project in your existing shell environment.

 .. _databricks_execution:

-Run a project on Databricks
+Run a Project on Databricks
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^


Run a Project -> Run an MLflow Project

ghost · 2019-07-12T22:07:16Z

docs/source/projects.rst

 ^^^^^^^^^^^^^^^^^^^^^^^^^^^

-Support for running projects remotely on Databricks is in public preview. To use this feature, you must have an enterprise Databricks account (Community Edition is not supported) and you must have set up the `Databricks CLI <https://github.com/databricks/databricks-cli>`_. Find more detailed instructions in the Databricks docs (`Azure Databricks <https://docs.databricks.com/applications/mlflow/index.html>`_, `Databricks on AWS <https://docs.databricks.com/applications/mlflow/index.html>`_). A brief overview of how to use the feature is as follows:
+You can run MLflow projects remotely on Databricks. To use this feature, you must have an enterprise


We should be consistent how we spell MLflow Projects. At the top of this page it says:

At the core, MLflow Projects

So I think it should be spelled that way throughout.

Fixed! Thanks for catching this

andrewmchen · 2019-07-12T23:24:10Z

docs/source/projects.rst

-          command: ["cmd"]
+        - name: "{replaced with MLflow Project name}"
+          image: "{replaced with URI of Docker image created during Project execution}"
+          command: [{replaced with MLflow Project entry point command}]


Does this example work out of the box? I think this needs to be a string right?

Actually can you make https://github.com/mlflow/mlflow/blob/master/examples/docker/kubernetes_job_template.yaml match this then?

It does, but I've changed it to a string with braces based on our discussion. I also updated the template in examples/docker

andrewmchen · 2019-07-15T23:56:53Z

LGTM as well.

* Edit Kub proj docs. * Fix command. * Doc progress * Attributes * Image name > repo uri * Image name > repo uri * Tests fix * Lint * Fix tests * Job template docs * Project docs * Test fix * Add missing word 'launch' * Address review comments

Edit Kub proj docs.

7f4b96c

ghost requested a review from dbczumar July 10, 2019 22:43

Stephanie Bodoff and others added 15 commits July 10, 2019 15:44

Fix command.

72d0274

Doc progress

152ef63

Attributes

c2aa305

Image name > repo uri

f3bda18

Image name > repo uri

9de14e3

Tests fix

5bbbf5d

Lint

7786880

New sections

932b91d

Fix tests

fcb4993

Merge branch 'k8s_img_to_repo' into kub-proj-docs

ff07d20

Job template docs

0381ab7

Merge remote-tracking branch 'origin/master' into kub-proj-docs

3a8295f

Project docs

96ed800

Test fix

d64d2a3

Merge branch 'k8s_img_to_repo' into kub-proj-docs

292e55f

dbczumar changed the title ~~Edit Kub proj docs~~ Improve docs for MLflow Project execution on Kubernetes Jul 12, 2019

dbczumar requested review from tomasatdatabricks and andrewmchen July 12, 2019 21:21

Add missing word 'launch'

2dbcb67

ghost commented Jul 12, 2019

View reviewed changes

andrewmchen reviewed Jul 12, 2019

View reviewed changes

dbczumar added 2 commits July 12, 2019 17:12

Address review comments

d687d39

Merge master

19c93b8

dbczumar merged commit f2d13fc into mlflow:master Jul 16, 2019

ghost deleted the kub-proj-docs branch November 12, 2019 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve docs for MLflow Project execution on Kubernetes #1566

Improve docs for MLflow Project execution on Kubernetes #1566

ghost commented Jul 10, 2019 •

edited by dbczumar

Loading

ghost Jul 12, 2019

dbczumar Jul 13, 2019

ghost Jul 12, 2019

dbczumar Jul 13, 2019

ghost Jul 12, 2019

dbczumar Jul 13, 2019

ghost Jul 12, 2019

ghost Jul 12, 2019 •

edited by ghost

Loading

dbczumar Jul 13, 2019

andrewmchen Jul 12, 2019

andrewmchen Jul 12, 2019

dbczumar Jul 13, 2019

andrewmchen commented Jul 15, 2019


		You can run your MLflow Project on Kubernetes by following these steps:

		1. First, add a Docker environment to your MLflow Project, if one does not already exist. For

Improve docs for MLflow Project execution on Kubernetes #1566

Improve docs for MLflow Project execution on Kubernetes #1566

Conversation

ghost commented Jul 10, 2019 • edited by dbczumar Loading

What changes are proposed in this pull request?

How is this patch tested?

Release Notes

Is this a user-facing change?

What component(s) does this PR affect?

How should the PR be classified in the release notes? Choose one:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost Jul 12, 2019 • edited by ghost Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewmchen commented Jul 15, 2019

ghost commented Jul 10, 2019 •

edited by dbczumar

Loading

ghost Jul 12, 2019 •

edited by ghost

Loading