Skip to content

Commit

Permalink
Merge pull request #15 from whisk-ml/docs_index_refactor
Browse files Browse the repository at this point in the history
Refactor docs structure and update project readme
  • Loading branch information
itsderek23 committed May 15, 2020
2 parents 365c9b0 + 4350d8e commit cbcc06a
Show file tree
Hide file tree
Showing 3 changed files with 132 additions and 180 deletions.
104 changes: 9 additions & 95 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,9 @@ https://docs.python-guide.org/writing/structure/). Whisk does the structuring wh

Read more about our [beliefs](#beliefs).

## A quick tour of whisk
## Getting Started

The best way to experience whisk is by creating your first project. The short self-guided tour below shows the power of whisk.

_We use __demo__ as the project name in the examples below. If you use a different project name, be sure to replace __demo__ with the name of your project._

### Create a project

Start by creating a project. Begin a terminal session and run the commands below:
Start by creating a project. Begin a terminal session and run the commands below. _Note: We use __demo__ as the project name in the examples below. If you use a different project name, be sure to replace __demo__ with the name of your project._

```
$ pip install whisk
Expand All @@ -31,103 +25,23 @@ The commands above do the following:
* [Generate the project directory structure](project_structure.html)
* Activate the project's venv

### Try the sample model

The project contains a [naive model stub](autoapi/whisk/model_stub/index.html) that returns the length of each inner list within a 2D array. The model is saved to the disk using pickle. We can use the model stub to demonstrate the full capabilities of a whisk project.

Since a whisk project uses a standard Python project code structure, there are multiple ways to invoke the model.

#### From Python code

The `Model` class resides in the `src/demo/models/model.py` file. The source code is installed as an editable Python package, so you can run the model from the Python console or from within a notebook cell:

```py
from demo.models.model import Model
model = Model()
model.predict([[0,1],[2,3]])
```

The `Model` class loads the model from disk and `Model.predict` passes the data into the model. Try making a change to `src/demo/models/model.py` to see how model inference works.

#### From the command line

The project is pre-wired with [Click](https://click.palletsprojects.com/en/7.x/), a Python package for creating beautiful command line interfaces. Invoke the model right from the terminal:
__To try out all of the features, continue the [quick tour of whisk →](tour_of_whisk.html#try-the-sample-model).__

```
$ demo predict [[0,1],[2,3]]
[2, 2]
```

The CLI code is in `src/demo/cli/main.py`.

#### From the Flask App

There's a ready-to-go Flask web service in the `app/main.py` file. Start the web service:

```
$ whisk app start
```

Make a request in another terminal session:

```
$ curl --location --request POST 'http://localhost:5000/predict' \
--header 'Content-Type: application/json' \
--data-raw '{"data":[[0, 1], [2, 3]]}'
```

### Sharing the model

You can let others use your model by releasing a Python package and deploying a web service.

#### Release a Python Package for your model

Create a Python package containing your model. This lets anyone use your model that has Python installed on their computer:

```
$ whisk package dist
Python Package created in /Users/dlite/projects/whisk_examples/demo/dist:
demo-0.1.0.tar.gz
$ pip install dist/demo-0.1.0.tar.gz
```

After installing the package, another user can invoke the model via the CLI or by importing the `demo` module like we did earlier. You can modify the package metadata (author, description, etc) and list the package dependencies in `setup.py`.

#### Deploy a web service

Deploy the model inference web service to Heroku ([a free account](https://signup.heroku.com/) works fine with this demo):

```
$ whisk app create demo-[INSERT YOUR NAME]
```

### What next?

#### Change the example model

To change the example model, head over to `src/demo/models/model.py` and make a small change to the `predict` method. After saving this, you'll see all whisk functionality automatically update with this new logic.

#### Open the example notebook


The project includes an interactive notebook that shows how to save your trained model to disk, use the saved model to generate predictions, how to load Python functions and classes from the project's `src` directory, and more. It's the guide rails for your own ML project.

```
$ jupyter-notebook notebooks/getting_started.ipynb
```
## Examples

#### Explore a real whisk project
The [whisk-ml](https://github.com/whisk-ml) GitHub org contains example whisk projects. Check out these examples and clone them locally. Since whisk makes reproducibility "just work", in most cases you simply need to run [`whisk setup`](cli_reference.html#whisk-setup) to use the models generated by the projects. Here are few examples to start with:

The [whisk-ml](https://github.com/whisk-ml) GitHub org contains example whisk projects. Checkout these examples and clone them locally. Since whisk makes reproducibility "just work", in most cases you simply need to run [`whisk setup`](cli_reference.html#whisk-setup) to use the models generated by the projects.
* [Iris Classification with SKLearn](#)
* [Text Classification with Pytorch](#)
* [Image Classification with Tensorflow](#)

## Documentation

```eval_rst
.. toctree::
:maxdepth: 1
index
tour_of_whisk
key_concepts
installation
project_structure
Expand Down
117 changes: 117 additions & 0 deletions docs/tour_of_whisk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# A Quick Tour

The best way to experience whisk is by creating your first project. The short self-guided tour below shows the power of whisk.

_We use __demo__ as the project name in the examples below. If you use a different project name, be sure to replace __demo__ with the name of your project._

### Create a project

Start by creating a project. Begin a terminal session and run the commands below:

```
$ pip install whisk
$ whisk create demo
$ cd demo
$ source venv/bin/activate
```

The commands above do the following:

* [Install the whisk package](installation.html)
* [Create a project](cli_reference.html#whisk-create) named "demo"
* [Generate the project directory structure](project_structure.html)
* Activate the project's venv

### Try the sample model

The project contains a [naive model stub](autoapi/whisk/model_stub/index.html) that returns the length of each inner list within a 2D array. The model is saved to the disk using pickle. We can use the model stub to demonstrate the full capabilities of a whisk project.

Since a whisk project uses a standard Python project code structure, there are multiple ways to invoke the model.

#### From Python code

The `Model` class resides in the `src/demo/models/model.py` file. The source code is installed as an editable Python package, so you can run the model from the Python console or from within a notebook cell:

```py
from demo.models.model import Model
model = Model()
model.predict([[0,1],[2,3]])
```

The `Model` class loads the model from disk and `Model.predict` passes the data into the model. Try making a change to `src/demo/models/model.py` to see how model inference works.

#### From the command line

The project is pre-wired with [Click](https://click.palletsprojects.com/en/7.x/), a Python package for creating beautiful command line interfaces. Invoke the model right from the terminal:

```
$ demo predict [[0,1],[2,3]]
[2, 2]
```

The CLI code is in `src/demo/cli/main.py`.

#### From the Flask App

There's a ready-to-go Flask web service in the `app/main.py` file. Start the web service:

```
$ whisk app start
```

Make a request in another terminal session:

```
$ curl --location --request POST 'http://localhost:5000/predict' \
--header 'Content-Type: application/json' \
--data-raw '{"data":[[0, 1], [2, 3]]}'
```

### Sharing the model

You can let others use your model by releasing a Python package and deploying a web service.

#### Release a Python Package for your model

Create a Python package containing your model. This lets anyone use your model that has Python installed on their computer:

```
$ whisk package dist
Python Package created in /Users/dlite/projects/whisk_examples/demo/dist:
demo-0.1.0.tar.gz
$ pip install dist/demo-0.1.0.tar.gz
```

After installing the package, another user can invoke the model via the CLI or by importing the `demo` module like we did earlier. You can modify the package metadata (author, description, etc) and list the package dependencies in `setup.py`.

Learn more about packaging a model in our [packaging guide](packaging.html).

#### Deploy a web service

Deploy the model inference web service to Heroku ([a free account](https://signup.heroku.com/) works fine with this demo):

```
$ whisk app create demo-[INSERT YOUR NAME]
```

Learn more about [deploying a model to Heroku](heroku.html).

### What next?

#### Change the example model

To change the example model, head over to `src/demo/models/model.py` and make a small change to the `predict` method. After saving this, you'll see all whisk functionality automatically update with this new logic.

#### Open the example notebook


The project includes an interactive notebook that shows how to save your trained model to disk, use the saved model to generate predictions, how to load Python functions and classes from the project's `src` directory, and more. It's the guide rails for your own ML project.

```
$ jupyter-notebook notebooks/getting_started.ipynb
```

#### Explore a real whisk project

The [whisk-ml](https://github.com/whisk-ml) GitHub org contains example whisk projects. Checkout these examples and clone them locally. Since whisk makes reproducibility "just work", in most cases you simply need to run [`whisk setup`](cli_reference.html#whisk-setup) to use the models generated by the projects.
91 changes: 6 additions & 85 deletions whisk/template/{{ cookiecutter.repo_name }}/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# {{cookiecutter.project_name}}

This readme was auto-generated using `whisk`. `whisk` creates a logical and flexible project structure for ML with reproducible results and lets you release your model to the world without becoming a software engineer.
This readme was auto-generated using [whisk](https://github.com/whisk-ml/whisk). whisk creates a logical and flexible project structure for ML with reproducible results and lets you release your model to the world without becoming a software engineer.

Once your project is setup, edit this readme directly to add context and documentation for your project.

Expand All @@ -16,94 +16,15 @@ The following is required to run this project:

After cloning this repo and `cd {{cookiecutter.project_name}}`:

1. If you haven't yet installed `whisk`, run `pip install whisk`
1. If you haven't yet installed whisk, run `pip install whisk`
2. Run `whisk setup`. The install script creates a `venv`, installs the Python dependencies specified, and initializes DVC.
3. Activate the venv: `source venv/bin/activate`
4. If DVC is used, Download the latest data files: `dvc pull`.

## Whisk CLI Commands

To see a list of available whisk commands and command groups:

```
whisk --help
```

You can view help on specific command groups like this:

```
whisk app --help
```

## {{cookiecutter.project_name}} CLI Commands

{{cookiecutter.project_name}} also allows for prediction directly in the CLI:

```
{{cookiecutter.project_name}} predict <input_format>
```

## Accessing {{cookiecutter.project_name}} in Python

{{cookiecutter.project_name}} also allows for prediction directly in the CLI:

```
from {{cookiecutter.project_name}}.models.model import Model
model.predict(<input_format>)
```


Project Organization
------------

├── LICENSE
├── README.md <- The top-level README for developers using this project.
├── data
│   ├── external <- Data from third party sources.
│   ├── interim <- Intermediate data that has been transformed.
│   ├── processed <- The final, canonical data sets for modeling.
│   └── raw <- The original, immutable data dump.
├── docs <- A default Sphinx project; see sphinx-doc.org for details
├── notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
│ the creator's initials, and a short `-` delimited description, e.g.
│ `1.0-jqp-initial-data-exploration`.
├── references <- Data dictionaries, manuals, and all other explanatory materials.
├── reports <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures <- Generated graphics and figures to be used in reporting
├── requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
│ generated with `pip freeze > requirements.txt`
├── setup.py <- makes project pip installable (pip install -e .) so src can be imported
├── src/{{cookiecutter.project_name}} <- Source code for use in this project.
│   ├── __init__.py <- Makes src a Python module
│ │
│   ├── artifacts <- Artifacts needed use by the source code
│   │  
│   ├── data <- Scripts to download or generate data
│   │   └── make_dataset.py
│ │
│   ├── features <- Scripts to turn raw data into features for modeling
│   │   └── build_features.py
│ │
│   ├── models <- Scripts to train models and then use trained models to make
│ │ │ predictions
│   │   └── model.py <- Template model class for use with whisk
│ │
│   └── visualization <- Scripts to create exploratory and results oriented visualizations
│   └── visualize.py
├── tox.ini <- tox file with settings for running tox; see tox.testrun.org
└─── whisk_commands
   └── app.py <- Script to add or change whisk commands for your project

To learn more about whisk, here are a few helpful doc pages:
* [A Quick Tour of whisk](https://whisk.readthedocs.io/en/latest/tour_of_whisk.html)
* [Key Concepts](https://whisk.readthedocs.io/en/latest/key_concepts.html)
* [Project Structure](https://whisk.readthedocs.io/en/latest/project_structure.html)

--------

Expand Down

0 comments on commit cbcc06a

Please sign in to comment.