Skip to content

Commit

Permalink
Merge pull request #261 from datmo/readme-consolidation
Browse files Browse the repository at this point in the history
Readme consolidation changes
  • Loading branch information
shabazpatel committed Aug 29, 2018
2 parents 8385551 + 37f2a47 commit 783dbb9
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 42 deletions.
47 changes: 5 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

# Datmo Alpha Release

**Datmo** is an open source model tracking and reproducibility tool for developers. Use `datmo init` to turn any repository into a trackable task record with reusable environments and metrics logging.
**Datmo** is an open source model tracking and reproducibility tool for developers. Use `datmo init` to turn any repository into a trackable experiment record with reusable environments and metrics logging.


**Note**: The current version of Datmo is an alpha release. This means commands are subject to change. If you find any bugs please
Expand Down Expand Up @@ -48,8 +48,8 @@ feel free contribute by adding issues so the contributors can address them.
* [Hello World](#hello-world)
* [Examples](#examples)
* [Documentation](#documentation)
* [Transform a Current Project](#transform)
* [Sharing](#sharing)
* [Transform a Current Project](#transform-a-current-project)
* [Sharing](#sharing-workaround)
* [Contributing to Datmo](/CONTRIBUTING.md)

## Installation
Expand Down Expand Up @@ -210,32 +210,8 @@ In order to run the above code you can do the following.
### Project Structure
When running `datmo init`, Datmo adds a hidden `.datmo` directory which keeps track of all of the various entities at play. This is ncessary to render a repository datmo-enabled.

### Snapshots

<p align="center">
The fundamental unit of record in the Datmo ecosystem is a <b>Snapshot</b>, which contains 5 first-class components.
<br><br>
<img size="250px" src="https://github.com/datmo/datmo/blob/master/images/snapshot-badge-readme.png">
</p>


#### Code
Source code should be managed with current source control management tools. Datmo currently is built on top of git, but could theoretically be ported to work with any similar SCM protocol. While datmo will track all of your local changes and experiments on your machine, you will still need to push changes to a remote repository for them to be continually synced with a manager of choice (like GitHub).

For sharing Datmo entities directly with others (beta), see [this section](#sharing-beta) of the README below.

#### Environment
Dependencies should be encoded using standard best practices for your source code. Python packages should be enumerated in a `requirements.txt` file, while system level dependencies (typically found during GPU workflows) should be written into a `Dockerfile`.

#### Configuration
Variables used in your experiment that are necessary for reproducibility. These typically include algorithm hyperparameter values, train/test data split, etc.

#### Files
Large files that cannot be stored in source code (ie: untrackable in git due to size) should be stored separately. For data sources that are not discretizable into files (or are stored elsewhere), it is advised to write out the location/directory of these data sources/files as an entry in the `stats` property.

#### Stats
Model metrics are written to the `stats` property of a snapshot. Datmo does not enforce any type of formal metric definition, the user is free to pass any key-value dictionary during snapshot creation. This enables users to abide by their own metric logging convention while having the flexibility of being able to natively compare metrics across algorithms or frameworks.

### Environments, Snapshots, and Runs
See our [concepts page](https://datmo.readthedocs.io/en/latest/concepts.html) in the documentation to see how the moving parts work together in datmo.

## Documentation
The full docs are hosted [here](https://datmo.readthedocs.io/en/latest/index.html). If you wish to contribute to the docs (source code located here in `/docs`), follow the procedure outlined in `CONTRIBUTING.md`.
Expand All @@ -250,19 +226,6 @@ or you can run the following command
```
$ datmo cleanup
```
### Optional: Mark your GitHub repository as a Datmo project
Once you initialize your project, you can denote your repository as a datmo project by adding the following badge to your README file.
This helps someone pulling the code to know how to setup and run Datmo commands, as the badge will link them to usage instructions here.

#### Markdown
```markdown
[![Datmo Model](https://github.com/datmo/datmo/blob/master/images/badge.svg)](https://github.com/datmo/datmo)
```
#### ReStructuredText
```
.. image:: https://github.com/datmo/datmo/blob/master/images/badge.svg
:target: https://github.com/datmo/datmo
```

## Sharing (Workaround)
**DISCLAIMER:** This is not currently an officially supported option and only works for
Expand Down
6 changes: 6 additions & 0 deletions docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ A **run** is comprised of *tasks* and *snapshots*. In Datmo's paradigm, states (

Each run contains the initial state (snapshot), followed by the action that was performed to it (task), as well as the final state of the repository (another snapshot).

For a typical use case, this would appear as follows:

.. image:: img/run-diagram.png
:width: 800
:alt: Diagram explaining recording of state during an experiment run

You can view all of your past runs at any time with ``$ datmo run ls``

You can replicate a run at any time with the ``$ datmo rerun`` command.
Expand Down
Binary file added docs/img/run-diagram.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 783dbb9

Please sign in to comment.