train model + fixes

deephdc · Jan 9, 2019 · 9063bfd · 9063bfd
2 parents c40e871 + 37461eb
commit 9063bfd
Show file tree

Hide file tree

Showing 5 changed files with 83 additions and 60 deletions.
diff --git a/source/user/develop-model.rst b/source/user/develop-model.rst
@@ -0,0 +1,75 @@
+Develop a model using DEEP UC template
+======================================
+
+Prepare DEEP UC environment
+---------------------------
+
+Install cookiecutter (if not yet done)
+::
+	$ pip install cookiecutter
+	
+Run the DEEP UC cookiecutter template
+::
+	$ cookiecutter https://github.com/indigo-dc/cookiecutter-data-science
+	
+Answer all questions from DEEP UC cookiecutter template with attentions to 
+``repo_name`` i.e. the name of your github repositories, etc.
+
+This creates two project directories:
+::
+	~/DEEP-OC-your_project
+	~/your_project
+	
+Go to ``github.com/your_account`` and 
+create corresponding repositories: ``DEEP-OC-your_project`` and ``your_project``
+
+Do ``git push origin master`` in both created directories. This puts your initial code to ``github``.
+
+
+Develop a model according to DEEP UC template
+---------------------------------------------
+
+The structure of ``your_project`` created using 
+`DEEP UC template <https://github.com/indigo-dc/cookiecutter-data-science>`__ contains 
+the following core items needed to develop a model
+::
+	requirements.txt
+	data/
+	models/
+	{{cookiecutter.repo_name}}/dataset/make_dataset.py
+	{{cookiecutter.repo_name}}/features/build_features.py
+	{{cookiecutter.repo_name}}/models/model.py
+	
+**Installing development requirements**
+
+Modify ``requirements.txt`` according to your needs (e.g. add more libraries) then run
+::
+	$ pip install -r requirements.txt
+	
+
+**Improve the initial code**
+
+You can modify as well as add more source files and put them accordingly into the directory structure.
+
+**1. Make datasets:** source files in this directory aim to manipulate with raw dataset(s). 
+The output of this step is raw data, which can be cleaned and/or pre-formatted.
+::
+	{{cookiecutter.repo_name}}/dataset/make_dataset.py
+	{{cookiecutter.repo_name}}/dataset/
+
+**2. Build features** takes the output from the previous step (Make datasets) and 
+creates ML train, test as well as validation data from raw data.
+The concrete realisation is depend on concrete UC, the aim of the application as well as 
+technological background (e.g. high-performance supports).
+::
+	{{cookiecutter.repo_name}}/features/build_features.py
+	{{cookiecutter.repo_name}}/features/
+
+**3. Develop models** dealing with the most interesting ML core i.e. modelling. 
+The most important thing in the ``model.py`` are implementations for DEEP entry points, 
+which are defined according to :ref:`API methods <user/overview/api:Methods>`. 
+You don't need to implement all the methods, just the ones you need.
+::
+	{{cookiecutter.repo_name}}/models/model.py
+	{{cookiecutter.repo_name}}/models/
+
diff --git a/source/user/index.rst b/source/user/index.rst
@@ -12,6 +12,8 @@ If you want a quickstart guide, please check the following link.
    :maxdepth: 1
 
    quickstart
+   try-model-locally
+   develop-model
 
 Overview
 ---------

diff --git a/source/user/overview/cookiecutter-develop-model.rst b/source/user/overview/cookiecutter-develop-model.rst
diff --git a/source/user/overview/index.rst b/source/user/overview/index.rst
@@ -8,7 +8,7 @@ Overview
    architecture
    user-roles
    cookiecutter
-   cookiecutter-develop-model
+   develop-model
    train-model-locally
    try-model-locally
    api
diff --git a/source/user/overview/try-model-locally.rst → source/user/try-model-locally.rst b/source/user/overview/try-model-locally.rst → source/user/try-model-locally.rst
@@ -5,18 +5,21 @@ Try a model locally
 
 
 2. **Search for a model in the marketplace**
-TBD 
+
 
 3. **Get the model**
+
 clone the repo
+
 build the container or get from DockerHub
 
 
 4. **Run the model**
 
 
 5. **Go to the API, get the results**
-MODS
+
+MODS - probs as outpus
 
 Need reviewing/improvements for other UCs