Skip to content
This repository has been archived by the owner on Jun 14, 2023. It is now read-only.

Commit

Permalink
fix minor format typos
Browse files Browse the repository at this point in the history
  • Loading branch information
ignacio committed Dec 17, 2019
1 parent 6607dc6 commit 186dcfb
Show file tree
Hide file tree
Showing 15 changed files with 246 additions and 212 deletions.
119 changes: 79 additions & 40 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,43 +1,82 @@
# sphinx build folder
_build
build

# Compiled source #
###################
*.com
*.class
*.dll
*.exe
*.o
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# C extensions
*.so

# Packages #
############
# it's better to unpack these files and commit the raw source
# git has its own built in compression methods
*.7z
*.dmg
*.gz
*.iso
*.jar
*.rar
*.tar
*.zip

# Logs and databases #
######################
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover

# Translations
*.mo
*.pot

# Django stuff:
*.log
*.sql
*.sqlite

# OS generated files #
######################
.DS_Store?
ehthumbs.db
Icon?
Thumbs.db

# Editor backup files #
#######################
*~
.*.swp

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# DotEnv configuration
.env

# Database
*.db
*.rdb

# Pycharm
.idea

# VS Code
.vscode/

# Spyder
.spyproject/

# Jupyter NB Checkpoints
.ipynb_checkpoints/

# exclude data from source control by default
#data/

# Mac OS-specific storage files
.DS_Store
35 changes: 18 additions & 17 deletions source/user/howto/develop-model.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,29 @@ Install cookiecutter (if not yet done)
::

$ pip install cookiecutter

Run the DEEP DS cookiecutter template
::

$ cookiecutter https://github.com/indigo-dc/cookiecutter-data-science

Answer all questions from DEEP DS cookiecutter template with attentions to
``repo_name`` i.e. the name of your github repositories, etc.
This creates two project directories:
::

~/DEEP-OC-your_project
~/your_project
Go to ``github.com/your_account`` and

Go to ``github.com/your_account`` and
create corresponding repositories: ``DEEP-OC-your_project`` and ``your_project``
Do ``git push origin master`` in both created directories. This puts your initial code to ``github``.


2. Improve the initial code of the model
----------------------------------------

The structure of ``your_project`` created using
The structure of ``your_project`` created using
`DEEP DS template <https://github.com/indigo-dc/cookiecutter-data-science>`_ contains
the following core items needed to develop a DEEP DS model:
::
Expand All @@ -49,17 +49,18 @@ the following core items needed to develop a DEEP DS model:


2.1 Installing development requirements
=======================================

Modify ``requirements.txt`` according to your needs (e.g. add more libraries) then run
::

$ pip install -r requirements.txt
You can modify and add more ``source files`` and put them

You can modify and add more ``source files`` and put them
accordingly into the directory structure.


2.2 Make datasets
2.2 Make datasets
=================

Source files in this directory aim to manipulate raw datasets.
Expand All @@ -86,10 +87,10 @@ available technological backgrounds (e.g. high-performance supports for data pro
2.4 Develop models
==================

This step deals with the most interesting phase in ML i.e. modelling.
This step deals with the most interesting phase in ML i.e. modelling.
The most important thing of DEEP DS models is located in ``model.py``
containing DEEP entry point implementations.
DEEP entry points are defined using :ref:`API methods <user/overview/api:Methods>`.
containing DEEP entry point implementations.
DEEP entry points are defined using :ref:`API methods <user/overview/api:Methods>`.
You don't need to implement all of them, just the ones you need.
::

Expand All @@ -110,15 +111,15 @@ To create a python installable package the initial directory structure should lo
LICENSE
README

* The top level directory will be the root of your repo, e.g. your_model_package.git. The subdir, also called your_model_package, is the actual python module.
* The top level directory will be the root of your repo, e.g. your_model_package.git. The subdir, also called your_model_package, is the actual python module.
* ``setup.py`` is the build script for setuptools. It tells setuptools about your package (such as the name and version) as well as which code files to include. You can find an example of a setup.py file `here <https://github.com/deephdc/image-classification-tf/blob/master/setup.py>`__. For the official documentation on how to write your setup script, you can go `here <https://docs.python.org/2/distutils/setupscript.html>`__.
* ``setup.cfg`` can be used to get some information from the user, or from the user's system in order to proceed. Configuration files also let you providedefault values for any command option. An example of a setup.cfg file can be found `here <https://github.com/deephdc/image-classification-tf/blob/master/setup.cfg>`__. The official python documentation on the setup configuration file can be found `here <https://docs.python.org/3/distutils/configfile.html>`__.
* ``requirements.txt`` contains any external requirement needed to run the package. You can see an example of a requirements file `here <https://github.com/deephdc/image-classification-tf/blob/master/requirements.txt>`__. An example of a requirements file can be found `here <https://github.com/deephdc/image-classification-tf/blob/master/requirements.txt>`_.
* The ``README`` file will contain information on how to run the package or anything else that you may find useful for someone running your package.
* ``LICENSE`` It’s important for every package uploaded to the Python Package Index to include a license. This tells users who install your package the terms under which they can use your package. For help choosing a license, go `here <https://choosealicense.com/>`__.

To see how to install your model package, check the Dockerfile in the next section.

4. Create a docker container for your model
===========================================

Expand All @@ -129,7 +130,7 @@ The simplest Dockerfile could look like this::
FROM ubuntu:18.04

WORKDIR /srv

#Download and install your model package
RUN git clone https://github.com/your_git/your_model_package && \
cd image-classification-tf && \
Expand All @@ -152,15 +153,15 @@ The simplest Dockerfile could look like this::
# Expose API on port 5000 and tensorboard on port 6006
EXPOSE 5000 6006

CMD deepaas-run --listen-ip 0.0.0.0
CMD deepaas-run --listen-ip 0.0.0.0


For more details on rclone or on DEEPaas API you can check :doc:`here <rclone>` and `here <https://github.com/indigo-dc/DEEPaaS>`__ respectively.

If you want to see an example of a more complex Dockerfile, you can check it `here <https://github.com/indigo-dc/DEEP-OC-image-classification-tf/blob/master/Dockerfile>`__.
If you want to see an example of a more complex Dockerfile, you can check it `here <https://github.com/deephdc/DEEP-OC-image-classification-tf/blob/master/Dockerfile>`__.

In order to compile the Dockerfile, you should choose a name for the container and use the docker build command::

docker build -t your_container_name -f Dockerfile


Expand Down
1 change: 0 additions & 1 deletion source/user/howto/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,3 @@ HowTo's
Use rclone <rclone>
Install and configure oidc-agent <oidc-agent>
Video demos <video-demos>

6 changes: 3 additions & 3 deletions source/user/howto/oidc-agent.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ oidc-agent is a tool to manage OpenID Connect tokens and make them easily usable

$ oidc-gen

You will be asked for the name of the account to configure. Let's call it **deep-iam**.
You will be asked for the name of the account to configure. Let's call it **deep-iam**.
After that you will be asked for the additional client-name-identifier, you should choose the option::

[2] https://iam.deep-hybrid-datacloud.eu/
Expand All @@ -36,8 +36,8 @@ Then just click Enter to accept the default values for Space delimited list of s
Registering Client ...
Generating account configuration ...
accepted
* At this point you will be given a URL. You should visit it in the browser of your choice in order to continue and approve the registered client.

* At this point you will be given a URL. You should visit it in the browser of your choice in order to continue and approve the registered client.
* For this you will have to login into your DEEP-IAM account and accept the permissions you are asked for.

* Once you have done this you will see the following message::
Expand Down

0 comments on commit 186dcfb

Please sign in to comment.