Skip to content
MODS - Massive Online Data Streams
Python Shell Jupyter Notebook
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
checkpoints
data initial Jun 12, 2019
docker initial commit Sep 19, 2018
docs initial commit Sep 19, 2018
models
mods
notebooks initial commit Sep 19, 2018
references
reports preparation Aug 3, 2019
.dockerignore
.gitignore
.rclone.conf
Jenkinsfile
LICENSE setup Oct 12, 2018
README.md
requirements.txt
setup.cfg training done Feb 1, 2019
setup.py initial commit Sep 19, 2018
test-requirements.txt Create test-requirements.txt Dec 18, 2018
test_environment.py
tox.ini

README.md

DEEP Open Catalogue: Massive Online Data Streams (MODS)

DEEP-Hybrid-DataCloud logo

Build Status

DEEP Open Catalog entry: DEEP Open Catalog

Project: This work is part of the DEEP Hybrid-DataCloud project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.

To start using this framework run:

git clone https://github.com/deephdc/mods
cd mods
pip install -e .

Requirements:

  • This project has been tested in Ubuntu 18.04 with Python 3.6. Further package requirements are described in the requirements.txt file.
  • (TBD later)

Project Organization

├── LICENSE
├── README.md          <- The top-level README for developers using this project.
├── data
│   └── raw            <- The original, immutable data dump.
│
├── docs               <- A default Sphinx project; see sphinx-doc.org for details
│
├── docker             <- Directory for Dockerfile(s)
│
├── models             <- Trained and serialized models, model predictions, or model summaries
│
├── notebooks          <- Jupyter notebooks. Naming convention is a number (for ordering),
│                         the creator's initials (if many user development),
│                         and a short `_` delimited description, e.g.
│                         `1.0-jqp-initial_data_exploration.ipynb`.
│
├── references         <- Data dictionaries, manuals, and all other explanatory materials.
│
├── reports            <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures        <- Generated graphics and figures to be used in reporting
│
├── requirements.txt   <- The requirements file for reproducing the analysis environment, e.g.
│                         generated with `pip freeze > requirements.txt`
│
├── setup.py           <- makes project pip installable (pip install -e .) so mods can be imported
├── mods    <- Source code for use in this project.
│   ├── __init__.py    <- Makes mods a Python module
│   │
│   ├── dataset        <- Scripts to download or generate data
│   │   └── make_dataset.py
│   │
│   ├── features       <- Scripts to turn raw data into features for modeling
│   │   └── build_features.py
│   │
│   ├── models         <- Scripts to train models and then use trained models to make
│   │   │                 predictions
│   │   └── model.py
│   │
│   └── tests          <- Scripts to perfrom code testing + pylint script
│   │
│   └── visualization  <- Scripts to create exploratory and results oriented visualizations
│       └── visualize.py
│
└── tox.ini            <- tox file with settings for running tox; see tox.testrun.org

Project based on the DEEP DS template. #cookiecutter #datascience

Workflow

1. Data preprocessing

1.1 Prepare the dataset

1.2 Build features

2. Train and test NNs

2.1 Set the configuration

2.1 Training

3. Prediction throught DEEPaaS API

4. DEEP as a Service: MODS container

5. Docker Hub: MODS container image in Docker Hub deephdc organization

You can’t perform that action at this time.