Skip to content

Distributed Automated Parameter Testing Initial Release
Compare
Choose a tag to compare
@BenSDuggan BenSDuggan released this 16 Nov 01:13
· 89 commits to master since this release
v0.9.0
0061220

Distributed Automated Parameter Testing

A library to assist with running parameter sets across multiple systems. The goal of this library is to provide a tool set and pipeline that make organizing, running and analyzing a large amount of parameter easier. Some of the highlights include:

  • Provide an easy way to run paramater sets.
  • Protocol for allowing teams to run parameter sets concurrently.
  • Use Google Sheets as a database to host and manage paramater sets.
  • Access to the Box API which allows files to be uploaded to box.

Overview

When working on a project with or without access to high performance computing (HPC), there is often a need to perform large parameter sweeps. Before developing DAPT, there were several problems the ECM team in Dr. Paul Macklin's research lab identified. First, it was difficult to manage a large number of parameter sets with a large number of parameters. Second, it would be nice to use Google Sheets to run the parameters for easier collaboration and management. Third, only one person in the group would be running all the parameters, making their computer useless for the duration of the runs. Finally, we needed to upload the data to Box for permanent storage and to allow the rest of the team to view the data.

DAPT was written to solve these problems. A "database" (CSV or Google Sheet) is used to store a list of parameter sets. This database is managed by the Param class and provides methods to interact with and manage parameter sets. the Box class allows data to be uploaded to Box.com. Sensitive API credentials can be stored in a config file (via the Config class) which can also be accessed by users to get other variables.

Install

The easiest way to install DAPT is using pip. To do so type:

pip install dapt

Alternatively, you can dowload the project. It is recommended to download a release of the project from GitHub for improved stability. If you would like to download the most up to date version, then download the repo or clone it on your machine git clone https://github.com/BenSDuggan/DAPT. Once downloaded navigate to the root of the project (DAPT) and run pip install -r requirements.txt to install all of the dependences. If you use this method of installation, you will need to write all of your Python scripts using DAPT in the root directory of the project. For these reasons, it's recommended to only use this method if would like to contribute to the project.

You can then test to make sure everything installed by starting a python session and then running:

import dapt
dapt.__version__

You should see 0.9.0.

Dependencies

Python: >=3.5

All dependences are located in requirements.txt.

Documentation

You can view the most recent documentation on RTD here. Documentation is performed using Sphinx. The docs folder holds all of the resources to document the code. If you're not familiar with Sphinx you can read this Medium tutorial for an introduction. Google docstrings are used for inline commenting inside each file.

To build docs on your local machine simply type make html inside the docs folder.

Future plans

Future versions of the project will work to improve documentation, add examples, cleanup current functionality and add more features. Some of the features we plan to add are:

  • Improve documentation of the dapt module i
  • Add more examples and document current examples better
  • Add error and information logging
  • Add email and Slack notification
Assets 2