Skip to content

Commit

Permalink
refactor benchmarking to kiosk_client
Browse files Browse the repository at this point in the history
  • Loading branch information
willgraf committed May 29, 2020
1 parent 7161082 commit f2d5b67
Show file tree
Hide file tree
Showing 16 changed files with 49 additions and 49 deletions.
2 changes: 1 addition & 1 deletion .coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ show_missing = True

omit =
**/*_test.py
benchmarking/__main__.py
client/__main__.py
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ install:
- pip install -U pytest pytest-twisted pytest-cov pytest-pep8 coveralls attrs>=19.2.0

script:
- pytest --cov=benchmarking --pep8 benchmarking
- pytest --cov=kiosk_client --pep8 kiosk_client

jobs:
include:
Expand Down
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# kiosk-benchmarking
# kiosk-client

[![Build Status](https://travis-ci.com/vanvalenlab/kiosk-benchmarking.svg?branch=master)](https://travis-ci.com/vanvalenlab/kiosk-benchmarking)
[![Coverage Status](https://coveralls.io/repos/github/vanvalenlab/kiosk-benchmarking/badge.svg?branch=master)](https://coveralls.io/github/vanvalenlab/kiosk-benchmarking?branch=master)
[![Build Status](https://travis-ci.com/vanvalenlab/kiosk-client.svg?branch=master)](https://travis-ci.com/vanvalenlab/kiosk-client)
[![Coverage Status](https://coveralls.io/repos/github/vanvalenlab/kiosk-client/badge.svg?branch=master)](https://coveralls.io/github/vanvalenlab/kiosk-client?branch=master)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](/LICENSE)

`kiosk-benchmarking` is tool for interacting with the [Kiosk](https://github.com/vanvalenlab/kiosk-console) in order to create and monitor deep learning image processing jobs. It uses the asynchronous HTTP client [treq](https://github.com/twisted/treq) and the [Kiosk-Frontend API](https://github.com/vanvalenlab/kiosk-frontend) to create and monitor many jobs at once. Once all jobs are completed, [costs are estimated](./docs/cost_computation_notes.md) by using the cluster's [Grafana API](https://grafana.com/docs/http_api/). An output file is then generated with statistics on each job's performance and resulting output files.
`kiosk-client` is tool for interacting with the [DeepCell Kiosk](https://github.com/vanvalenlab/kiosk-console) in order to create and monitor deep learning image processing jobs. It uses the asynchronous HTTP client [treq](https://github.com/twisted/treq) and the [Kiosk-Frontend API](https://github.com/vanvalenlab/kiosk-frontend) to create and monitor many jobs at once. Once all jobs are completed, [costs are estimated](./docs/cost_computation_notes.md) by using the cluster's [Grafana API](https://grafana.com/docs/http_api/). An output file is then generated with statistics on each job's performance and resulting output files.

This repository is part of the [DeepCell Kiosk](https://github.com/vanvalenlab/kiosk-console). More information about the Kiosk project is available through [Read the Docs](https://deepcell-kiosk.readthedocs.io/en/master) and our [FAQ](http://www.deepcell.org/faq) page.

Expand All @@ -14,10 +14,10 @@ First, clone the git repository and install the required dependencies.

```bash
# clone the repository
git clone https://github.com/vanvalenlab/kiosk-benchmarking.git
git clone https://github.com/vanvalenlab/kiosk-client.git

# move into the new repository directory
cd kiosk-benchmarking
cd kiosk-client

# install the requirements
pip install -r requirements.txt
Expand All @@ -28,17 +28,17 @@ pip install -r requirements.txt
The only thing necessary to use the CLI is the image file to process, the type of job, and the IP address or FQDN of the DeepCell Kiosk.

```bash
# from within the kiosk-benchmarking repository
python benchmarking path/to/image.png \
# from within the kiosk-client repository
python kiosk_client path/to/image.png \
--job-type segmentation \
--host 123.456.789.012
```

It is also possible to override the default model and post-processing function for a given job type.

```bash
# from within the kiosk-benchmarking repository
python benchmarking path/to/image.png \
# from within the kiosk-client repository
python kiosk_client path/to/image.png \
--job-type segmentation \
--host 123.456.789.012 \
--model ModelName:0 \
Expand All @@ -54,8 +54,8 @@ A new job is created every `START_DELAY` seconds up to `COUNT` jobs.
The upload time can be simulated by changing the start delay.

```bash
# from within the kiosk-benchmarking repository
python benchmarking path/to/image.png \
# from within the kiosk-client repository
python kiosk_client path/to/image.png \
--job-type segmentation \
--host 123.456.789.012 \
--model ModelName:0 \
Expand Down
2 changes: 1 addition & 1 deletion bin/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

for (( i=0; i<${NUM_CYCLES}; i++ ))
do
python -m benchmarking \
python -m kiosk_client \
$FILE \
--job-type=$JOB_TYPE \
--host=$API_HOST \
Expand Down
10 changes: 5 additions & 5 deletions benchmarking/__init__.py → kiosk_client/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand All @@ -27,10 +27,10 @@
from __future__ import division
from __future__ import print_function

from benchmarking import cost
from benchmarking import job
from benchmarking import manager
from benchmarking import utils
from kiosk_client import cost
from kiosk_client import job
from kiosk_client import manager
from kiosk_client import utils

del absolute_import
del division
Expand Down
8 changes: 4 additions & 4 deletions benchmarking/__main__.py → kiosk_client/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -36,8 +36,8 @@

from twisted.internet import reactor

from benchmarking import manager
from benchmarking import settings
from kiosk_client import manager
from kiosk_client import settings


def valid_filepath(parser, arg):
Expand All @@ -54,7 +54,7 @@ def valid_filepath(parser, arg):

def get_arg_parser():
parser = argparse.ArgumentParser(
prog='kioskcli',
prog='kioskclient',
description='The Kicsk-CLI is a Command Line Interface (CLI) for '
'interacting with the DeepCell Kiosk.'
)
Expand Down
4 changes: 2 additions & 2 deletions benchmarking/cost.py → kiosk_client/cost.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand All @@ -34,7 +34,7 @@

import requests

from benchmarking import settings
from kiosk_client import settings


# defining Google Cloud prices
Expand Down
4 changes: 2 additions & 2 deletions benchmarking/cost_test.py → kiosk_client/cost_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand All @@ -34,7 +34,7 @@
import pytest
import requests.exceptions

from benchmarking import cost
from kiosk_client import cost


class FakeCreationData:
Expand Down
4 changes: 2 additions & 2 deletions benchmarking/job.py → kiosk_client/job.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -38,7 +38,7 @@
from twisted.internet import error as twisted_errors
from twisted.web import _newclient as twisted_client

from benchmarking.utils import sleep, strip_bucket_prefix
from kiosk_client.utils import sleep, strip_bucket_prefix


class Job(object):
Expand Down
4 changes: 2 additions & 2 deletions benchmarking/job_test.py → kiosk_client/job_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -37,7 +37,7 @@

from twisted.internet import defer

from benchmarking import job
from kiosk_client import job

global FAILED
FAILED = False # global toggle for failed responses
Expand Down
10 changes: 5 additions & 5 deletions benchmarking/manager.py → kiosk_client/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -38,11 +38,11 @@
from twisted.internet import defer, reactor
from twisted.web.client import HTTPConnectionPool

from benchmarking.job import Job
from benchmarking.utils import iter_image_files, sleep, strip_bucket_prefix
from benchmarking import settings
from kiosk_client.job import Job
from kiosk_client.utils import iter_image_files, sleep, strip_bucket_prefix
from kiosk_client import settings

from benchmarking.cost import CostGetter
from kiosk_client.cost import CostGetter


class JobManager(object):
Expand Down
6 changes: 3 additions & 3 deletions benchmarking/manager_test.py → kiosk_client/manager_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -37,8 +37,8 @@
import pytest
import pytest_twisted

from benchmarking import manager
from benchmarking import settings
from kiosk_client import manager
from kiosk_client import settings


class TestJobManager(object):
Expand Down
2 changes: 1 addition & 1 deletion benchmarking/settings.py → kiosk_client/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down
2 changes: 1 addition & 1 deletion benchmarking/utils.py → kiosk_client/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down
4 changes: 2 additions & 2 deletions benchmarking/utils_test.py → kiosk_client/utils_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand Down Expand Up @@ -36,7 +36,7 @@
from PIL import Image
import pytest

from benchmarking import utils
from kiosk_client import utils


class TestUtils(object):
Expand Down
10 changes: 5 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.github.com/vanvalenlab/kiosk-benchmarking/LICENSE
# http://www.github.com/vanvalenlab/kiosk-client/LICENSE
#
# The Work provided may be used for non-commercial academic purposes only.
# For any other use of the Work, including commercial use, please contact:
Expand All @@ -26,14 +26,14 @@
from setuptools import setup
from setuptools import find_packages

setup(name='Kiosk_Benchmarking',
setup(name='Kiosk_Client',
version='0.4.0',
description='Benchmarking utility for the DeepCell Kiosk.',
description='ClI client for the DeepCell Kiosk.',
author='Van Valen Lab',
author_email='vanvalenlab@gmail.com',
url='https://github.com/vanvalenlab/kiosk-benchmarking',
url='https://github.com/vanvalenlab/kiosk-client',
download_url='https://github.com/vanvalenlab/'
'kiosk-benchmarking/tarball/0.4.0',
'kiosk-client/tarball/0.4.0',
license='LICENSE',
install_requires=['boto3',
'google-cloud-storage',
Expand Down

0 comments on commit f2d5b67

Please sign in to comment.