Add test framework for TxPy

Summary:
- Add pytest framework on top of vanilla unittest
- Set up necessary plumbing such as .tox and conftest.py
- Update contributing and readme.md
- Add read me for testing

Test Plan: `tox` works, `py.test` works, `python setup.py test` works

Reviewers: evan

Differential Revision: https://work.r23s.net/D4624

.coveragerc

@@ -0,0 +1,4 @@
+[report]
+exclude_lines =
+    pragma: no cover
+    def __repr__

CONTRIBUTING.md

@@ -20,17 +20,18 @@ matplotlib and plotly is preferred.
 
 
 ## Version Compatibility
-We use the (future)[https://pypi.python.org/pypi/future] module to maintain Python 2/3 compatibility. As a result, all
+We use the [future](https://pypi.python.org/pypi/future) module to maintain Python 2/3 compatibility. As a result, all
 code written should be Python 2.6/2.7 and Python 3.3+ compatible.
 
+
 ## Styling and Documentation
-All code written should follow the (PEP8 standard)[https://www.python.org/dev/peps/pep-0008/]
+All code written should follow the [PEP8 standard](https://www.python.org/dev/peps/pep-0008/)
 
-For documentation purposes, we follow (NumPy style doc strings)[https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt]
+For documentation purposes, we follow [NumPy style doc strings](https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt)
 
 
 ## Testing
-For testing purposes, we write tests in the `test` folder in the (unittests)[https://docs.python.org/2.7/library/unittest.html]
-format. We also use (tox)[https://tox.readthedocs.org/en/latest/] for automating tests.
+For testing purposes, we write tests in the `test` folder in the [pytest](http://pytest.org/latest/getting-started.html)
+format. We also use [tox](https://tox.readthedocs.org/en/latest/) for automating tests.
 
 Ensure that all tests pass when you run `tox` in the main folder.

README.md

@@ -1,47 +1,25 @@
-# ![](https://www.transcriptic.com/images/logo-transcriptic-blobs-880e2f7b.svg) Transcriptic Python Library
+# Testing
 
-The Transcriptic Python Library (TxPy) provides a Python interface for managing Transcriptic organizations, projects, runs, datasets and more.
-One can either interface with our library through the bundled command line interface (CLI) or through a Jupyter notebook.
+## Overview
 
-We recommend using the Jupyter interface as it provides a nice rendering and presentation of the objects, as well as provide
-additional analysis and properties functions specific to the Transcriptic objects.
+We use the [pytest](http://pytest.org/latest/getting-started.html) framework for writing tests. A helpful resource for a
+list of supported assertions can be found [here](https://pytest.org/latest/assert.html).
+We also use [tox](https://tox.readthedocs.org/en/latest/) for automating tests.
 
-Transcriptic is the robotic cloud laboratory for the life sciences. [https://www.transcriptic.com](https://www.transcriptic.com)
+Ensure that all tests pass when you run `tox` in the main folder.
+Alternatively, one can run `py.test <path_to_test>` if you are trying to test a specific module.
 
-## Setup
+## Structure
+Helper functions should go into the helpers folder, and autoprotocol json in the autoprotocol folder.
+The `conftest.py` file contains any pytest related configurations.
 
-```
-$ pip install transcriptic
-```
+## Writing API Tests
+For API tests, we overwrite the base call command with a mock call that draws from the `mockDB` dictionary.
+To mock responses, we use the `MockResponse` class, which basically mirrors the `Response` class from the
+requests library. The three key fields to mock here are the `status_code`, `json` and `text` fields.
+Similar to the `Response` object, the `status_code` corresponds to the HTML response codes (e.g. 404, 201),
+the `json` method corresponds to the `json` response which is usually returned when the call is succesful.
+Lastly, the `text` field is what's commonly used for displaying logs/error messages.
 
-or
+The `api_test` module is a nice reference for how one typically sets up responses and test.
 
-```
-$ git clone https://github.com/transcriptic/transcriptic.git
-$ cd transcriptic
-$ pip install .
-```
-
-to upgrade to the latest version using pip or check whether you're already up to date:
-```
-$ pip install transcriptic --upgrade
-```
-
-Then, login to your Transcriptic account:
-
-```
-$ transcriptic login
-Email: me@example.com
-Password:
-Logged in as me@example.com (example-lab)
-```
-
-## Documentation
-
-See the [Transcriptic Developer Documentation](https://developers.transcriptic.com/docs/getting-started-with-the-cli) for detailed information about how to use this package, including learning about how to package protocols and build releases.
-
-View [developer specific documentation] (http://transcriptic.github.io/transcriptic/index.html)
-
-## Contributing
-
-Read CONTRIBUTING.mmd for more information on contributing to TxPy

setup.cfg

@@ -1,2 +1,11 @@
 [wheel]
 universal = 1
+
+[aliases]
+test=pytest
+
+[pytest]
+minversion = 2.3.3
+python_files = "test/*_test.py" "test/test_*.py"
+python_classes = Test
+norecursedirs = ".tox" "build" "docs"

setup.py

@@ -14,7 +14,8 @@
     url='https://github.com/transcriptic/transcriptic',
     version=__version__,
     packages=['transcriptic', 'transcriptic.analysis'],
-    test_suite='test',
+    setup_requires=['pytest-runner'],
+    test_requires=['pytest'],
     install_requires=[
         'Click>=5.1',
         'requests',

test/api_test.py

@@ -1,11 +1,10 @@
-import unittest
-from .mockAPI import mockRoute, MockResponse, _req_call as mockCall
-from .helper import load_protocol
+import pytest
+
+from util import load_protocol
+from mockAPI import mockRoute, MockResponse, _req_call as mockCall
 from transcriptic import api
 from transcriptic.config import Connection, AnalysisException
 
-
-
 api._req_call = mockCall
 
 test_ctx = Connection(email="mock@api.com", organization_id="mock", api_root="mock-api")
@@ -16,7 +15,7 @@
 single_transfer_response = load_protocol('singleTransfer_response')
 
 
-class AnalyzeTestCase(unittest.TestCase):
+class TestAnalyze:
     def testDefaultResponses(self):
         # Setup default parameters
         route = test_ctx.get_route('analyze_run')
@@ -28,16 +27,16 @@ def testDefaultResponses(self):
         mock200 = MockResponse(status_code=200, json=single_transfer_response)
 
         mockRoute(call, route, mock404, max_calls=1)
-        with self.assertRaises(Exception):
+        with pytest.raises(Exception):
             test_ctx.analyze_run(invalid_transfer_protocol)
 
         mockRoute(call, route, mock400, max_calls=1)
-        with self.assertRaises(Exception):
+        with pytest.raises(Exception):
             test_ctx.analyze_run(invalid_transfer_protocol)
 
         mockRoute(call, route, mock422, max_calls=1)
-        with self.assertRaises(AnalysisException):
+        with pytest.raises(AnalysisException):
             test_ctx.analyze_run(invalid_transfer_protocol)
 
         mockRoute(call, route, mock200, max_calls=1)
-        self.assertEqual(test_ctx.analyze_run(single_transfer_protocol), single_transfer_response)
+        assert test_ctx.analyze_run(single_transfer_protocol) == single_transfer_response

test/conftest.py

@@ -0,0 +1,3 @@
+import sys
+import os
+sys.path.append(os.path.join(os.path.dirname(__file__), 'helpers'))

test/mockAPI.py → test/helpers/mockAPI.py

@@ -10,17 +10,21 @@ class MockResponse(object):
 
     def __init__(self, status_code=None, json=None, text=None):
         self.status_code = status_code
+        self.text = text
         self.json_data = json
-        self.text_data = text
 
+    @property
     def status_code(self):
         return self.status_code
 
+    @property
+    def text(self):
+        return self.text_data
+
     def json(self):
         return self.json_data
 
-    def text(self):
-        return self.text_data
+
 
 
 def _req_call(method, route, **kwargs):

tox.ini

@@ -1,16 +1,19 @@
 [tox]
-envlist = clean,py{27,35},stats
+envlist = clean, py{27,35}, stats
 
 [testenv]
-commands =
-  coverage run --source transcriptic -a setup.py test
 deps =
-  coverage
+    pytest
+    coverage
+    pytest-cov
+
+commands =
+    py.test --cov={envsitepackagesdir}/transcriptic --cov-report=term
 
 [testenv:clean]
 commands =
   coverage erase
 
 [testenv:stats]
 commands =
-  coverage report -m --rcfile={toxinidir}/.coveragerc
+  coverage report -m --rcfile={toxinidir}/.coveragerc