Browse files

update script usage testing docs

now that we don't have output files/dirs included under qiime_test_data/
  • Loading branch information...
1 parent 485c649 commit de7b2a6e460196e5bdfa797dc2f44b9260496fdb @jairideout jairideout committed Dec 12, 2013
Showing with 7 additions and 18 deletions.
  1. +1 −1 CONTRIBUTING.md
  2. +6 −17 doc/developer/script_testing.rst
View
2 CONTRIBUTING.md
@@ -77,7 +77,7 @@ git checkout -b my-topic-branch
7. Make your changes, add them (with ``git add``), and commit them (with ``git commit``). Don't forget to update associated scripts and tests as necessary. You should make incremental commits, rather than one massive commit at the end. Write descriptive commit messages to accompany each commit.
-8. When you think you're ready to submit your code, again ensure that you have the latest version of all files in case some changed while you were working on your edits. You can do this by merging master into your topic branch::
+8. When you think you're ready to submit your code, again ensure that you have the latest version of all files in case some changed while you were working on your edits. You can do this by merging master into your topic branch:
```
git checkout my-topic-branch
git pull upstream master
View
23 doc/developer/script_testing.rst
@@ -4,7 +4,7 @@ Automated script testing
QIIME is well-tested: we make automated software testing a high priority, as you can see by reviewing the ``Qiime/tests/`` directory `here <https://github.com/qiime/qiime/tree/master/tests>`_. After installing QIIME, users can run the automated test suite by changing to the ``Qiime/tests`` directory and running::
- all_tests.py
+ ./all_tests.py
These tests are additionally run on a nightly basis, and developers are alerted to test failures.
@@ -47,11 +47,10 @@ The recommended way of testing QIIME is to run QIIME's unit tests and script usa
How the script usage tests work
===============================
-You'll see many sub-directories in ``qiime_test_data`` with names corresponding to the names of QIIME scripts. Each of these directories contains example input and output for the corresponding QIIME script. For example, the ``make_otu_table`` directory contains the following test data for ``make_otu_table.py``::
+You'll see many sub-directories in ``qiime_test_data`` with names corresponding to the names of QIIME scripts. Each of these directories contains example input files for the corresponding QIIME script. For example, the ``make_otu_table`` directory contains the following test data for ``make_otu_table.py``::
ls make_otu_table/
- chimeric_seqs.txt otu_table.biom otu_table_non_chimeric.biom tax_assignments.txt
- otu_map.txt otu_table_no_pynast_failures.biom pynast_failures.fna
+ chimeric_seqs.txt otu_map.txt pynast_failures.fna tax_assignments.txt
If you call ``make_otu_table.py -h``, you'll see the following usage examples::
@@ -68,16 +67,16 @@ If you call ``make_otu_table.py -h``, you'll see the following usage examples::
Make OTU table, excluding a list of OTU ids: Make an OTU table, excluding the sequences listed in chimeric_seqs.txt
make_otu_table.py -i otu_map.txt -o otu_table_non_chimeric.biom -e chimeric_seqs.txt
-What you'll notice is that the usage example input and output files correspond to the files in ``qiime/qiime_test_data/make_otu_table``. The script interface testing works by copying all of the files in ``make_otu_table`` to a temporary directory, changing into that directory, running each of the usage examples, and confirming that the script exited successfully (i.e., with an exit status of ``0``).
+What you'll notice is that the usage example input files correspond to the files in ``qiime/qiime_test_data/make_otu_table``. The script interface testing works by copying all of the files in ``make_otu_table`` to a temporary directory, changing into that directory, running each of the usage examples, and confirming that the script exited successfully (i.e., with an exit status of ``0``).
- .. warning:: Currently the script usage tests only test whether a script exits successfully: they do not check whether the results correspond to the example output. The reasoning is that that would duplicate the functionality of the unit tests (which isn't a bad thing, except that implementing this would be a lot of work). These are tests that the interfaces themselves are working.
+ .. warning:: Currently the script usage tests only test whether a script exits successfully: they do not check whether the results correspond to expected output. The reasoning is that that would duplicate the functionality of the unit tests (which isn't a bad thing, except that implementing this would be a lot of work). These are tests that the interfaces themselves are working.
If you don't see a directory corresponding to a script name in the ``qiime_test_data`` directory, that means that a script interface test has not been defined for the given script. We're currently working on extending this framework to cover all QIIME scripts.
Adding script interface testing for new scripts
===============================================
-Adding new script interface tests is easy. All you do is create a new test directory in ``qiime_test_data``, where the name of the directory corresponds to the script's name. For example, if you're adding tests for ``my_script.py``, you'd add a directory called ``my_script``. In that directory you would create example input and output files for all of the script usage examples that are defined in your script. Make several usage examples that make use of different paths through your script.
+Adding new script interface tests is easy. All you need to do is create a new test directory in ``qiime_test_data``, where the name of the directory corresponds to the script's name. For example, if you're adding tests for ``my_script.py``, you'd add a directory called ``my_script``. In that directory you would create example input files for all of the script usage examples that are defined in your script. Make several usage examples that make use of different paths through your script.
Full paths
----------
@@ -94,13 +93,3 @@ We recommend specifying full paths for many of QIIME scripts, and importantly fo
subdirectories as appropriate. ALWAYS SPECIFY ABSOLUTE FILE PATHS (absolute path represented
here as $PWD, but will generally look something like /home/ubuntu/my_analysis/).
pick_de_novo_otus.py -i $PWD/seqs.fna -o $PWD/otus/
-
-Cleaning up output files
-------------------------
-Some scripts require that the user-specified output directory does not exist when the script runs, but we provide example output in the test directory. To automatically remove output directories prior to running the tests, add the ``script_usage_output_to_remove`` entry to your script info. For example, from ``pick_de_novo_otus.py``::
-
- script_info['script_usage_output_to_remove'] = ['$PWD/otus/']
-
-
-
-

0 comments on commit de7b2a6

Please sign in to comment.