docs/notebooks/4_config_single_workflow_test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Define a single test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "See the folder *example*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "EXAMPLES_FOLDER = \"../examples\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "... where you can find the following files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['change_case',\n",
       " 'fluxomics_stationary',\n",
       " 'multivariate',\n",
       " 'sacurine',\n",
       " 'workflow-test-suite-full.yml',\n",
       " 'workflow-test-suite-min.yml',\n",
       " 'workflow-test-suite.yml',\n",
       " 'workflows.json']"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os, pprint\n",
    "[f for f in os.listdir(EXAMPLES_FOLDER) if not f.startswith('.')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Consider the **definition file** ``workflow-test-suite-min.yml`` (steps[3-4]), which contains the two workflow tests named ``change_case`` and ``multivariate`` respectively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "suite_conf_filename = os.path.join(EXAMPLES_FOLDER, \"workflow-test-suite-min.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"enable_logger\": false, \n",
      "    \"workflows\": {\n",
      "        \"change_case\": {\n",
      "            \"expected\": {\n",
      "                \"OutputText\": \"change_case/expected_output\"\n",
      "            }, \n",
      "            \"inputs\": {\n",
      "                \"InputText\": \"change_case/input\"\n",
      "            }, \n",
      "            \"file\": \"change_case/workflow.ga\"\n",
      "        }, \n",
      "        \"multivariate\": {\n",
      "            \"expected\": {\n",
      "                \"variableMetadata_out\": \"multivariate/variableMetadata_out\", \n",
      "                \"sampleMetadata_out\": \"multivariate/sampleMetadata_out\"\n",
      "            }, \n",
      "            \"inputs\": {\n",
      "                \"DataMatrix\": \"multivariate/dataMatrix.tsv\", \n",
      "                \"SampleMetadata\": \"multivariate/sampleMetadata.tsv\", \n",
      "                \"VariableMetadata\": \"multivariate/variableMetadata.tsv\"\n",
      "            }, \n",
      "            \"params\": {\n",
      "                \"3\": {\n",
      "                    \"predI\": \"1\", \n",
      "                    \"respC\": \"gender\", \n",
      "                    \"orthoI\": \"NA\", \n",
      "                    \"testL\": \"FALSE\"\n",
      "                }\n",
      "            }, \n",
      "            \"file\": \"multivariate/workflow.ga\"\n",
      "        }\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json, yaml\n",
    "with open(suite_conf_filename, \"r\") as fp:\n",
    "    data = yaml.load(fp)\n",
    "    print(json.dumps(data, indent=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "... and focus on the first example ``change_case``:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<center>![wf-changecase-workflow](../images/ChangeCaseWorkflow.png \"Workflow ChangeCase\")</center>\n",
    "<center>**Figure 1.** Workflow *\"ChangeCase\"*</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "As Fig. 1 shows, the workflow has one input and one output. To test it, both the input and the output must be uniquely identified. Typically, Galaxy identifies them using **Names** and **Labels**, respectively.  For our sample workflow, the identifiers are:\n",
    "\n",
    "* Name **\"InputText\"** for the input (Fig. 2);\n",
    "* Label **\"OutputText\"** for the output (Fig. 3)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<div>\n",
    "<div style=\"display: inline-block; width: 50%; margin: 0 auto;\">\n",
    "<center>![wf-changecase-inputdataset](../images/ChangeCaseInputDataset.png \"Workflow “Input Dataset“\")</center>\n",
    "<center>**Figure 2.** Workflow input: *\"Input Dataset\"*</center>\n",
    "</div>\n",
    "<div style=\"display: inline-block\">\n",
    "<center>![wf-changecase-outputdataset](../images/ChangeCaseOutputDataset.png \"Workflow “Output Dataset“\")</center>\n",
    "<center>**Figure 3.** Workflow output: *\"output1\"*</center>\n",
    "</div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "You can programmatically inpect the Galaxy workflow definition by means of the ``Workflow`` class, which allows you to see which are the workflow inputs and outputs (steps[5-8]):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from wft4galaxy.wrapper import Workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "wf = Workflow.load(os.path.join(EXAMPLES_FOLDER, \"change_case/workflow.ga\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-  InputText \n"
     ]
    }
   ],
   "source": [
    "wf.show_inputs()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'1': OutputText\n"
     ]
    }
   ],
   "source": [
    " wf.show_outputs()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Steps 9-13 define a ``WorkflowTestCase`` instance programmatically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from wft4galaxy.core import WorkflowTestCase"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# prepare the workflow definition file name\n",
    "base_path = os.path.join(EXAMPLES_FOLDER, \"change_case\")\n",
    "workflow_filename = \"workflow.ga\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# prepare test inputs\n",
    "inputs = {\"InputText\": {\"file\": \"input\"}}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# prepare test outputs\n",
    "expected_outputs = {\"OutputText\": {\"file\": \"expected_output\"}}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# create test case instance\n",
    "test_case = WorkflowTestCase(base_path=base_path, \n",
    "                             workflow_filename=workflow_filename, \n",
    "                             inputs=inputs, expected_outputs=expected_outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "You can now run your test case and inspect the results of its execution (steps [14-16]), as described in \"[Run a single test](3_run_single_workflow_test.html)\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-03-30 14:53:38,244 [wft4galaxy] [ INFO]  Create a history '_WorkflowTestHistory_e46ecb26-1547-11e7-8812-a45e60c4fc6b' (id: u'795b4cb2a2107ba7')\n",
      "2017-03-30 14:53:40,120 [wft4galaxy] [ INFO]  Workflow '_WorkflowTest_Change Case (imported from API)' (id: 795b4cb2a2107ba7) running ...\n",
      "2017-03-30 14:53:45,298 [wft4galaxy] [ INFO]  waiting for datasets\n",
      "2017-03-30 14:53:45,690 [wft4galaxy] [ INFO]  44015a2f24f1f430: queued\n",
      "2017-03-30 14:53:46,639 [wft4galaxy] [ INFO]  44015a2f24f1f430: queued\n",
      "2017-03-30 14:53:47,330 [wft4galaxy] [ INFO]  44015a2f24f1f430: running\n",
      "2017-03-30 14:53:48,075 [wft4galaxy] [ INFO]  44015a2f24f1f430: running\n",
      "2017-03-30 14:53:49,025 [wft4galaxy] [ INFO]  44015a2f24f1f430: running\n",
      "2017-03-30 14:53:49,660 [wft4galaxy] [ INFO]  44015a2f24f1f430: ok\n",
      "2017-03-30 14:53:50,165 [wft4galaxy] [ INFO]  Workflow '_WorkflowTest_Change Case (imported from API)' (id: 795b4cb2a2107ba7) executed\n",
      "2017-03-30 14:53:50,168 [wft4galaxy] [ INFO]  Checking test output: ...\n",
      "2017-03-30 14:53:50,341 [wft4galaxy] [ INFO]  Checking test output: DONE\n"
     ]
    }
   ],
   "source": [
    "test_result = test_case.run(enable_logger=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test e46ecb26-1547-11e7-8812-a45e60c4fc6b:\n",
      "\t - workflow: [_WorkflowTest_Change Case (imported from API)] \n",
      "\t - results: {u'OutputText': True}\n"
     ]
    }
   ],
   "source": [
    "print(\"Test %s:\\n\\t - workflow: [%s] \\n\\t - results: %r\" % \\\n",
    "      (test_result.test_id, test_result.workflow.name, test_result.results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_result.check_output(\"OutputText\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}