Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better Error Summaries for API Tests #4397

Merged
merged 3 commits into from Aug 14, 2017

Conversation

Projects
None yet
3 participants
@jmchilton
Copy link
Member

commented Aug 9, 2017

  • Modify existing code that would summarize a history in error state to summarize all datasets and dataset collections - include more information. I've found this is much more helpful in debugging problems with tests downstream.
    • The peak is included now which is helpful to see what tools actually generated during tests.
    • Collections are included at least a little bit now which is useful.
    • Non-error datasets are included which makes it easier to debug problems with datasets that are green but shouldn't be.
  • Implement a new contextmanager for creating histories that calls this summary code if any exception is thrown inside of the context. Makes this summary code more useful in more settings.
  • Update a random assortment of tool API tests to use this new context manager - mostly just as a demonstration - these particular tests aren't particularly interesting.

Again useful test code stemming from downstream work in the CWL branch.

jmchilton added some commits Jul 26, 2017

Rework summarizing histories on errors to include all datasets.
```
-------------------- >> begin captured stdout << ---------------------
Problem in history with id adb5f5c93f827949 - summary of datasets below.
--------------------------------------
| 1 - Test Dataset (HID - NAME)
| Dataset State:
|  ok
| Dataset Blurb:
|  1 line
| Dataset Info:
|  uploaded txt file
| Peek:
|  <table cellspacing="0" cellpadding="3"><tr><td>1     2       3</td></tr></table>
| Dataset Job Standard Output:
|  *Standard output was empty.*
| Dataset Job Standard Error:
|  *Standard error was empty.*
|
--------------------------------------
| 2 - Test Dataset (HID - NAME)
| Dataset State:
|  ok
| Dataset Blurb:
|  1 line
| Dataset Info:
|  uploaded txt file
| Peek:
|  <table cellspacing="0" cellpadding="3"><tr><td>4     5       6</td></tr></table>
| Dataset Job Standard Output:
|  *Standard output was empty.*
| Dataset Job Standard Error:
|  *Standard error was empty.*
|
--------------------------------------
| 3 - multi_data_param on data 1 and data 2 (HID - NAME)
| Dataset State:
|  ok
| Dataset Blurb:
|  2 lines
| Dataset Info:
|  *Dataset info is empty.*
| Peek:
|  <table cellspacing="0" cellpadding="3"><tr><td>1     2       3</td></tr><tr><td>4    5       6</td></tr></table>
| Dataset Job Standard Output:
|  *Standard output was empty.*
| Dataset Job Standard Error:
|  *Standard error was empty.*
|
--------------------------------------
| 4 - multi_data_param on data 1 and data 2 (HID - NAME)
| Dataset State:
|  ok
| Dataset Blurb:
|  2 lines
| Dataset Info:
|  *Dataset info is empty.*
| Peek:
|  <table cellspacing="0" cellpadding="3"><tr><td>4     5       6</td></tr><tr><td>1    2       3</td></tr></table>
| Dataset Job Standard Output:
|  *Standard output was empty.*
| Dataset Job Standard Error:
|  *Standard error was empty.*
|
--------------------------------------
```
@mvdbeek

This comment has been minimized.

Copy link
Member

commented Aug 14, 2017

Wonderful, I really like this!

@mvdbeek mvdbeek merged commit 8e192db into galaxyproject:dev Aug 14, 2017

5 checks passed

api test Build finished. 280 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 150 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 37 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 579 tests run, 0 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.