Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow outputing test results as allure framework results. #1115

Merged
merged 2 commits into from Dec 28, 2020

Conversation

jmchilton
Copy link
Member

@jmchilton jmchilton commented Dec 16, 2020

Implements #1113.

xref http://allure.qatools.ru/

Quick Usage:

brew install allure
# Convert a Planemo/Ephemeris tool_test_outpt.json into a allure results directory 
planemo test_reports --test_output_allure allure_results/ ~/tool_test_outpt.json
# Generate a static site from that.
allure generate --clean allure_results/
# Serve and open with a browser (allure has a little shortcut for this)
allure open allure-repors/

Landing summary.

Screen Shot 2020-12-16 at 1 12 29 AM


Can dig into "Categories" and filter by exception vs failure:

Screen Shot 2020-12-16 at 1 26 34 AM


Category view group similar problems together:

Screen Shot 2020-12-16 at 1 28 21 AM


Screen Shot 2020-12-16 at 1 12 45 AM


Suite view allows digging into tools -> versions -> test index. Clicking a test allows access to a lot of data collected for debugging the problem.

Screen Shot 2020-12-16 at 1 15 46 AM


Pretty graphs that don't do a whole lot in our context.

Screen Shot 2020-12-16 at 1 15 57 AM


Pretty timeline that doesn't do a whole lot in our context.

Screen Shot 2020-12-16 at 1 16 26 AM

@bgruening
Copy link
Member

That is pretty cool!

John is this --test_output_allure allure not redundant? Should this be --test_output allure?

@mvdbeek
Copy link
Member

mvdbeek commented Dec 16, 2020

That is in line with --test_output_markdown, --test_output_xunit etc.

@bgruening
Copy link
Member

And is this not redundant? Having --test_output_allure and the positional argument allure seems unnecessary to me. Should I open another issue about that?

@jmchilton
Copy link
Member Author

Here allure is the directory that is being created, I don't think things are redundant. I'll update the example commands to clarify that.

@bgruening
Copy link
Member

Thanks @jmchilton for the explanation!

@jmchilton jmchilton merged commit ce17645 into galaxyproject:master Dec 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants