Export HTML report with --export #165

Closed
wants to merge 6 commits into
from

Conversation

Projects
None yet
2 participants
@nilbus
Contributor

nilbus commented Jan 25, 2014

This PR implements #121.

The Teaspoon reports can be exported as a static HTML page. This requires that wget is installed.

teaspoon --export="output_directory"
teaspoon --export # output directory defaults to './teaspoon-export'.

To accomplish this, wget downloads from the teaspoon server the HTML and any assets the page needs to load.
The page (by default) is downloaded to ./teaspoon-export/suite_name/index.html.

Specs have been added to cover the new functionality.
This is also manually tested and working in my JRuby app, which mounts teaspoon at a nonstandard route (/web/teaspoon).

Cookie session storage is not available when viewing static html files via file://.
Therefore instead, the HTML reporter uses localStorage when available. If localStorage is not available, it falls back to using cookies.

Please give me any feedback you have. If you're happy with this implementation, I'd love to see this released as a new gem version soon. Thanks!

nilbus added some commits Nov 19, 2013

Fix toggle buttons when teaspoon page is loaded statically via localhost
Using cookies does not work on localhost.
Instead, use localStorage when available.
If localStorage is not implemented, fall back to using cookies.

Added a spec to ensure button toggles still work.
Cookie storage is not tested in this feature spec because localStorage cannot be disabled using capybara.
Add the --export option with specs
Export is not yet implemented.
@jejacks0n

This comment has been minimized.

Show comment
Hide comment
@jejacks0n

jejacks0n Jan 25, 2014

Owner

Thanks. =) Can you give me some background so I understand why you want this? Before I dig into it, I want to understand what this provides or solves for you.

Owner

jejacks0n commented Jan 25, 2014

Thanks. =) Can you give me some background so I understand why you want this? Before I dig into it, I want to understand what this provides or solves for you.

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jan 26, 2014

Contributor

Good question. In addition to running the tests, our Jenkins CI builds
create build artifacts that give us an HTML representation of each of our
test suites' results (rspec, java dbunit, etc., and now teaspoon) and save
them for later viewing. This is useful because our test suites run
concurrently, and some generate lots and lots of output in the build log
that is not usually helpful when trying to examine the test results.
Teaspoon was the odd one out that was only able to output the test results
to the CI build log and not an HTML file.

Contributor

nilbus commented Jan 26, 2014

Good question. In addition to running the tests, our Jenkins CI builds
create build artifacts that give us an HTML representation of each of our
test suites' results (rspec, java dbunit, etc., and now teaspoon) and save
them for later viewing. This is useful because our test suites run
concurrently, and some generate lots and lots of output in the build log
that is not usually helpful when trying to examine the test results.
Teaspoon was the odd one out that was only able to output the test results
to the CI build log and not an HTML file.

@jejacks0n

This comment has been minimized.

Show comment
Hide comment
@jejacks0n

jejacks0n Jan 26, 2014

Owner

Can I ask why none of the existing build artifact/formats worked? eg. TeamCity, jUnit, or TAP?

Owner

jejacks0n commented Jan 26, 2014

Can I ask why none of the existing build artifact/formats worked? eg. TeamCity, jUnit, or TAP?

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jan 26, 2014

Contributor

I really like the HTML format that Teaspoon already provides. I considered
trying to recreate something similar, translating from TAP data, but I have
a hard time trying to justify the work to recreate that when I can instead
capture what is already provided.

There's an important difference too in that the HTML export is an active
test, whereas HTML output that translated from TAP data would be static.
Since the export is actually running the test against the javascript code
on each load, it can be investigated in different browsers right on the
export page rather than having to check out the right branch and run
teaspoon locally.

Mostly I just wanted artifacts that look and act like the Teaspoon web
interface itself.

Contributor

nilbus commented Jan 26, 2014

I really like the HTML format that Teaspoon already provides. I considered
trying to recreate something similar, translating from TAP data, but I have
a hard time trying to justify the work to recreate that when I can instead
capture what is already provided.

There's an important difference too in that the HTML export is an active
test, whereas HTML output that translated from TAP data would be static.
Since the export is actually running the test against the javascript code
on each load, it can be investigated in different browsers right on the
export page rather than having to check out the right branch and run
teaspoon locally.

Mostly I just wanted artifacts that look and act like the Teaspoon web
interface itself.

@jejacks0n

This comment has been minimized.

Show comment
Hide comment
@jejacks0n

jejacks0n Feb 13, 2014

Owner

I played around with this concept in the 0.8 branch, and I think we can do it more efficiently using phantomjs/selenium instead of wget.

I'll see if this can go in cleanly.

I had a question though -- you mention saving all the assets -- is this needed, or can they be re-served from the CI box? As in, I would rather just save the index.html and not all the assets that were loaded. I'll have to look at what you did there.

Owner

jejacks0n commented Feb 13, 2014

I played around with this concept in the 0.8 branch, and I think we can do it more efficiently using phantomjs/selenium instead of wget.

I'll see if this can go in cleanly.

I had a question though -- you mention saving all the assets -- is this needed, or can they be re-served from the CI box? As in, I would rather just save the index.html and not all the assets that were loaded. I'll have to look at what you did there.

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Feb 13, 2014

Contributor

They cannot be re-served from the CI box. Every build has a different
branch or commit with potentially different versions of assets. Once the CI
build is complete, not only does the server stop, but the build tree is
deleted. This is why we need to save a copy of all the assets.

As far as I can tell, phantom has no mechanism for downloading a page's
assets and rewriting the URLs on the page so that it can load the assets
locally. You would have to implement that yourself, which would be quite an
undertaking.

Contributor

nilbus commented Feb 13, 2014

They cannot be re-served from the CI box. Every build has a different
branch or commit with potentially different versions of assets. Once the CI
build is complete, not only does the server stop, but the build tree is
deleted. This is why we need to save a copy of all the assets.

As far as I can tell, phantom has no mechanism for downloading a page's
assets and rewriting the URLs on the page so that it can load the assets
locally. You would have to implement that yourself, which would be quite an
undertaking.

jejacks0n added a commit that referenced this pull request Feb 16, 2014

adds back the concept of html export. #121
- similar structure as the original pull request from @nilbus #165
@jejacks0n

This comment has been minimized.

Show comment
Hide comment
@jejacks0n

jejacks0n Feb 16, 2014

Owner

Getting this into 0.8.. thanks. =)

Owner

jejacks0n commented Feb 16, 2014

Getting this into 0.8.. thanks. =)

@jejacks0n jejacks0n closed this Feb 16, 2014

jejacks0n added a commit that referenced this pull request Feb 16, 2014

adds back the concept of html export. #121
- similar structure as the original pull request from @nilbus #165
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment