Handle Ant's JUnit XML formatted output from test/spec runs #239

Closed
ndbroadbent opened this Issue Sep 5, 2011 · 35 comments

Comments

Projects
None yet
@ndbroadbent
Contributor

ndbroadbent commented Sep 5, 2011

Ant's JUnit XML format is a standard output for almost every test framework, and can be read by many continuous integration systems, such as Hudson or Atlassian Bamboo:

We could start by handling the ci_reporter gem, since a lot of projects on Travis CI are Ruby / Rails. ci_reporter provides formatters for Test::Unit, RSpec and Cucumber. If we assume that some projects will be configured to use ci_reporter, then we could just test for the presence of the XML output at the end of the test run, and handle it if it exists.
Alternatively, we could detect the presence of a ruby test framework and inject ci_reporter into the project's Gemfile before running the test suite. This idea is possibly related to the issue about generating a Gemfile, since they will both automatically alter the project's environment.

This would allow us to store / notify which tests passed / failed, and include backtraces for each error, etc.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Sep 5, 2011

Member

This is an interesting idea, not keen about injecting any gem deps, but we should do some form of smart parsing ourselves with the saved log output and maybe show it in the ui and in the emails.

Is there a way we can use ci reporter ourselves with saved log output?


Sent from my Sega Master System

On 5/09/2011, at 1:31 PM, ndbroadbentreply@reply.github.com wrote:

This is probably on the roadmap, but I haven't seen it mentioned anywhere.

The ci_reporter gem generates XML reports of your test and/or spec runs. Travis CI could then parse the generated XML, and send the formatted failure info (backtrace, line number, etc.) along with the notification email.
Could also show the test details in the UI somewhere.

It could be an optional flag in .travis.yml to state that the project already uses ci_reporter. OR, we could do something fancy and inject ci_reporter before the test suite is run.

I think features like this would really start to make Travis CI on par with Hudson / CruiseControl / Bamboo. (is that the aim?)

Reply to this email directly or view it on GitHub:
#239

Member

joshk commented Sep 5, 2011

This is an interesting idea, not keen about injecting any gem deps, but we should do some form of smart parsing ourselves with the saved log output and maybe show it in the ui and in the emails.

Is there a way we can use ci reporter ourselves with saved log output?


Sent from my Sega Master System

On 5/09/2011, at 1:31 PM, ndbroadbentreply@reply.github.com wrote:

This is probably on the roadmap, but I haven't seen it mentioned anywhere.

The ci_reporter gem generates XML reports of your test and/or spec runs. Travis CI could then parse the generated XML, and send the formatted failure info (backtrace, line number, etc.) along with the notification email.
Could also show the test details in the UI somewhere.

It could be an optional flag in .travis.yml to state that the project already uses ci_reporter. OR, we could do something fancy and inject ci_reporter before the test suite is run.

I think features like this would really start to make Travis CI on par with Hudson / CruiseControl / Bamboo. (is that the aim?)

Reply to this email directly or view it on GitHub:
#239

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Sep 5, 2011

Contributor

The idea comes from Heroku, who do code injection for Rails apps.

One example of this is rails_log_stdout, a plugin injected into Rails apps to cause their logs to go to stdout, as is required for routing logs into the logging system.

There's probably a heroku dev out there who could give us some pointers, although it doesn't seem like vendoring an extra plugin would be too dangerous.

ci_reporter extends RSpec with a new formatter, so as far as I'm aware it can't be used to post-process the saved log output.

Contributor

ndbroadbent commented Sep 5, 2011

The idea comes from Heroku, who do code injection for Rails apps.

One example of this is rails_log_stdout, a plugin injected into Rails apps to cause their logs to go to stdout, as is required for routing logs into the logging system.

There's probably a heroku dev out there who could give us some pointers, although it doesn't seem like vendoring an extra plugin would be too dangerous.

ci_reporter extends RSpec with a new formatter, so as far as I'm aware it can't be used to post-process the saved log output.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Sep 5, 2011

Member

Although heroku does it, I'm not sure we should.

Since it's ruby and rspec specific, it seems like an inflexible solution for other test frameworks and langs.

I'm not adverse to something like this, but I would prefer a log file parser which we can extend for other langs and test frameworks.

Maybe we can talk about it further when I am back from holiday, typing on an iPhone is a pain :)


Sent from my Sega Master System

On 5/09/2011, at 2:56 PM, ndbroadbentreply@reply.github.com wrote:

The idea comes from Heroku, who do code injection for Rails apps.

One example of this is rails_log_stdout, a plugin injected into Rails apps to cause their logs to go to stdout, as is required for routing logs into the logging system.

There's probably a heroku dev out there who could give us some pointers, although it doesn't seem like vendoring an extra plugin would be too dangerous.

ci_reporter extends RSpec with a new formatter, so as far as I'm aware it can't be used to post-process the saved log output.

Reply to this email directly or view it on GitHub:
#239 (comment)

Member

joshk commented Sep 5, 2011

Although heroku does it, I'm not sure we should.

Since it's ruby and rspec specific, it seems like an inflexible solution for other test frameworks and langs.

I'm not adverse to something like this, but I would prefer a log file parser which we can extend for other langs and test frameworks.

Maybe we can talk about it further when I am back from holiday, typing on an iPhone is a pain :)


Sent from my Sega Master System

On 5/09/2011, at 2:56 PM, ndbroadbentreply@reply.github.com wrote:

The idea comes from Heroku, who do code injection for Rails apps.

One example of this is rails_log_stdout, a plugin injected into Rails apps to cause their logs to go to stdout, as is required for routing logs into the logging system.

There's probably a heroku dev out there who could give us some pointers, although it doesn't seem like vendoring an extra plugin would be too dangerous.

ci_reporter extends RSpec with a new formatter, so as far as I'm aware it can't be used to post-process the saved log output.

Reply to this email directly or view it on GitHub:
#239 (comment)

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Sep 5, 2011

Contributor

Sorry if it was misleading to only mention RSpec, but ci_reporter handles Test::Unit, RSpec and Cucumber. It produces XML output that can be read by any continuous integration system that understands Ant’s JUnit report XML format, such as Hudson or Atlassian Bamboo.

Ant's JUnit XML format is a standard output for almost every test framework:

So I guess this issue is really about handling that XML format, and the ci_reporter injection might be a good first step in that direction. While parsing log files is possible, I don't think it's the ideal solution although it could be a fall-back.

Alright, well thanks for your response, and I'm happy to leave it there unless someone else wants to join the discussion.
I'll also update this issue with all of this information.

Have a good holiday!

Contributor

ndbroadbent commented Sep 5, 2011

Sorry if it was misleading to only mention RSpec, but ci_reporter handles Test::Unit, RSpec and Cucumber. It produces XML output that can be read by any continuous integration system that understands Ant’s JUnit report XML format, such as Hudson or Atlassian Bamboo.

Ant's JUnit XML format is a standard output for almost every test framework:

So I guess this issue is really about handling that XML format, and the ci_reporter injection might be a good first step in that direction. While parsing log files is possible, I don't think it's the ideal solution although it could be a fall-back.

Alright, well thanks for your response, and I'm happy to leave it there unless someone else wants to join the discussion.
I'll also update this issue with all of this information.

Have a good holiday!

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Sep 5, 2011

Member

Ok, you sold me, we should get this to work somehow.

Let's talk when I get back and see how to get this to work in a clean way.

Thanks a bundle,

J


Sent from my Sega Master System

On 5/09/2011, at 5:03 PM, ndbroadbentreply@reply.github.com wrote:

Sorry if it was misleading to only mention RSpec, but ci_reporter handles Test::Unit, RSpec and Cucumber. It produces XML output that can be read by any continuous integration system that understands Ant’s JUnit report XML format, such as Hudson or Atlassian Bamboo.

Ant's JUnit XML format is a standard output for almost every test framework:

So I guess this issue is really about handling that XML format, and the ci_reporter injection might be a good first step in that direction. While parsing log files is possible, I don't think it's the ideal solution although it could be a fall-back.

Alright, well thanks for your response, and I'm happy to leave it there unless someone else wants to join the discussion.
I'll also update this issue with all of this information.

Have a good holiday!

Reply to this email directly or view it on GitHub:
#239 (comment)

Member

joshk commented Sep 5, 2011

Ok, you sold me, we should get this to work somehow.

Let's talk when I get back and see how to get this to work in a clean way.

Thanks a bundle,

J


Sent from my Sega Master System

On 5/09/2011, at 5:03 PM, ndbroadbentreply@reply.github.com wrote:

Sorry if it was misleading to only mention RSpec, but ci_reporter handles Test::Unit, RSpec and Cucumber. It produces XML output that can be read by any continuous integration system that understands Ant’s JUnit report XML format, such as Hudson or Atlassian Bamboo.

Ant's JUnit XML format is a standard output for almost every test framework:

So I guess this issue is really about handling that XML format, and the ci_reporter injection might be a good first step in that direction. While parsing log files is possible, I don't think it's the ideal solution although it could be a fall-back.

Alright, well thanks for your response, and I'm happy to leave it there unless someone else wants to join the discussion.
I'll also update this issue with all of this information.

Have a good holiday!

Reply to this email directly or view it on GitHub:
#239 (comment)

@svenfuchs

This comment has been minimized.

Show comment
Hide comment
@svenfuchs

svenfuchs Sep 7, 2011

Member

I have a couple of questions about this.

  • First and foremost I don't understand who's gonna use this. I'm not aware of people asking for JUnit report XML format. So why would we support it? Just so?
  • Why can't we do whatever ci_reporter does and re-format the build output on the server side? Because we'd reimplement things?
  • If we use ci_reporter on the worker side, will the resulting build log be written to stdout? Or do we have to extend the worker to pass along other build artefacts?
Member

svenfuchs commented Sep 7, 2011

I have a couple of questions about this.

  • First and foremost I don't understand who's gonna use this. I'm not aware of people asking for JUnit report XML format. So why would we support it? Just so?
  • Why can't we do whatever ci_reporter does and re-format the build output on the server side? Because we'd reimplement things?
  • If we use ci_reporter on the worker side, will the resulting build log be written to stdout? Or do we have to extend the worker to pass along other build artefacts?
@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Sep 7, 2011

Contributor

Hi, happy to answer those questions:

  • Firstly, it would be us who would be consuming the XML output of the test suite. Instead of having to write parsers for all the different log formats out there, we would be able to directly import the test results in a consistent, machine readable format. We wouldn't be providing the XML itself to the users.. We would be importing it, and providing users with a UI to show how many times each test has failed / passed. For example, we could mention which tests were fixed when the build most recently passed.
  • ci_reporter actually integrates with Test::Unit, RSpec and Cucumber, to provide the raw test results in an XML format. It doesn't just reformat the human readable logs... It provides deeper information than what is visible in the build logs. And as I mentioned above, if we build in support for this XML file format, then we would be supporting multiple test frameworks for multiple languages.
  • I was thinking that ci_reporter's output would be invisible to the user, while the human readable logs would continue to be displayed. So yes, we would have to extend the worker to pass along the XML file as a build artifact.

The issue is about obtaining more fine-grained information about the build, instead of just a 0 or 1 exit status.
It's great that Travis CI can email me to tell me that something broken. But it would be fantastic if the notification email and the UI could tell me that "these tests have broken in these files, and at these line numbers. Here are the colorized backtraces."

Contributor

ndbroadbent commented Sep 7, 2011

Hi, happy to answer those questions:

  • Firstly, it would be us who would be consuming the XML output of the test suite. Instead of having to write parsers for all the different log formats out there, we would be able to directly import the test results in a consistent, machine readable format. We wouldn't be providing the XML itself to the users.. We would be importing it, and providing users with a UI to show how many times each test has failed / passed. For example, we could mention which tests were fixed when the build most recently passed.
  • ci_reporter actually integrates with Test::Unit, RSpec and Cucumber, to provide the raw test results in an XML format. It doesn't just reformat the human readable logs... It provides deeper information than what is visible in the build logs. And as I mentioned above, if we build in support for this XML file format, then we would be supporting multiple test frameworks for multiple languages.
  • I was thinking that ci_reporter's output would be invisible to the user, while the human readable logs would continue to be displayed. So yes, we would have to extend the worker to pass along the XML file as a build artifact.

The issue is about obtaining more fine-grained information about the build, instead of just a 0 or 1 exit status.
It's great that Travis CI can email me to tell me that something broken. But it would be fantastic if the notification email and the UI could tell me that "these tests have broken in these files, and at these line numbers. Here are the colorized backtraces."

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Sep 7, 2011

Contributor

This is an example of the kind of interface that we could offer (screenshot from Hudson):

Hudson example

Contributor

ndbroadbent commented Sep 7, 2011

This is an example of the kind of interface that we could offer (screenshot from Hudson):

Hudson example

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Sep 11, 2011

Member

REALLY LOVE THIS!!! :)

Member

joshk commented Sep 11, 2011

REALLY LOVE THIS!!! :)

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Feb 10, 2012

Contributor

I'm really stoked to see the future plans for Travis CI on https://love.travis-ci.org/! I saw the 'Build artifacts' section under 'Future Plans', and was reminded of this feature.

I still really think that it would be awesome if Travis CI could parse the formatted build output, and be able to report that "3 tests failed from the RSpec test suite".

It would still be up to the developers to ensure that the test results are formatted correctly and saved in the right place. Travis could detect the presence of these formatted test results and parse them.
This would result in two levels of reporting: Pass/Fail (which we have now), plus more in-depth details (if available), such as which tests were fixed and which were broken.

I need to catch up on the changes to the architecture, but I'd love to discuss the implementation further and help to build this :)

Contributor

ndbroadbent commented Feb 10, 2012

I'm really stoked to see the future plans for Travis CI on https://love.travis-ci.org/! I saw the 'Build artifacts' section under 'Future Plans', and was reminded of this feature.

I still really think that it would be awesome if Travis CI could parse the formatted build output, and be able to report that "3 tests failed from the RSpec test suite".

It would still be up to the developers to ensure that the test results are formatted correctly and saved in the right place. Travis could detect the presence of these formatted test results and parse them.
This would result in two levels of reporting: Pass/Fail (which we have now), plus more in-depth details (if available), such as which tests were fixed and which were broken.

I need to catch up on the changes to the architecture, but I'd love to discuss the implementation further and help to build this :)

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Feb 10, 2012

Member

Hey @ndbroadbent,

It is great to hear that you are excited to get involved with this feature!

I do think this feature could grow into a monster, so many we should start planning out some simple first steps.

The simpler the better!

Let me know what you think,

Josh

Member

joshk commented Feb 10, 2012

Hey @ndbroadbent,

It is great to hear that you are excited to get involved with this feature!

I do think this feature could grow into a monster, so many we should start planning out some simple first steps.

The simpler the better!

Let me know what you think,

Josh

@michaelklishin

This comment has been minimized.

Show comment
Hide comment
@michaelklishin

michaelklishin Feb 10, 2012

Contributor

First question we need to answer is what application this logic should go, worker or Hub or even something new.

Contributor

michaelklishin commented Feb 10, 2012

First question we need to answer is what application this logic should go, worker or Hub or even something new.

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Feb 10, 2012

Contributor

Parsing and storing the test results will only be a small part of the work. Travis CI will be affected in many different places:

  • Travis Worker will need to send the test results XML to Travis Hub (or separate app)
  • Travis Hub (or separate app) will need to process these results and store them with the build.
  • Travis Hub will also need to deliver notifications with information from these test results.
  • Travis Server will need to provide an interface to the test results, as well as the build log.

So, I think it would be best for the test results to be sent from the Worker, and processed in Travis Hub. That processing code could live in a separate library. The test results would need to be processed before delivering notifications, because we'd want notifications to contain that information. (e.g. "5 tests failed").

This means that the build log would still be displayed in the browser while the build is running, but after the build has finished, more fine-grained information about the test suite is notified and displayed.

The following paragraph also seems to be very relevant to this discussion:

It is fair to say that Travis Hub is the most complex of all Travis CI applications we have today. This is why we carefully evaluate what features are worth adding to the Hub: many features simply won’t be worth the complexity.

I would say that a CI system is mainly about tests, so I think it makes sense for us to provide more information than just pass / fail.

Contributor

ndbroadbent commented Feb 10, 2012

Parsing and storing the test results will only be a small part of the work. Travis CI will be affected in many different places:

  • Travis Worker will need to send the test results XML to Travis Hub (or separate app)
  • Travis Hub (or separate app) will need to process these results and store them with the build.
  • Travis Hub will also need to deliver notifications with information from these test results.
  • Travis Server will need to provide an interface to the test results, as well as the build log.

So, I think it would be best for the test results to be sent from the Worker, and processed in Travis Hub. That processing code could live in a separate library. The test results would need to be processed before delivering notifications, because we'd want notifications to contain that information. (e.g. "5 tests failed").

This means that the build log would still be displayed in the browser while the build is running, but after the build has finished, more fine-grained information about the test suite is notified and displayed.

The following paragraph also seems to be very relevant to this discussion:

It is fair to say that Travis Hub is the most complex of all Travis CI applications we have today. This is why we carefully evaluate what features are worth adding to the Hub: many features simply won’t be worth the complexity.

I would say that a CI system is mainly about tests, so I think it makes sense for us to provide more information than just pass / fail.

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Feb 10, 2012

Contributor

Note: The following comment is probably way out of scope for now, and is probably the 'monster' you were talking about...

In addition to parsing the XML build results at the end of each build, we could also provide developers with test formatters, which would allow the Travis Worker to parse the stdout stream on the fly. Travis Worker could detect that the stream is parseable (perhaps by a known header and footer), and then send these results to Travis Hub as a test results chunk message (instead of build log chunk).
We could start small, by only supporting Test::Unit and RSpec.

Contributor

ndbroadbent commented Feb 10, 2012

Note: The following comment is probably way out of scope for now, and is probably the 'monster' you were talking about...

In addition to parsing the XML build results at the end of each build, we could also provide developers with test formatters, which would allow the Travis Worker to parse the stdout stream on the fly. Travis Worker could detect that the stream is parseable (perhaps by a known header and footer), and then send these results to Travis Hub as a test results chunk message (instead of build log chunk).
We could start small, by only supporting Test::Unit and RSpec.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Feb 10, 2012

Member

I think we need to make this even simpler for now, for example, instead of sending extra info in the email lets add that feature later.

Also, instead of parsing the XML or related output, lets just have it either displayable in its raw format or have it downloadable.

Try to break this down into smaller simpler features which we can build off and enhance.

Member

joshk commented Feb 10, 2012

I think we need to make this even simpler for now, for example, instead of sending extra info in the email lets add that feature later.

Also, instead of parsing the XML or related output, lets just have it either displayable in its raw format or have it downloadable.

Try to break this down into smaller simpler features which we can build off and enhance.

@michaelklishin

This comment has been minimized.

Show comment
Hide comment
@michaelklishin

michaelklishin Feb 10, 2012

Contributor

@ndbroadbent keep in mind that workers stream the output more or less as it comes in, this is why near real-time UI is possible. So parsing XML in the worker is not an option (making workers too aware of what they stream and how that data is used is probably a bad idea). Adding this kind of functionality (build log analysis) to the Hub is OK (especially if this can be isolated enough to be an internal library) but this needs to be done carefully.

I agree with what @joshk says. Not sure what's the minimum viable feature set here, maybe we can discuss this first?

Contributor

michaelklishin commented Feb 10, 2012

@ndbroadbent keep in mind that workers stream the output more or less as it comes in, this is why near real-time UI is possible. So parsing XML in the worker is not an option (making workers too aware of what they stream and how that data is used is probably a bad idea). Adding this kind of functionality (build log analysis) to the Hub is OK (especially if this can be isolated enough to be an internal library) but this needs to be done carefully.

I agree with what @joshk says. Not sure what's the minimum viable feature set here, maybe we can discuss this first?

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Feb 10, 2012

Contributor

I also think that's a good idea. In fact, I think that's how our CI system operates (Atlassian Bamboo). IIRC, it treats the XML file as a build artifact, and processes the test results after retrieving it.

Alright, then let's come back to this issue when Travis supports build artifacts. :)

Contributor

ndbroadbent commented Feb 10, 2012

I also think that's a good idea. In fact, I think that's how our CI system operates (Atlassian Bamboo). IIRC, it treats the XML file as a build artifact, and processes the test results after retrieving it.

Alright, then let's come back to this issue when Travis supports build artifacts. :)

@michaelklishin

This comment has been minimized.

Show comment
Hide comment
@michaelklishin

michaelklishin Feb 10, 2012

Contributor

@ndbroadbent please try putting together a minimum viable feature design. This issue is very much related to build artifacts and there are many open questions right now. Coming up with a small but already useful solution will help pushing both features (and probably some more related ones).

It is not something we need to come up with the next week but good ideas often come after you spend some time thinking about something :)

Contributor

michaelklishin commented Feb 10, 2012

@ndbroadbent please try putting together a minimum viable feature design. This issue is very much related to build artifacts and there are many open questions right now. Coming up with a small but already useful solution will help pushing both features (and probably some more related ones).

It is not something we need to come up with the next week but good ideas often come after you spend some time thinking about something :)

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Feb 10, 2012

Member

@ndbroadbent Just confirming though, are you interested in helping us with Build Artifacts?

Member

joshk commented Feb 10, 2012

@ndbroadbent Just confirming though, are you interested in helping us with Build Artifacts?

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Feb 10, 2012

Contributor

@michaelklishin - Sure, I will look into it.

@joshk - Yes, very interested in helping. Is there something I can read to learn about the current plans & discussion? Or just hang out on IRC more often?

Contributor

ndbroadbent commented Feb 10, 2012

@michaelklishin - Sure, I will look into it.

@joshk - Yes, very interested in helping. Is there something I can read to learn about the current plans & discussion? Or just hang out on IRC more often?

@michaelklishin

This comment has been minimized.

Show comment
Hide comment
@michaelklishin

michaelklishin Feb 10, 2012

Contributor

Hanging around the irc is the best way.

MK

El 10/feb/2012, a las 14:01, Nathan Breply@reply.github.com escribió:

Is there something I should read, to find out more about the current plans & discussion? Or just hang out on IRC more often?

Contributor

michaelklishin commented Feb 10, 2012

Hanging around the irc is the best way.

MK

El 10/feb/2012, a las 14:01, Nathan Breply@reply.github.com escribió:

Is there something I should read, to find out more about the current plans & discussion? Or just hang out on IRC more often?

@ndbroadbent

This comment has been minimized.

Show comment
Hide comment
@ndbroadbent

ndbroadbent Mar 30, 2012

Contributor

Hey, sorry guys, I'm going to have to back down from my offer to help with the build artifacts :(
It feels like it's a bit too much for me to take on at the moment, but I'd still like to contribute to Travis from time to time.
I'm also just about to switch from full-time work to freelancing, so I might have some more free time to donate in the future!

Sorry again for not following up on this sooner, and all the best with your work on Travis CI pro!

Contributor

ndbroadbent commented Mar 30, 2012

Hey, sorry guys, I'm going to have to back down from my offer to help with the build artifacts :(
It feels like it's a bit too much for me to take on at the moment, but I'd still like to contribute to Travis from time to time.
I'm also just about to switch from full-time work to freelancing, so I might have some more free time to donate in the future!

Sorry again for not following up on this sooner, and all the best with your work on Travis CI pro!

@svenfuchs

This comment has been minimized.

Show comment
Hide comment
@svenfuchs

svenfuchs Mar 30, 2012

Member

No worries, Nathan. That's how it works with OSS projects.

Thanks for letting us know!

On Mar 30, 2012, at 6:41 PM, Nathan B wrote:

Hey, sorry guys, I'm going to have to back down from my offer to help with the build artifacts :(
It feels like it's a bit too much for me to take on at the moment, but I'd still like to contribute to Travis from time to time.
I'm also just about to switch from full-time work to freelancing, so I might have some more free time to donate in the future!

Sorry again for not following up on this sooner, and all the best with your work on Travis CI pro!


Reply to this email directly or view it on GitHub:
#239 (comment)

Member

svenfuchs commented Mar 30, 2012

No worries, Nathan. That's how it works with OSS projects.

Thanks for letting us know!

On Mar 30, 2012, at 6:41 PM, Nathan B wrote:

Hey, sorry guys, I'm going to have to back down from my offer to help with the build artifacts :(
It feels like it's a bit too much for me to take on at the moment, but I'd still like to contribute to Travis from time to time.
I'm also just about to switch from full-time work to freelancing, so I might have some more free time to donate in the future!

Sorry again for not following up on this sooner, and all the best with your work on Travis CI pro!


Reply to this email directly or view it on GitHub:
#239 (comment)

@DracoBlue

This comment has been minimized.

Show comment
Hide comment
@DracoBlue

DracoBlue Apr 2, 2012

+1! Would be really cool to have this feature!

+1! Would be really cool to have this feature!

@fread2281

This comment has been minimized.

Show comment
Hide comment
@fread2281

fread2281 Jan 3, 2013

Would be wonderful to have this, another idea is to display a grid of tests (by job or time) and color the cells based on the test status. Also python's nose has junit support

Would be wonderful to have this, another idea is to display a grid of tests (by job or time) and color the cells based on the test status. Also python's nose has junit support

@jelder

This comment has been minimized.

Show comment
Hide comment
@jelder

jelder Jul 3, 2014

+1

Reading the logs in Travis is a very unpleasant experience. Given the choice, I'm sure most people would rather consume a semantic, structured test result like would be provided by Travis' consumption of JUnit XML.

jelder commented Jul 3, 2014

+1

Reading the logs in Travis is a very unpleasant experience. Given the choice, I'm sure most people would rather consume a semantic, structured test result like would be provided by Travis' consumption of JUnit XML.

@dexterhu

This comment has been minimized.

Show comment
Hide comment
@dexterhu

dexterhu Jul 10, 2014

+1
We need test report visualization somewhere online for each test run.

+1
We need test report visualization somewhere online for each test run.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Jul 11, 2014

Member

We have something in the works for this, but I don't have an ETA sorry.

Member

joshk commented Jul 11, 2014

We have something in the works for this, but I don't have an ETA sorry.

@bkimminich

This comment has been minimized.

Show comment
Hide comment

ping 👍

@alexkalderimis

This comment has been minimized.

Show comment
Hide comment
@alexkalderimis

alexkalderimis Nov 25, 2014

I would be happy to help with this if there is still interest. We have our own system, but it is hacky.

I would be happy to help with this if there is still interest. We have our own system, but it is hacky.

@marscher

This comment has been minimized.

Show comment
Hide comment
@marscher

marscher Dec 11, 2014

this would be an extremely useful feature. E.g Appveyor provides an API to upload xunit.xml (.net testing output xml) and displays it nicely on a results page. Directly seeing which tests brake eg. for a pull request is a killer feature for maintainers, as they could give direct advice to commiters, what might be wrong.

this would be an extremely useful feature. E.g Appveyor provides an API to upload xunit.xml (.net testing output xml) and displays it nicely on a results page. Directly seeing which tests brake eg. for a pull request is a killer feature for maintainers, as they could give direct advice to commiters, what might be wrong.

@dguido

This comment has been minimized.

Show comment
Hide comment
@dguido

dguido Dec 19, 2014

👍 I would pay for this feature.

dguido commented Dec 19, 2014

👍 I would pay for this feature.

@flozano

This comment has been minimized.

Show comment
Hide comment

flozano commented Dec 28, 2014

👍

@travis-ci travis-ci locked and limited conversation to collaborators Dec 28, 2014

@roidrage

This comment has been minimized.

Show comment
Hide comment
@roidrage

roidrage Jul 23, 2015

Member

Closing this issue for now, as we have no immediate plans to add this feature.

Should we add it to the roadmap eventually, we'll make sure to update this ticket.

Member

roidrage commented Jul 23, 2015

Closing this issue for now, as we have no immediate plans to add this feature.

Should we add it to the roadmap eventually, we'll make sure to update this ticket.

@roidrage roidrage closed this Jul 23, 2015

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.