Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upSupport for TeamCity Test history via Teamcity Service Messages #44813
Conversation
gilescope
added some commits
Sep 22, 2017
rust-highfive
assigned
aturon
Sep 24, 2017
This comment has been minimized.
This comment has been minimized.
|
r? @aturon (rust_highfive has picked a reviewer for you, use r? to override) |
carols10cents
added
the
S-waiting-on-review
label
Sep 25, 2017
This comment has been minimized.
This comment has been minimized.
rust-highfive
assigned
alexcrichton
and unassigned
aturon
Sep 25, 2017
This comment has been minimized.
This comment has been minimized.
|
Thanks for the PR! Integration with existing test frameworks has long been a desire for us and historically hasn't seen much action unfortunately. At this time though I think we're not quite ready to accept a PR like this. We'd ideally not be adding one-off support for all sorts of test suites and frameworks to libtest, but rather we'd have a more fleshed out design of where we want to go to start out with. We've thought about this in the past and it tends to boil down to "bring your own libtest" rather than adding support to libtest directly itself. I realize though this probably isn't the response you wanted :(. There's some more info at rust-lang/rfcs#816 as well I believe. I'm curious about what you think about this? |
This comment has been minimized.
This comment has been minimized.
|
Totally understand - hence wanted an early review. Ideally we want to split test runners from test listeners. We'd like to be agnostic of the test-runner for team city service messages. The testing framework should have a logging/reporting interface and the teamcity service messages could be one such listener (the console and crate cargo-test-junit being other implementations of test listeners). Is it currently possible to switch out the test runner for another? It feels a little like bench is a separate test runner. |
This comment has been minimized.
This comment has been minimized.
|
(Btw, on a separate note, just want to say how much I love that when you compile tests the compiler complains about unused code that isn't referenced from the tests. It's effectively a compile time coverage tool - super cool.) |
This comment has been minimized.
This comment has been minimized.
|
I indeed share many of the same desires! Unfortunately right now though it's not possible to switch out the test runner, that's mostly what rust-lang/rfcs#816 is targeted at. |
This comment has been minimized.
This comment has been minimized.
|
@alexcrichton so should this PR be closed, pending resolution to rust-lang/rfcs#816? |
gilescope commentedSep 24, 2017
This PR enables Teamcity Service Messages ( https://confluence.jetbrains.com/display/TCD10/Build+Script+Interaction+with+TeamCity ) to be output when running tests or benchmarks. This enables easy integration of Rust projects into TeamCity and hopefully enables easier test runner integration into Intelli Rust ( #43381 ). This will help Rust be easily integrated into existing enterprise environments.
One neat touch is that the rust benchmarks are turned into TeamCity Stats and can then be easily graphed over time through the TeamCity UI.
To turn on having a '--teamcity' option like NUnit3 or by having an environment variable set (RUST_TEST_TEAMCITY_SERVICE_MESSAGES=1) should turn these on. They should not be on by default.
(I believe no corresponding changes to the Cargo project need be made.)
All feedback welcome!