[Feature Request] Update diff with Linter and Unit Test information

[lavahot]

Not sure if this is what the Phabricator workflow should be used for, but I think I'd like the option to publish the results of linting and unit tests performed on Jenkins as a part of a Differential build to that differential build in the lint and unit test sections of that diff.

[ascandella]

Thanks for the request, and I'll leave this open as this is something we want as well.

The reality is that currently harbormaster is in early beta, (not even v1 yet), and currently only supports pass/fail. There are many tasks to track improvements to this, such as being able to report lint/unit results over harbormaster (https://secure.phabricator.com/T8097 https://secure.phabricator.com/T8095 https://secure.phabricator.com/T6139) but those aren't there yet.

To work around this, we currently provide a "comment" option, documented https://github.com/uber/phabricator-jenkins-plugin#additional-comments -- it's pretty rudimentary, but we use it to provide reasons for build failures in our jobs. So for example, my Go project might have a test script that looks like:

#!/bin/bash
set -ex -o pipefail

PHABRICATOR_COMMENT=".phabricator-comment"
gofmt -s -e -d . | tee -a $PHABRICATOR_COMMENT
golint ./... | tee -a $PHABRICATOR_COMMENT

go build ./... | tee -a $PHABRICATOR_COMMENT

That way, the failure of any one of those commands gets reported back by jenkins:

I know it's kinda hacky, but maybe could be useful in the interim until Harbormaster is more mature?

[ascandella]

By the way, I realize now that I forgot to document this on the Jenkins wiki page. It's now there under https://wiki.jenkins-ci.org/display/JENKINS/Phabricator+Plugin#PhabricatorPlugin-PostingResultstoPhabricator -- feel free to edit if you find this useful / see any issues.

[ascandella]

Looks like this just landed in https://secure.phabricator.com/D13380 -- will look at updating this soon.

[ascandella]

@lavahot what format are your test results in? We use jUnit at Uber, want to make sure that will cover your case as well.

[ChaitanyaPramod]

We have a similar requirement for our project. We also use jUnit.

[cburroughs]

https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin knows how to transform most things into junit-esque format for jenkins. It's possible there is something re-usable there.

[ascandella]

Just underwent some heavy refactoring in #60 -- once that's landed, we should be clear to add unit/lint/coverage results. I'll probably target jUnit initially, since that's going to be a hard requirement at Uber, but leave it open to extensibility.

I won't have time to work on this for a few days though, so if somebody else wants to take a look, feel free.

[ascandella]

Refactor is landed and released. Next up is this task. I'm out of town this weekend so will start looking at this early next week.

[ascandella]

I started sketching this out in https://github.com/uber/phabricator-jenkins-plugin/tree/junit-test-results (d7a0221)

I have a ton of work this week so I may not be able to look at this for a little bit.

[ascandella]

Started work on using the new Harbormaster APIs in #90

[ascandella]

This was released today in version 1.8.0 of the plugin