Update documentation: Add how to generate HTML coverage report from unit tests

[alexellis]

Steps to reproduce the issue:

When making test-unit I saw a lot of useful information about code paths and coverage percentages. I'd like to generate a HTML coverage report to view in a browser and see if I can contribute by increasing coverage somewhere. I.e.

---> Making bundle: test-unit (in bundles/1.12.0-dev/test-unit)
Sun May 22 16:22:42 UTC 2016
ok      github.com/docker/docker/api    0.026s  coverage: 92.5% of statements
ok      github.com/docker/docker/api/client 0.010s  coverage: 1.3% of statements
ok      github.com/docker/docker/api/client/formatter   0.016s  coverage: 93.5% of statements
ok      github.com/docker/docker/api/client/inspect 0.011s  coverage: 92.7% of statements
ok      github.com/docker/docker/api/server 0.008s  coverage: 6.7% of statements
ok      github.com/docker/docker/api/server/httputils   0.012s  coverage: 19.2% of statements
ok      github.com/docker/docker/api/server/middleware  0.020s  coverage: 22.0% of statements

Describe the results you received:

Percentages on the console

Describe the results you expected:

An option or flag described at https://docs.docker.com/opensource/project/test-and-docs/ to give a html report such as go tool cover -html=coverage.out found at:

https://blog.golang.org/cover

Since the tests are run through Make and hack/make.sh I was not sure if it was easy to pass the flag in through TESTARGS or TESTFLAGS and also wasn't sure where the appropriate place was to ask a question like this.

[AkihiroSuda]

I'm working on a PR to support emitting the coverage profile in hack/make/test-unit

[thaJeztah]

I think there were also plans to integrate https://codecov.io, but not sure someone is currently working on that

[alexellis]

I'd like to help test the change when you have it ready. I was also thinking of having Jenkins output the code coverage to its workspace for viewing that way?

[AkihiroSuda]

opened PR #22930

[tianon]

hack/make/test-unit should be emitting coverprofiles (for quite some time now; at least as far back as #3211), and if it isn't, that's a bug 😞 (and I'm keen on understanding what's happening to cause it and why)

We also have hack/make/cover which is supposed to be responsible for creating reports based on those (so something like ./hack/make.sh test-unit cover should create bundles/cover/ filled with code coverage reports).

I've just tested (via make shell followed by ./hack/make.sh test-unit cover, which fails), and I can verify that the current support is not working, but I think there's still value in figuring out exactly why it's not working (so we can fix it appropriately, or discuss/decide how it should work instead). 😢

[tianon]

Aha: #17491

this disabled coverage profiles generation

[tianon]

So to summarize:

1. we did support generating cover profiles once (hack/make/cover is a leftover of that)
2. Run unit tests as single command #17491 intentionally disabled them

I think we have a couple options here:

1. adjust documentation about how to use TESTFLAGS to get cover profiles (and remove hack/make/cover)
2. adjust hack/make/cover to add to TESTFLAGS, invoke test-unit, and then generate reports (probably should then rename to hack/make/cover-test-unit or hack/make/test-unit-coverage for clarity of purpose)
3. add a new variable to toggle the behavior, making hack/make/cover work again (only including this as a potential option for completion -- I think either of the two previous options are superior at this point)

[bsousaa]

It looks like this issue went stale. Let me close it.