New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage of go test #140

Open
GinFungYJF opened this Issue Oct 13, 2016 · 27 comments

Comments

Projects
None yet
8 participants
@GinFungYJF
Contributor

GinFungYJF commented Oct 13, 2016

Does the go rule still have no support for calculating the coverage of go test now? If so, is there any way to calculate it?

@yugui

This comment has been minimized.

Show comment
Hide comment
@yugui

yugui Oct 13, 2016

Member

It does not support coverage right now.
Probably we need to wait for bazelbuild/bazel#1118.

Member

yugui commented Oct 13, 2016

It does not support coverage right now.
Probably we need to wait for bazelbuild/bazel#1118.

@yugui yugui added the enhancement label Oct 13, 2016

@GinFungYJF

This comment has been minimized.

Show comment
Hide comment
@GinFungYJF

GinFungYJF Dec 1, 2016

Contributor

Any update on this? Or is there any usable workaround like
bazelbuild/bazel#1118 ?

Contributor

GinFungYJF commented Dec 1, 2016

Any update on this? Or is there any usable workaround like
bazelbuild/bazel#1118 ?

@pmbethe09

This comment has been minimized.

Show comment
Hide comment
@pmbethe09

pmbethe09 Dec 1, 2016

Member
Member

pmbethe09 commented Dec 1, 2016

@ghasemloo

This comment has been minimized.

Show comment
Hide comment
@ghasemloo

ghasemloo Jan 13, 2017

Contributor

I think we can fix this directly, go tool itself support coverage independent of bazel.
I think what needs to be done is adding something similar to the block regarding coverage from go test main (https://golang.org/src/cmd/go/test.go?h=cover+flag#L1522)
to generate test main template: (https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go#L129).

I tried it but I don't fully understand how flags are parsed in the go test, they are defined in
https://golang.org/src/cmd/go/testflag.go

Contributor

ghasemloo commented Jan 13, 2017

I think we can fix this directly, go tool itself support coverage independent of bazel.
I think what needs to be done is adding something similar to the block regarding coverage from go test main (https://golang.org/src/cmd/go/test.go?h=cover+flag#L1522)
to generate test main template: (https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go#L129).

I tried it but I don't fully understand how flags are parsed in the go test, they are defined in
https://golang.org/src/cmd/go/testflag.go

@GinFungYJF

This comment has been minimized.

Show comment
Hide comment
@GinFungYJF

GinFungYJF Jan 14, 2017

Contributor

@ghasemloo When I run go test -cover -x pkg, I see that it will do some preproccessing for the files that are not *_test.go files in the pkg before they are compiled.

go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .A.go pkg/A.go
go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .B.go pkg/B.go
...

Is it helpful to you?

Contributor

GinFungYJF commented Jan 14, 2017

@ghasemloo When I run go test -cover -x pkg, I see that it will do some preproccessing for the files that are not *_test.go files in the pkg before they are compiled.

go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .A.go pkg/A.go
go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .B.go pkg/B.go
...

Is it helpful to you?

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang Jan 21, 2017

Contributor

It would be great if the future bazel-go-coverage also supports external-test besides standard unit tests. Here are a few references of this technique:

https://blog.cloudflare.com/go-coverage-with-external-tests/
https://husobee.github.io/golang/test/coverage/2015/11/17/external-test-coverage.html
https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests

I hope I could find time to work on it.

Contributor

linuxerwang commented Jan 21, 2017

It would be great if the future bazel-go-coverage also supports external-test besides standard unit tests. Here are a few references of this technique:

https://blog.cloudflare.com/go-coverage-with-external-tests/
https://husobee.github.io/golang/test/coverage/2015/11/17/external-test-coverage.html
https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests

I hope I could find time to work on it.

@pmbethe09

This comment has been minimized.

Show comment
Hide comment
@pmbethe09

pmbethe09 Jan 23, 2017

Member
Member

pmbethe09 commented Jan 23, 2017

@GinFungYJF

This comment has been minimized.

Show comment
Hide comment
@GinFungYJF

GinFungYJF Mar 16, 2017

Contributor

Anybody has any idea about implementing it ? I don't know where to start :(

Contributor

GinFungYJF commented Mar 16, 2017

Anybody has any idea about implementing it ? I don't know where to start :(

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Mar 16, 2017

Contributor

@GinFungYJF We haven't given this a lot of thought. It seems clear this will be a lot of work though.

Bazel has some support for exposing coverage information to Skylark. ctx.coverage_instrumented() would be called inside go_library_impl to check whether coverage instrumentation is needed. An instrumented_files provider should be returned (I'm not sure how Bazel uses that information though).

The Go code itself needs to be modified to include coverage instrumentation. The cover story provides some information on how this works. It looks like go test inserts coverage instrumentation at the source level before the compiler sees it. I'm not familiar with the implementation of go test, but test.go is the starting point. A lot of that implementation is private, and if it looks like we would need to duplicate it, we should talk to the Go Libraries team about providing a stable API we could use.

If you decide to work on this, please post a design doc that we can discuss and comment on.

Contributor

jayconrod commented Mar 16, 2017

@GinFungYJF We haven't given this a lot of thought. It seems clear this will be a lot of work though.

Bazel has some support for exposing coverage information to Skylark. ctx.coverage_instrumented() would be called inside go_library_impl to check whether coverage instrumentation is needed. An instrumented_files provider should be returned (I'm not sure how Bazel uses that information though).

The Go code itself needs to be modified to include coverage instrumentation. The cover story provides some information on how this works. It looks like go test inserts coverage instrumentation at the source level before the compiler sees it. I'm not familiar with the implementation of go test, but test.go is the starting point. A lot of that implementation is private, and if it looks like we would need to duplicate it, we should talk to the Go Libraries team about providing a stable API we could use.

If you decide to work on this, please post a design doc that we can discuss and comment on.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Mar 16, 2017

Contributor

@GinFungYJF I spoke with @alandonovan, and he filled in some gaps in my knowledge here.

go tool cover is the tool that adds coverage instrumentation to Go sources. That's already part of the standard distribution, so we wouldn't need to re-implement that. We'd just need to generate actions and files inside the rules to make sure that sources are processed with that before they are compiled when coverage is enabled.

Bazel expects coverage information to be in lcov format. We have some code in Blaze (Google-internal version of Bazel) that produces lcov data at the end of a Go program using the counters added by the instrumentation. This is linked into instrumented binaries. We should be able to open source this or re-implement it without too much work.

Contributor

jayconrod commented Mar 16, 2017

@GinFungYJF I spoke with @alandonovan, and he filled in some gaps in my knowledge here.

go tool cover is the tool that adds coverage instrumentation to Go sources. That's already part of the standard distribution, so we wouldn't need to re-implement that. We'd just need to generate actions and files inside the rules to make sure that sources are processed with that before they are compiled when coverage is enabled.

Bazel expects coverage information to be in lcov format. We have some code in Blaze (Google-internal version of Bazel) that produces lcov data at the end of a Go program using the counters added by the instrumentation. This is linked into instrumented binaries. We should be able to open source this or re-implement it without too much work.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang Mar 16, 2017

Contributor

@jayconrod Thank you for providing these info, although I've figured them out by myself. What held me back was actually I could not find a way to elegantly enable it in rules_go:

Suppose we have pkg deps like: A (main) -> B -> C. If we want to enable test coverage for A, we have to add corresponding BUILD targets also for B and C.

go_binary(name = "A",deps = ["B", "C"]
go_binary(name = "A_cover", test_coverage = true, deps = ["B_cover", "C_cover"])
go_library(name = "B")
go_library(name = "B_cover", test_coverage = true)
go_library(name = "C")
go_library(name = "C_cover", test_coverage = true)

This is too cumbersome. The ideal way is as follows:

go_binary(name = "A",deps = ["B", "C"]
go_test_coverage(name = "A_cover", deps = ["A"])

But it's probably impossible in bazel, since if A has been built before A_cover, A will not be triggered by A_cover to rebuild.

Contributor

linuxerwang commented Mar 16, 2017

@jayconrod Thank you for providing these info, although I've figured them out by myself. What held me back was actually I could not find a way to elegantly enable it in rules_go:

Suppose we have pkg deps like: A (main) -> B -> C. If we want to enable test coverage for A, we have to add corresponding BUILD targets also for B and C.

go_binary(name = "A",deps = ["B", "C"]
go_binary(name = "A_cover", test_coverage = true, deps = ["B_cover", "C_cover"])
go_library(name = "B")
go_library(name = "B_cover", test_coverage = true)
go_library(name = "C")
go_library(name = "C_cover", test_coverage = true)

This is too cumbersome. The ideal way is as follows:

go_binary(name = "A",deps = ["B", "C"]
go_test_coverage(name = "A_cover", deps = ["A"])

But it's probably impossible in bazel, since if A has been built before A_cover, A will not be triggered by A_cover to rebuild.

@pmbethe09

This comment has been minimized.

Show comment
Hide comment
@pmbethe09
Member

pmbethe09 commented Mar 16, 2017

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Mar 17, 2017

Contributor

@linuxerwang Ideally, you shouldn't need to define separate targets with and without coverage; it should be a configuration option. Bazel already has a coverage command that runs tests in coverage mode. We just need to wire up the rules to instrument code when that option is enabled and generate output in the appropriate format.

Contributor

jayconrod commented Mar 17, 2017

@linuxerwang Ideally, you shouldn't need to define separate targets with and without coverage; it should be a configuration option. Bazel already has a coverage command that runs tests in coverage mode. We just need to wire up the rules to instrument code when that option is enabled and generate output in the appropriate format.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang Mar 17, 2017

Contributor

@pmbethe09, @jayconrod Cool! I'll invest some time on it this weekend.

Contributor

linuxerwang commented Mar 17, 2017

@pmbethe09, @jayconrod Cool! I'll invest some time on it this weekend.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang Mar 19, 2017

Contributor

Alright, I made an early stage experiment, the instrumentation is working. But we need more work on enable the coverage hookup in the generated test main func.

linuxerwang@42f5056

You can run "bazel coverage examples/lib:lib_test". It simply passes, without coverage collected. And the test binary has not coverage enabled:

$ ./bazel-bin/examples/lib/lib_test -test.coverprofile=report.txt
testing: cannot use -test.coverprofile because test binary was not built with coverage enabled

Of course, we also need more work on lcov format mentioned by @jayconrod.

External test coverage might be feasible through aspects, but I didn't get a chance to try.

Contributor

linuxerwang commented Mar 19, 2017

Alright, I made an early stage experiment, the instrumentation is working. But we need more work on enable the coverage hookup in the generated test main func.

linuxerwang@42f5056

You can run "bazel coverage examples/lib:lib_test". It simply passes, without coverage collected. And the test binary has not coverage enabled:

$ ./bazel-bin/examples/lib/lib_test -test.coverprofile=report.txt
testing: cannot use -test.coverprofile because test binary was not built with coverage enabled

Of course, we also need more work on lcov format mentioned by @jayconrod.

External test coverage might be feasible through aspects, but I didn't get a chance to try.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Mar 20, 2017

Contributor

@linuxerwang Thanks for working on this, it looks promising. A couple comments:

  • It would be better to run go tool cover through an action for each file. This will let Bazel parallelize the coverage transformation and cache the results. Basically, define a new Skylark function which takes a list of sources as an argument, creates new files for the transformed sources (ctx.new_file), runs go tool cover for each of them (ctx.action), then returns the transformed files. _emit_go_compile_action can call this function if coverage is enabled and use the result as a replacement for srcs, before calling symlink_tree_commands.
  • Any new Skylark functions in def.bzl should be private (name starts with '_'). This prevents other projects from depending on implementation details.
Contributor

jayconrod commented Mar 20, 2017

@linuxerwang Thanks for working on this, it looks promising. A couple comments:

  • It would be better to run go tool cover through an action for each file. This will let Bazel parallelize the coverage transformation and cache the results. Basically, define a new Skylark function which takes a list of sources as an argument, creates new files for the transformed sources (ctx.new_file), runs go tool cover for each of them (ctx.action), then returns the transformed files. _emit_go_compile_action can call this function if coverage is enabled and use the result as a replacement for srcs, before calling symlink_tree_commands.
  • Any new Skylark functions in def.bzl should be private (name starts with '_'). This prevents other projects from depending on implementation details.
@ghasemloo

This comment has been minimized.

Show comment
Hide comment
@ghasemloo

ghasemloo Apr 8, 2017

Contributor

what I was suggesting was something similar to what I did for benchmark:
https://github.com/ghasemloo/rules_go/commit/fa2088cd8d3e367902d6d896d1deccfec5da76c2
Essentially modify the main file that is generated by bazel for go tests (https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go) to be similar to what go cover does.

I looked at it a bit but it seemed complicated than I had to time to dig into. I still feels it should be possible to just modify https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go
to make bazel test with something like -- --cover work but I am not completely sure it is really viable or even the right thing to do.

Contributor

ghasemloo commented Apr 8, 2017

what I was suggesting was something similar to what I did for benchmark:
https://github.com/ghasemloo/rules_go/commit/fa2088cd8d3e367902d6d896d1deccfec5da76c2
Essentially modify the main file that is generated by bazel for go tests (https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go) to be similar to what go cover does.

I looked at it a bit but it seemed complicated than I had to time to dig into. I still feels it should be possible to just modify https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go
to make bazel test with something like -- --cover work but I am not completely sure it is really viable or even the right thing to do.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Apr 10, 2017

Contributor

In the long run, it's important for us to be compatible with bazel coverage. That means transforming code in response to ctx.coverage_instrumented() and generating data in lcov format. Bazel has some (rather sparse) documentation on controlling instrumentation in Skylark.

I'm not sure what the current status of this in Bazel is right now. It doesn't seem like it works yet. I'm watching bazelbuild/bazel#1118 for updates though.

Contributor

jayconrod commented Apr 10, 2017

In the long run, it's important for us to be compatible with bazel coverage. That means transforming code in response to ctx.coverage_instrumented() and generating data in lcov format. Bazel has some (rather sparse) documentation on controlling instrumentation in Skylark.

I'm not sure what the current status of this in Bazel is right now. It doesn't seem like it works yet. I'm watching bazelbuild/bazel#1118 for updates though.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang May 14, 2017

Contributor

Finished the change as @jayconrod suggested. Created pull request #455.

Will work on generate_test_main.go in a later pull request.

Contributor

linuxerwang commented May 14, 2017

Finished the change as @jayconrod suggested. Created pull request #455.

Will work on generate_test_main.go in a later pull request.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang May 15, 2017

Contributor

Actually, added the change to test main generator in the same pull request.

For an example run, execute:

$ bazel coverage examples/lib:lib_test

It generates the following files:

$ ls -l bazel-out/local-fastbuild/bin/examples/lib/
total 2016
-r-xr-xr-x 1 linuxerwang linuxerwang     471 May 14 20:57 asm_GoCover_0.go <====
drwxrwxr-x 3 linuxerwang linuxerwang    4096 Mar 18 17:49 bazel-out
drwxrwxr-x 4 linuxerwang linuxerwang    4096 May 14 12:42 deep
drwxrwxr-x 3 linuxerwang linuxerwang    4096 May 13 21:01 github.com
-r-xr-xr-x 1 linuxerwang linuxerwang     623 May 14 20:57 lib_GoCover_1.go <====
-r-xr-xr-x 1 linuxerwang linuxerwang 1931232 May 14 20:57 lib_test
drwxrwxr-x 2 linuxerwang linuxerwang    4096 May 13 21:01 lib_test.dir
-r-xr-xr-x 1 linuxerwang linuxerwang     630 May 14 20:57 lib_test.GoTestGenTest.params
-r-xr-xr-x 1 linuxerwang linuxerwang   22936 May 14 20:57 lib_test_main_test.a
-r-xr-xr-x 1 linuxerwang linuxerwang     876 May 14 20:57 lib_test_main_test.a.GoLinkFile.params
-r-xr-xr-x 1 linuxerwang linuxerwang    1940 May 14 20:57 lib_test_main_test.go
-r-xr-xr-x 1 linuxerwang linuxerwang   22637 May 14 20:57 lib_test_main_test.o
-r-xr-xr-x 1 linuxerwang linuxerwang   32892 May 14 20:57 lib_test.o
drwxrwxr-x 3 linuxerwang linuxerwang    4096 Mar 18 17:49 lib_test.runfiles
-r-xr-xr-x 1 linuxerwang linuxerwang     183 May 14 20:57 lib_test.runfiles_manifest

The instrumented files are named in such a way that the generated test main can extract cover vars easily. The code logic in generate_test_main.go is borrowed shamelessly from $GOROOT/src/cmd/go/test.go and related files.

$ ./bazel-out/local-fastbuild/bin/examples/lib/lib_test
2 + 3 = 5
PASS
coverage: 40.0% of statements

It has the following features unimplemented:

  1. make -test.coverprofile=report.txt work
  2. add lcov output support
  3. make cover mode configurable
  4. add aspect to make external coverage tests work
Contributor

linuxerwang commented May 15, 2017

Actually, added the change to test main generator in the same pull request.

For an example run, execute:

$ bazel coverage examples/lib:lib_test

It generates the following files:

$ ls -l bazel-out/local-fastbuild/bin/examples/lib/
total 2016
-r-xr-xr-x 1 linuxerwang linuxerwang     471 May 14 20:57 asm_GoCover_0.go <====
drwxrwxr-x 3 linuxerwang linuxerwang    4096 Mar 18 17:49 bazel-out
drwxrwxr-x 4 linuxerwang linuxerwang    4096 May 14 12:42 deep
drwxrwxr-x 3 linuxerwang linuxerwang    4096 May 13 21:01 github.com
-r-xr-xr-x 1 linuxerwang linuxerwang     623 May 14 20:57 lib_GoCover_1.go <====
-r-xr-xr-x 1 linuxerwang linuxerwang 1931232 May 14 20:57 lib_test
drwxrwxr-x 2 linuxerwang linuxerwang    4096 May 13 21:01 lib_test.dir
-r-xr-xr-x 1 linuxerwang linuxerwang     630 May 14 20:57 lib_test.GoTestGenTest.params
-r-xr-xr-x 1 linuxerwang linuxerwang   22936 May 14 20:57 lib_test_main_test.a
-r-xr-xr-x 1 linuxerwang linuxerwang     876 May 14 20:57 lib_test_main_test.a.GoLinkFile.params
-r-xr-xr-x 1 linuxerwang linuxerwang    1940 May 14 20:57 lib_test_main_test.go
-r-xr-xr-x 1 linuxerwang linuxerwang   22637 May 14 20:57 lib_test_main_test.o
-r-xr-xr-x 1 linuxerwang linuxerwang   32892 May 14 20:57 lib_test.o
drwxrwxr-x 3 linuxerwang linuxerwang    4096 Mar 18 17:49 lib_test.runfiles
-r-xr-xr-x 1 linuxerwang linuxerwang     183 May 14 20:57 lib_test.runfiles_manifest

The instrumented files are named in such a way that the generated test main can extract cover vars easily. The code logic in generate_test_main.go is borrowed shamelessly from $GOROOT/src/cmd/go/test.go and related files.

$ ./bazel-out/local-fastbuild/bin/examples/lib/lib_test
2 + 3 = 5
PASS
coverage: 40.0% of statements

It has the following features unimplemented:

  1. make -test.coverprofile=report.txt work
  2. add lcov output support
  3. make cover mode configurable
  4. add aspect to make external coverage tests work
@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang May 28, 2017

Contributor

I took a short time trying the coverprofile output. The currently generated test binary has all the mechanism outputting everything:

$ ./bazel-bin/examples/lib/lib_test -test.coverprofile=a.out
2 + 3 = 5
PASS
coverage: 40.0% of statements

$ cat examples/lib/a.out 
mode: set
bazel-out/local-fastbuild/bin/examples/lib/asm_GoCover_0.cover.go:8.22,10.2 1 1
bazel-out/local-fastbuild/bin/examples/lib/asm_GoCover_0.cover.go:12.22,14.2 1 0
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:27.20,29.2 1 1
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:34.23,36.2 1 0
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:39.25,41.2 1 0

However, I am stuck at two problems:

  1. how to pass the argument "-test.coverprofile=a.out" to "bazel coverage" run.
    I can use flag.Value.Set(), but probably a bad idea.

  2. how to put "a.out" in bazel-testlogs/examples/lib/lib_test.

Can bazel team please shed some light on it?

Thanks.

Contributor

linuxerwang commented May 28, 2017

I took a short time trying the coverprofile output. The currently generated test binary has all the mechanism outputting everything:

$ ./bazel-bin/examples/lib/lib_test -test.coverprofile=a.out
2 + 3 = 5
PASS
coverage: 40.0% of statements

$ cat examples/lib/a.out 
mode: set
bazel-out/local-fastbuild/bin/examples/lib/asm_GoCover_0.cover.go:8.22,10.2 1 1
bazel-out/local-fastbuild/bin/examples/lib/asm_GoCover_0.cover.go:12.22,14.2 1 0
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:27.20,29.2 1 1
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:34.23,36.2 1 0
bazel-out/local-fastbuild/bin/examples/lib/lib_GoCover_1.cover.go:39.25,41.2 1 0

However, I am stuck at two problems:

  1. how to pass the argument "-test.coverprofile=a.out" to "bazel coverage" run.
    I can use flag.Value.Set(), but probably a bad idea.

  2. how to put "a.out" in bazel-testlogs/examples/lib/lib_test.

Can bazel team please shed some light on it?

Thanks.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod May 30, 2017

Contributor

If bazel coverage is run with no other arguments, I think it should communicate with Bazel in whatever format it expects (lcov?). I don't think coverage is really well-supported in Bazel yet though: bazelbuild/bazel#1118 is still open, although there has been some activity in the last few months.

You can still explicitly set the -test.coverageprofile flag using --test_arg. I don't think this should be set automatically though.

Contributor

jayconrod commented May 30, 2017

If bazel coverage is run with no other arguments, I think it should communicate with Bazel in whatever format it expects (lcov?). I don't think coverage is really well-supported in Bazel yet though: bazelbuild/bazel#1118 is still open, although there has been some activity in the last few months.

You can still explicitly set the -test.coverageprofile flag using --test_arg. I don't think this should be set automatically though.

@linuxerwang

This comment has been minimized.

Show comment
Hide comment
@linuxerwang

linuxerwang May 30, 2017

Contributor

@jayconrod Question is, after running bazel coverage --test_arg=test.coverprofile=cover.out examples/lib:lib_test, where is the file cover.out generated? I could not find it anywhere.

Contributor

linuxerwang commented May 30, 2017

@jayconrod Question is, after running bazel coverage --test_arg=test.coverprofile=cover.out examples/lib:lib_test, where is the file cover.out generated? I could not find it anywhere.

@Mistobaan

This comment has been minimized.

Show comment
Hide comment
@Mistobaan

Mistobaan Oct 4, 2017

Any progress on how to run coverage profiling using bazel? is it possible as of today? I can run bazel coverage on my tests but then is not clear how I inspect the results.

Mistobaan commented Oct 4, 2017

Any progress on how to run coverage profiling using bazel? is it possible as of today? I can run bazel coverage on my tests but then is not clear how I inspect the results.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Oct 4, 2017

Contributor

No change on this recently. We've been focusing on toolchains, proto support, and dependency management lately. I'm hoping to get coverage into a better state by the end of the year.

Contributor

jayconrod commented Oct 4, 2017

No change on this recently. We've been focusing on toolchains, proto support, and dependency management lately. I'm hoping to get coverage into a better state by the end of the year.

@dkt1412

This comment has been minimized.

Show comment
Hide comment
@dkt1412

dkt1412 Nov 10, 2017

So I ran this (quotes are significant!)
bazel coverage \<test-target\> --test_arg="-test.coverprofile=coverage.out"
and this generated the coverage file (woohoo) at
bazel-bin/\<test-target-path\>/\<target-test\>.runfiles/\<test-target-path\>/coverage.out
The problem I have now is how to turn this into a html report. I can't seem to use go tool cover --html=\<path-to-coverage.out\> b/c main.go doesn't actually live in my GOPATH (which I also can't set up b/c our bazel project doesn't have the go-required src/ setup).

Is there a way to get bazel to run go tool cover --html for us? I assume we'd have to add some sort of actions to link to go tool cover like @linuxerwang did, but it's unclear to me where that needs to go.

dkt1412 commented Nov 10, 2017

So I ran this (quotes are significant!)
bazel coverage \<test-target\> --test_arg="-test.coverprofile=coverage.out"
and this generated the coverage file (woohoo) at
bazel-bin/\<test-target-path\>/\<target-test\>.runfiles/\<test-target-path\>/coverage.out
The problem I have now is how to turn this into a html report. I can't seem to use go tool cover --html=\<path-to-coverage.out\> b/c main.go doesn't actually live in my GOPATH (which I also can't set up b/c our bazel project doesn't have the go-required src/ setup).

Is there a way to get bazel to run go tool cover --html for us? I assume we'd have to add some sort of actions to link to go tool cover like @linuxerwang did, but it's unclear to me where that needs to go.

@jayconrod

This comment has been minimized.

Show comment
Hide comment
@jayconrod

jayconrod Nov 10, 2017

Contributor

Sorry, we still don't have a good story for this yet. I hope we'll be able to get to it soon, but cross-compilation and dependency management are bigger pain points right now.

You could try using the go_path rule, which builds a GOPATH-like source tree. Maybe cover could be run in there. See BUILD.bazel for an example of that.

Contributor

jayconrod commented Nov 10, 2017

Sorry, we still don't have a good story for this yet. I hope we'll be able to get to it soon, but cross-compilation and dependency management are bigger pain points right now.

You could try using the go_path rule, which builds a GOPATH-like source tree. Maybe cover could be run in there. See BUILD.bazel for an example of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment