Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Goveralls counts braces but "go tool cover" doesn't #103

Open
cdennison opened this issue Dec 3, 2017 · 4 comments
Open

Goveralls counts braces but "go tool cover" doesn't #103

cdennison opened this issue Dec 3, 2017 · 4 comments

Comments

@cdennison
Copy link

cdennison commented Dec 3, 2017

Hi everyone,

I've been trying to contribute to and open source project and noticed that the coveralls coverage percent differs from what I get from go tool cover. The difference is relatively small but I thought you still might want to know about it. I think it's entirely caused by goveralls counting braces but go cover not counting them (go cover approach explained here).

Possible solutions (I'd be happy to contribute toward either)

  1. Don't count the braces (which as a downside will make the coveralls.io report looks weird because the first line of functions wouldn't be marked as covered - the same thing of course happens when you use go's html report go tool cover -html=profile.cov.

  2. Provide a way to output the coverage from goveralls without submitting the report to coveralls.io. Basically add a flag so I can do something like goveralls --report and get something similar to go tool cover -func=profile.cov. This would actually be pretty useful because goveralls correctly outputs coverage across multiple packages (by merging the coverage reports) which go cover doesn't - so this could be used for projects not even using coveralls.io.

I'd be happy to submit a PR if you want to go in either direction above.

Here's a contrived example:
(based on this)

//config.go
package config

type Config struct {
	data map[string]map[string]string
}

func NewConfig() () {
	_ = &Config{
		data: make(map[string]map[string]string),
	}
}

func (c *Config) parse(fname string) (err error) {
	return nil
}

//config_test.go
package config

import (
	"testing"
)

func TestGet(t *testing.T) {
	NewConfig()
}

Then I run the test like this
go test -covermode=count -coverprofile=profile.cov
And check my coverage like this
go tool cover -func=profile.cov
Which give 50%.

For coveralls I get 62.5%. Here's why:

cat profile.cov 
mode: count
github.com/casbin/casbin/config/config.go:8.21,12.2 1 1
github.com/casbin/casbin/config/config.go:14.50,16.2 1 0

Go cover I believe just sees (2) blocks:
block 1: 1/1 lines covered
block 2: 0/1 lines covered
Total = 1/2 = 50%

goveralls gives this output
(https://github.com/mattn/goveralls/blob/master/gocover.go#L102)
1
2
3
4
5
6
7 1
8 1
9 1
10 1
11 1
12
13 0
14 0
15 0

Which means it counts 8 lines (instead of 2 because 6 are braces) and 3 lines without coverage (instead of 1) giving the 5/8 and 62.5%.

Also to be fair - your coverage does match the HTML report (which shows 5/8 lines covered)
go tool cover -html=profile.cov
coverage_html_file0

@mattn
Copy link
Owner

mattn commented Dec 5, 2017

I understand what you mean. But coveralls never gave 100% if using go cover.

@cdennison
Copy link
Author

cdennison commented Dec 14, 2017

@mattn - sorry I missed your comment - I'm not sure I follow. You can get 100% coverage wth go cover - it just doesn't care about the braces. The only difference between goveralls and go cover is regarding missing coverage - the go cover percent will often be lower because it doesn't count braces. Don't get me wrong - most coverage calculators for languages other than go count braces so I think your approach makes more sense, but it's confusing if it's different than the standard library.

@gdm85
Copy link
Collaborator

gdm85 commented Nov 3, 2020

@cdennison I think at the very least we should document this behaviour. If we were to change goveralls (either with an option or in all cases), would skipping the first and the last line of the blocks always be correct?

@cdennison
Copy link
Author

@gdm85 - I believe you just skip all likes that only contain only closing brace "}" but there may be other differences between goveralls and go cover (I'm not an expert on either). I'm happy to create a PR that adds an experimental flag to output "go cover" coverage without closing braces to verify this theory but won't bother if no one cares.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants