Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: testing: normalize line endings in output comparisons #51269

Open
benhoyt opened this issue Feb 19, 2022 · 3 comments
Open

proposal: testing: normalize line endings in output comparisons #51269

benhoyt opened this issue Feb 19, 2022 · 3 comments
Labels
NeedsInvestigation
Projects
Milestone

Comments

@benhoyt
Copy link
Contributor

@benhoyt benhoyt commented Feb 19, 2022

If you have code which outputs \r\n (CR LF) as a newline delimiter on Windows, testable examples using this code will fail. I noticed this on my GoAWK project, which translates \n to \r\n on Windows to be a good Windows line-ending citizen on that platform. I'm currently using a !windows build constraint to skip this on Windows so these testable examples don't fail (source).

In addition, it's not obvious from the failure message what's going on, because even though it prints the got/want output, it looks correct (CR LF doesn't look any different from just LF when printed):

--- FAIL: Example (0.00s)
got:
foo
bar
want:
foo
bar
FAIL

Test output and source. My stripped-down example to show what's happening is as follows (though of course in GoAWK it's a bit more complex and hidden):

func Example() {
	if runtime.GOOS == "windows" {
		io.WriteString(os.Stdout, "foo\r\nbar\r\n")
	} else {
		io.WriteString(os.Stdout, "foo\nbar\n")
	}
	// Output:
	// foo
	// bar
}

It seems to me that examples should normalize line endings, and that these tests should succeed on Windows. When you're writing examples you don't want to be bothered by which kind of line ending the code is outputting. (If you care at that level, write a non-example test to ensure that behaviour.)

I'm on Go version 1.17, but this affects all versions going way back.

For reference, here are a few other "testable example whitespace" issues. Most of them are about spaces at the end of lines, so don't directly address this issue:

In #6416, Rob Pike says:

I would prefer to see the output of the examples use exact matching because that's
easiest to specify and allows one to test exact output. For the rare examples for which
that is problematic, sanitize/regularlize the output before matching. That is, write
code.

This could be taken as applying to this issue as well. I don't love this -- it is "easiest" to specify, but it seems to me not what you want, and definitely not visually obvious (which is what example tests are about). In any case, if that's the way it is, so be it. Maybe we could at least make the failure message more obvious in that case?

And if there are better ways to approach this problem in GoAWK or wherever else, I'm all ears!

@seankhliao seankhliao added the NeedsInvestigation label Feb 20, 2022
@bcmills bcmills added this to the Backlog milestone Feb 22, 2022
@bcmills
Copy link
Member

@bcmills bcmills commented Feb 22, 2022

#41980 could help somewhat here, in that the diff could at least (potentially) show the difference in line endings.

@bcmills
Copy link
Member

@bcmills bcmills commented Feb 22, 2022

It seems to me that examples should normalize line endings, and that these tests should succeed on Windows. When you're writing examples you don't want to be bothered by which kind of line ending the code is outputting.

I tend to agree, but I think this is a subtle enough change to send through the proposal process. Do you want to go that route, or does #41980 suffice?

@benhoyt
Copy link
Contributor Author

@benhoyt benhoyt commented Feb 22, 2022

Thanks @bcmills. #41980 would definitely have helped me debug the issue to begin with (way back when). But I think it'd be even better if these tests could work on Windows, i.e., that the line endings were normalized. Would be great if this can be discussed as a proposal -- even if rejected.

@bcmills bcmills changed the title testing: examples that output \r\n on Windows fail (with a non-obvious message) proposal: testing: normalize line endings in output comparisons Feb 22, 2022
@bcmills bcmills removed this from the Backlog milestone Feb 22, 2022
@bcmills bcmills added this to the Proposal milestone Feb 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
Proposals
Awaiting triage
Development

No branches or pull requests

3 participants