Skip to content

testing: normalize line endings in example output comparisons #51269

Closed
@benhoyt

Description

@benhoyt

If you have code which outputs \r\n (CR LF) as a newline delimiter on Windows, testable examples using this code will fail. I noticed this on my GoAWK project, which translates \n to \r\n on Windows to be a good Windows line-ending citizen on that platform. I'm currently using a !windows build constraint to skip this on Windows so these testable examples don't fail (source).

In addition, it's not obvious from the failure message what's going on, because even though it prints the got/want output, it looks correct (CR LF doesn't look any different from just LF when printed):

--- FAIL: Example (0.00s)
got:
foo
bar
want:
foo
bar
FAIL

Test output and source. My stripped-down example to show what's happening is as follows (though of course in GoAWK it's a bit more complex and hidden):

func Example() {
	if runtime.GOOS == "windows" {
		io.WriteString(os.Stdout, "foo\r\nbar\r\n")
	} else {
		io.WriteString(os.Stdout, "foo\nbar\n")
	}
	// Output:
	// foo
	// bar
}

It seems to me that examples should normalize line endings, and that these tests should succeed on Windows. When you're writing examples you don't want to be bothered by which kind of line ending the code is outputting. (If you care at that level, write a non-example test to ensure that behaviour.)

I'm on Go version 1.17, but this affects all versions going way back.

For reference, here are a few other "testable example whitespace" issues. Most of them are about spaces at the end of lines, so don't directly address this issue:

In #6416, Rob Pike says:

I would prefer to see the output of the examples use exact matching because that's
easiest to specify and allows one to test exact output. For the rare examples for which
that is problematic, sanitize/regularlize the output before matching. That is, write
code.

This could be taken as applying to this issue as well. I don't love this -- it is "easiest" to specify, but it seems to me not what you want, and definitely not visually obvious (which is what example tests are about). In any case, if that's the way it is, so be it. Maybe we could at least make the failure message more obvious in that case?

And if there are better ways to approach this problem in GoAWK or wherever else, I'm all ears!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Accepted

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions