testing: examples fail if output is colored

[GuillaumeLescure]

Issue :
go test fail if the output is colored.

Even if the tags are in the output (the comment line), it fails and seems to be litterally evaluated.

Exemple :
In myColored_test.go :

package main

import (
"fmt"
)

func Example_ColorTerm() {
fmt.Print("\033[31mHello World\033[0m")
//Output: Hello World
}

Expectation :
Be able to pass the test either :

• as litteral "Hello World" (colors ignored)
• as interpreted string "\033[31mHello World\033[0m" (colors are also checked)

I personally prefere the second solution because it allows more accurate tests.

Environment :

• go version go1.12.9 linux/amd64
• Manjaro (Linux 4.19.69-1)

[mvdan]

I believe that example outputs are only intended for plaintext; I haven't seen anyone attempt to use non-printable or special characters with them.

I'm also not sure that such an example should be supported. What should godoc.org show for it? It's not a terminal, so it won't color the words.

[GuillaumeLescure]

Agree but if you want to show the user what the output should look like (even without colors), you'll have to fill the "Output comment". But if you do so, it will automatically compare it to the "real" output.

If the colors are also displayed in the documentation, I would be ok with it, but that's not my point.
My point is : I cannot have a documentation with exemple and working test.

[ALTree]

Agree but if you want to show the user what the output should look like (even without colors), you'll have to fill the "Output comment".

It doesn't have to be an Output comment... you can just use a normal comment and then it won't be matched with the output. What am I missing?

[ianlancetaylor]

In my opinion examples should just match the output characters exactly. Anything else will just lead us into confusion. What if you want to use an example to show the difference between colorized and non-colorized output?

[GuillaumeLescure]

It doesn't have to be an Output comment... you can just use a normal comment and then it won't be matched with the output. What am I missing?

That's no the same result on godoc.

In my opinion examples should just match the output characters exactly. Anything else will just lead us into confusion. What if you want to use an example to show the difference between colorized and non-colorized output?

I completly agree, it's far better and simpler to keep the consistency between both. So the solution would be to colorize the output on the documentation and allow it in the Output comment. No ? Otherwise, it just can't be done.

[mvdan]

If you're suggesting to make godoc.org interpret terminal escape codes, or become a terminal emulator, I doubt that's likely to happen :)

Edit: to clarify, that seems like a lot of added complexity for little gain.

[ALTree]

@GuillaumeLescure yes, it doesn't look the same, it's obviously a workaround... My point is that there's a workaround that still somewhat allows you do what you want ("show the user what the output should look like (even without colors),") without changing godoc for everyone. It a corner case, after all.

[GuillaumeLescure]

If you're suggesting to make godoc.org interpret terminal escape codes, or become a terminal emulator, I doubt that's likely to happen :)

That would be nice to keep consistency indeed but I understand the price it cost. That's why I didn't suggest that at first and just do the check without the interpretation.

But at this point we have 3 solutions :

• do nothing and every terminal color tag will fail the test (could be very annoying for those that want a nice documentation but also use colored output like for logs)
• make godoc.org an other terminal emulator, heavy work, little gain, add complexity to godoc, etc ...
• allow to interpret string literally and ignore thoses tag in the render of the website (my original solution) but we lost consistency between input and output

I think all of them are bad and force a trade-off ... something I hate X)

[GuillaumeLescure]

@ALTree agree. I just want to show the issue, if it's already known and decision is made, I can live with it. But I think it's not corner case. It can happen very often for many people.

Ex : you use a package that have a dependence on a log system that colorize output but you cannot change the output (yeah a not-so-good package). So everytime you will use that package (maybe the whole project like Qt library) you will display colorized output in your shell. So after writting your app, you will not be able to write a documentation with exemple 😢

PS : and a workaround stands for "working around the issue" so there is an issue 😉

[ianlancetaylor]

I think that a package that always colorizes output even when not writing to a terminal, and that has no way to disable that colorization, is, as you say, a not-so-good package. I don't see a real need to make the support for examples more complex in order to support not-so-good packages.

[GuillaumeLescure]

I think that a package that always colorizes output even when not writing to a terminal, and that has no way to disable that colorization, is, as you say, a not-so-good package. I don't see a real need to make the support for examples more complex in order to support not-so-good packages.

Ok I understand. I think it's probably the best solution here.