Tasty output garbled on travis.ci

[andreasabel]

The ansi-terminal tricks are likely responsible for getting garbled output on travis.ci. See, e.g., https://travis-ci.org/agda/agda/jobs/527944949 for a typical protocol of the Agda test suite. When a test is failing, it is hard to impossible to see which case failed.
Running the testsuite on my local machine I get nice output, none of this garbling.

Have you heard this complaint before?

An option to get "plain output" or a fix that works for travis would be great!

[UnkindPartition]

Possibly the same issue as what @gromakovsky reported here: #152 (comment). I am looking into it.

[UnkindPartition]

There is now a fix/workaround in master: you can pass --ansi-tricks=false on the command line or set TASTY_ANSI_TRICKS=false in the environment to disable the ANSI tricks. Could you please try it and confirm if it works for you?

[andreasabel]

Thanks @feuerbach!
Trying with a development version of tasty will not be easy as we are building the test suite on travis, pulling the tasty package from hackage. It would be easier to try it with a hackage release of tasty.

[UnkindPartition]

Ok, this is now released as tasty-1.2.2 on hackage.

[andreasabel]

you can pass --ansi-tricks=false on the command line

I tried it at agda/agda#3906 to no avail. How does this work?

• This command line option is not listed here: https://github.com/feuerbach/tasty/blob/master/README.md#runtime
• It does not seem to be recognized: https://travis-ci.org/agda/agda/jobs/556472943
  I passed it in the same way as --hide-successes, is that right?
  See: agda/agda@f8b71f9

[L-TChen]

It seems that AnsiTricks is not exported in Test.Tasty.Ingredients.Basic. Adding AnsiTricks(..) to the following list

https://github.com/feuerbach/tasty/blob/047d297ea52a3ba395605318a66cc79d4db25c95/core/Test/Tasty/Ingredients/Basic.hs#L6-L19

should do the trick.

[andreasabel]

Good, I hang on for the next release.

[UnkindPartition]

@andreasabel sorry, I only saw @L-TChen's comment and missed yours (and was somewhat confused as to what "trick" Liang-Ting was referring to).

Liang-Ting's PR has nothing to do with your issue and is purely stylistic (adding a re-export for consistency with other options). Your issue looks very bizarre to me; I'll look into it.

[L-TChen]

That's even weird. I checked out the version 1.2.2 and found that --ansi-tricks is there already. Sorry for overlooking the issue.

[UnkindPartition]

@andreasabel ah, the problem is that you're not using tasty's console reporter (which now has the --ansi-tricks option) but tasty-silver's interactiveTests (which is, as far as I understand, a completely independent codebase). So you'll have to ask @phile314 to port the new option to his ingredient.

[UnkindPartition]

@andreasabel alternatively, if you are not using any of tasty-silver's specific features on the CI, you could adjust your code so that it uses tasty's console reporter when run on the CI.

You can do this either externally to tasty (by choosing whether to run defaultMain1 from tasty-silver or defaultMain from tasty based on, say, an environment variable) or internally, by creating a meta-ingredient that would switch between tasty's consoleReporter and tasty-silver's interactiveTests based on the value of a tasty option. (The code for the meta-ingredient would be similar to composeReporters, see https://hackage.haskell.org/package/tasty-1.2.2/docs/src/Test.Tasty.Ingredients.html#composeReporters)

[andreasabel]

(See phile314/tasty-silver#17 for a feature request from @phile314.)