Paths on windows get mangled

[Neppord]

I get this error for a very simple test

Call to `git diff` failed: (ExitFailure 1,"","error: Could not access 'C:UsersneppordAppDataLocalTempver6F55.golden'\n")

It seams like all the "\" are gone and that it cant find the correct file.

[phile314]

How exactly are you calling tasty-silver? Can you add a minimal, working example which reproduces the problem?

[Neppord]

#! /usr/bin/env stack
-- stack --resolver lts-13.4 script


import Test.Tasty.Silver
import Test.Tasty.Silver.Interactive

main = defaultMain $
  goldenVsFile
    "Verify the world"
    "approved.txt"
    "recived.txt"
    (writeFile "recived.txt" "Hello World\n")

The error is triggerd when you have a golden file "approved.txt" that does not contain the approved recived text.

Verify the world: FAIL (0.00s)
                  Result did not match golden value.
                  Diff between actual and golden value:
========================================
Main.hs: Call to `git diff` failed: (ExitFailure 1,"","error: Could not access 'C:UsersneppordAppDataLocalTempVerD8B5.golden'\n")
CallStack (from HasCallStack):
  error, called at .\\Test\\Tasty\\Silver\\Interactive.hs:187:16 in tasty-silver-3.1.12-9wdmuzsaGoxDzww3ZMbbTe:Test.Tasty.Silver.Interactive

[phile314]

Looks like the paths are passed to sh, but no escaping takes place:
https://github.com/phile314/tasty-silver/blob/master/Test/Tasty/Silver/Interactive.hs#L184

A quick&dirty fix is probably just escaping backslashes by doubling them. I will see what I can do.

[phile314]

I pushed a fix to https://github.com/phile314/tasty-silver/tree/fix_shell_escape . This is untested, as I do not have any Windows systems. Any chance you can test that branch?

[Neppord]

What do I do to test the changes?
i'm a stack novice and very new to develop "real" things in Haskell.

[andreasabel]

I just hit the OP when trying to run the Agda testsuite under cygwin. (After a maybe naive attempt for a Windows CI failed: https://github.com/agda/agda/runs/3450560107?check_suite_focus=true.)

@phile314

This is untested, as I do not have any Windows systems.

A way forward would be multi-platform CI to make sure tasty-silver is portable.

[phile314]

I am not using tasty-silver personally much any longer, so I am unlikely to investigate this further. Sorry. I am happy to merge PRs and make new releases if necessary, but don't expect anything beyond that. I am also happy to give maintainer access if anybody is interested.

[andreasabel]

@phile314 Because Agda depends on tasty-silver, I am happy to become co-maintainer, here and on hackage (https://hackage.haskell.org/package/tasty-silver/maintainers/). I'll likely be conservative (maintaining buildability, CI, fixing critical issues).

[phile314]

Sounds good to me, I will add you as co-maintainer here and and on hackage :-)

[andreasabel]

A way forward would be multi-platform CI to make sure tasty-silver is portable.

A multi-platform CI is now in place (ee8e237, horray!), but we need to populate the tests directory with system tests that expose the problem on Windows.

[andreasabel]

I can now reproduce the problem on the Windows CI: https://github.com/phile314/tasty-silver/runs/3527085480?check_suite_focus=true#step:8:12
The diff is not displayed, but the exception isn't propagated up; rather, the test result is simply FAILED (because of the mismatch between golden and actual value).

[andreasabel]

I found out that the indirection via "sh" is to blame, e.g.:

tasty-silver/Test/Tasty/Silver/Interactive.hs

Line 203 in e910c06

ret <- PTL.readProcessWithExitCode "sh" ["-c", "git diff --no-index --text --exit-code " ++ fGold ++ " " ++ fAct] T.empty

If arguments are not concatenated together as string but passed individually to "git", the mangling does not happen.
In this specific instance, the sh -c wrapper seems superfluous so it is easy to fix. There are other instances where the output is piped into less, this will be less direct to fix:

tasty-silver/Test/Tasty/Silver/Interactive.hs

Line 230 in e910c06

colorDiff fGold fAct = callProcess "sh" ["-c", "wdiff " ++ fGold ++ " " ++ fAct ++ " | colordiff | less -r > /dev/tty"]

[andreasabel]

Fix released in version 3.3.