vscode-tmgrammar-snap doesn't run test cases on Windows

[fachammer]

I tried running vscode-tmgrammar-snap as part of a CI process on Azure Pipelines. There it runs tests on Linux, MacOS and Windows environments. Running vscode-tmgrammar-snap works fine on Linux and MacOS but fails on Windows with the message ERROR no test cases found, but I expected it to just work on Windows. Then I tested it on a local Windows 10 VM and there it also fails with the same message.

I made a small reproduction example.

Steps to reproduce the bug on Windows 10 in a PowerShell with administrative privileges:

1. Clone and cd into repository and checkout reproduction commit

   git clone https://github.com/fachammer/bug-reproduction.git
   cd bug-reproduction
   git checkout 340b1b1ec5b2089a213c6b455d961eb4233b9ac0
   

2. Make sure to install windows-build-tools since there is a transitive dependency to onigurama which needs to compile natively. This step requires administrative privileges and can actually take quite a while to finish

   npm install --global windows-build-tools
   

3. Install dependencies and run tests

   npm install
   npm test
   

This should yield the ERROR no test cases found message. Doing the same steps (excluding the windows-build-tools install) on my local Ubuntu installation works fine and produces

✓ test/keyword.lang run successfully.
✓ test/string.lang run successfully.

as expected.

@PanAeon If you want I could take a stab at a bugfix, but maybe you already have an idea what's going on here?

By the way: Great tool! Snapshots really help to avoid regressions.

[fachammer]

Ok, so it has something to do with the way you pass the glob pattern over the command line. Using double quotes instead of single quotes for the testcases pattern works for me (on Ubuntu and on Windows). I've updated the reproduction repo, the updated commit is 46253fd12548fbb5fde09a90756ee29b4478d34d.

So the test cases are actually found now, but there is another issue. The test cases which succeed on Ubuntu now fail on Windows. The error message is:

ERROR running testcase test/keyword.lang source different snapshot at line 1.
 expected: if
 actual: if

ERROR running testcase test/string.lang source different snapshot at line 1.
 expected: "string"
 actual: "string"

My first guess is that it has to do with the fact that line endings are encoded differently on Windows. I'll have a look.

[PanAeon]

@fachammer Thank you so much for reporting this issue and for the fix! Your problem report is excellent. In retrospective I can't believe what a sloppy job I did with hardcoded line endings :) I've already published vscode-tmgrammar-test@0.0.6 to npm with your fix.

[fachammer]

Great, no worries. Thanks for the quick release! Works like a charm now.