Jenkins windows tests

[VMatthijs]

This PR contains work in progress on getting Jenkins to build and run our test models on our Windows machine.
The builds seem to be working on Jenkins and the tests even pass on my local Windows machine.
However, there seems to be a problem with the tests on Jenkins. I believe it has to do with line ending conflicts (LF vs CRLF).
The goal here is to have Jenkins make the Windows machine build a Windows binary on WSL and then to run the tests on that binary, also through WSL.

@serban-nicusor-toptal , would you be interested in having a look at this? Based on what Sean said, you'd likely be much more efficient at figuring out what the problem is here than I would. Thanks!

[seantalts]

Closing this for now - whenever Jenkins picks it up to run it, it blocks the Windows node forever, which stalls all other testing.

[VMatthijs]

Hmm I think it might've been stalling because of the dos2windows install. I've installed it on the Windows box now, but it still doesn't seem to be working.
Want to have another look at this, @serban-nicusor-toptal ?

[seantalts]

He's off this week. Would you mind keeping an eye on it to make sure it doesn't get stuck and block all the builds again?

[VMatthijs]

Will do!

[serban-nicusor-toptal]

Strange that this isn't working on Jenkins Windows Machine or on your local one.

I have it installed & working on my Ubuntu 18.04 WSL.

@VMatthijs I can take a look in the weekend to see what's wrong with it.

[VMatthijs]

Thanks, @serban-nicusor-toptal ! It's installed on the Jenkins Windows Machine and my local machine. That part is fine. It's just that it complains when I run dos2unix /test/*.

[serban-nicusor-toptal]

Hmh, dos2unix doesn't find files recursively ...

You can try this:

cd /stanc3/test/integration
find . -type f -print0 | xargs -0 dos2unix

find will iterate over all the files and pipe each to dos2unix.

On my local machine it does the job:

[VMatthijs]

Woo! This works now!! Thanks, @serban-nicusor-toptal ! @seantalts , could you maybe have a look if you're happy with this? The dos2unix solution is perhaps a bit rough. It also takes 1.5 minutes to complete. I guess we could be more selective with it if we'd want, e.g. only apply it to the *.expected files. Would that be better or is the 1.5 minute wait OK?

[VMatthijs]

Made the changes I proposed above. The dos2unix time is now down to 15s which seems more reasonable. Could you maybe have a look, @seantalts ?

[seantalts]

I think this is fine. Maybe one day someone can figure out the appropriate git configuration to make this automatic but this is good enough. Thank you!

[serban-nicusor-toptal]

@seantalts @VMatthijs

It's really easy to solve the LF/CRLF conversion issue directly from git.
See: https://help.github.com/en/articles/dealing-with-line-endings

How to?

Simply create a .gitattributes file with this inside:

# Declare files that will always have LF line endings on checkout.
*.expected text eol=lf

That's it!

Explanation:

`text eol=lf`
Git will always convert line endings to `LF` on checkout. You should use this for files that must keep LF endings, even on Windows.

---

This issue happens mostly when a GUI is involved like GitHub for Desktop because it sets the following:
git config --global core.autocrlf true
which translates to

If you’re on a Windows machine, set it to true — this converts LF endings into CRLF when you check out code.

Thing which should be set to:
git config --global core.autocrlf input
which translates to

You can tell Git to convert CRLF to LF on commit but not the other way around by setting core.autocrlf to input.

When working cross platform.

For more information see: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration

[seantalts]

I think we tried that at one point...

[VMatthijs]

Yep. That's what I was trying at first and for some reason, it didn't work. Glad you found this solution though!

[seantalts]

I think when it's running tests on windows it's actually executing the WSL linux build for the tests :(

[seantalts]

Yeah, you can see the run with --verbose using the default build instead of default.windows here: http://d1m1s1b1.stat.columbia.edu:8080/blue/organizations/jenkins/stanc3/detail/releases/16/pipeline