Set up Continuous Integration

[hoche]

Notes:

• GitHub's free level service is 500MB and 2000min/month of CPU time.
• They have VMs for Windows Server 2019, Ubuntu 18.04, Ubuntu 16.04, and MacOS 10.15.
• Windows CPU time is billed at 2x Linux time, and MacOS time is billed at 10x.

This is probably good enough to do a basic per-checkin level qualification, but the limited number of OSes available and very limited disk space means that it's not sufficient to do a full qualification, particularly since running splat-testsuite is both a CPU hog and requires about 400MB of disk.

I have a number of VMs (Vagrants) already set up on a moderately high-end host (Core i7-9700, 32GB, SSD), so I'm thinking I'll set that up to do automatically do extended checking and testing runs on that. I'll have to figure out how to report back errors though.

[watkipet]

What about Travis CI?

It's free for Open Source and has unlimited builds. It's what I used for my Splat! repo. Very easy to set up. I have some regression tests that I could migrate over.

[hoche]

What are you using for regression tests? I set up a suite with python's unittest, but I'm not real happy with it.

Travis CI might work, and they can compile for Windows, various Ubuntus, and OSX for free, as you indicated. That still doesn't cover the *BSD or SUSE/RH/etc variants of unix, but I can probably handle that with my vagrants on a less frequent basis.

[watkipet]

I misspoke. It's actually just a single build acceptance test. It was @drhirsch who wrote the test script and set up the travis config.

It's just simple Python script that downloads one tile of data, converts it to SDF, and then runs a Splat report on it (not a map generation). This shows that the executable is at least runnable.

[watkipet]

I'm planning on creating a branch off of 2.0-develop and then do the same for 1.5-develop. Then I'll work on regression tests.

[watkipet]

Well, it looks like Travis CI is no longer free for Open Source (or perhaps their bar is a little high for this project). Any objections to Azure Pipelines? I'm familiar with it since that's what I use at work.

[hoche]

I think that while the Azure Pipelines CI is free for Open Source, it costs money to maintain an Azure account.
I was leaning towards self-hosting a TeamCity server.

[scivision]

Since the very first comment in 2020 this thread, GitHub Actions is available at no cost with larger quotas.