On building with Docker:

I'm only familiar with this on GitLab CI and don't know the details of how Travis and Appveyor handle this, so take it with a grain of salt.

When you do docker builds in GitLab CI it effectively caches the state of the project as a container snapshot. That way when you make a change, it just pulls in the delta and likewise CMake + make will only have to rebuild the delta which can save a lot of time on builds in CI.

Generally this works well, so if it's possible to do something similar in Travis and Appveyor it will help build times in a much friendlier way than the current approach of downloading pre-compiled libs from older builds.