Testing on Windows: how to compile under MinGW-w64?

[latk]

There have been a lot of path-related problems on Windows, so I would like to add automated tests using AppVeyor. They offer a Windows-based continuous integration service similar to Travis CI. Their images provide MinGW, MSYS2, and so on.

The question is: how can I run the test suite under Windows? What environment variables must be set? What is the correct mixture of backslashes and forward slashes in path names? If anyone has managed to do that or has related tips/insights, please discuss those in this issue.

Because I don't have a Windows development machine, I'm dependent on your contributions for this.

The gcovr test suite is based on Makefiles that run the compiler (CXX) and generate gcovr reports. Some Makefiles use CMake to drive the build. The tests are driven by the gcovr/tests/test_gcovr.py script. The reports are then compared to baseline files.

To run the tests under Linux, I set the CC, CXX, GCOV environment variables to the GCC version I want to test (should be GCC 5, else the baseline files assume slightly different coverage). Then I run python -m nose -v for a convenient test-runner, or invoke python gcovr/tests/test_gcovr.py directly. See also the .travis.yml.

I've started to work on an AppVeyor config at latk/appveyor: .appveyor.yml. Note that this branch is rather volatile and will be rebased frequently.

Currently, the tests start to run but the Makefiles fail when the compiler is invoked. But the compiler does exist when I invoke it from the cmd shell. The CMake tests want to build with Visual Studio, ignoring my CXX setting.

You can see the build logs on AppVeyor.

The available software on AppVeyor images is documented here.

I expect Windows testing to discover a lot of bugs. I want these to be figured out and resolved before any large refactoring of gcovr is started so that regressions can be prevented.

[goriy]

In most cases all I need to build some software on Windows with MinGW-w64 compiler is to add path to compiler (i.e. bin subdirectory of MinGW) to system environment variable PATH.
After that most of my Makefiles works the same way as on Linux.

Sometimes it's also necessary to change some system utils like mkdir, rmdir, etc in Makefile (not everyone has standard unix utils on Windows installed).
In such a case I use something like that in my Makefiles to make them a bit more cross-platform:

ifeq ($(OS),Windows_NT)
  SHELL := cmd
  tmp   :=
  SEP   := \$(tmp)
  RM    := del /f /q
  RMDIR := rd /s /q
  MKDIR := mkdir
  EXE   := .exe
else
  SEP   := /
  RM    := rm -f
  RMDIR := rm -rf
  MKDIR := mkdir -p
  EXE   :=
endif

[goriy]

I've never used AppVeyor. I've read a little about its config and suggest to try to append to PATH in your .appveyor.yml file:

PATH: C:\mingw-w64\i686-5.3.0-posix-dwarf-rt_v4-rev0\mingw32\bin;$(PATH)

and left other vars like this:

    CXX:    g++
    CC:     gcc
    GCOV:   gcov
    MAKE:   mingw32-make

I'm not sure about mingw32-make. On Windows I usually use "native" make port for windows, but maybe it's just a personal preference and mingw32-make works fine.

Mingw compilers understand forward slashes in paths as good as backslashes, so it shouldn't be any issues about that. For me it's usually more convenient to use unix-style forward slashes (at least there is no confusion about backslash as a line continue marker in Makefiles).

Builtin windows mkdir, md, rd, del, cd, ... are not so tolerant to forward slashes in paths.
But as far as I can see in AppVeyor images there is msys installed which should contain unix utils, so I suggest to use them in order to use almost the same Makefiles for all platforms (to do this maybe path to them C:\msys64\usr\bin should also be appended to system PATH):

PATH: C:\mingw-w64\i686-5.3.0-posix-dwarf-rt_v4-rev0\mingw32\bin;C:\msys64\usr\bin;$(PATH)

[latk]

Thank you @goriy for those tips! The first tests have started passing!

In the failing tests, there are many pathname issues as expected. I think that can be solved with another regex during output scrubbing. Somewhat troubling, many reported coverage rates are wildly off mark, possibly also related to pathname issues. This is interesting!

[goriy]

I have looked at your build log. It seems that most C/C++ builds are done well.
I try to remember my little experience about using CMake. As far as I remember, in order to create normal Makefile (using gcc compiler) on Windows I've been calling cmake that way:

cmake -G "Unix Makefiles" && make

[latk]

CMake-based tests have started passing. I used the "MSYS Makefiles" generator.

The remaining failures (shared libs and filters) look manageable.

[mayeut]

Also added python3 in #200 : https://ci.appveyor.com/project/mayeut/gcovr/build/1.0.39