Spurious testsuite failures when run in parallel

[andhe]

Hello!

Filing this issue mostly to give feedback so that you're aware of it. Please feel free to close this issue at any time when you feel there's nothing more you'd like to fix up here, anyway...

While building libical 2.0.0 packages on Debian I ran into testsuite problems with "test 8" (icalrecur -r) on a seemingly random set of architectures: arm64, ppc64el, mips64el. The problems where not reproducible when running a manual build on Debian porter boxes for these architectures. I quickly assumed based on experience that it was parallel build related and can confirm that switching from "--parallel" to "--no-parallel" in our debian/rules file which invokes the build machinery when building packages made the problems go away.

I have not confirmed but my suspicion is that both the icalrecur tests (8 & 9) using the same test.out output filename is part of the problem. Likely the tests run in parallell and the test9 gets to overwrite the file before test8 has a chance to verify it against its expected results in certain racy conditions which happened to trigger reliably on the previously mentioned architecture buildds. Maybe the icalrecur test program can be changed to take output filename as an argument (and then be handed unique names per test) instead of hardcoding test.out in the fopen call? For now I'll live with marking the package as not being able to build in parallel to work around the issue.

Regards,
Andreas Henriksson

[winterz]

I don't recall running make test in a parallel mode, so I wouldn't be surprised that this is something we need to fix.

[winterz]

in the master branch I just commited 825e25a that might fix this issue. I tested every combination I could think of. Please test.

[winterz]

no response. closing. please re-open if you still encounter this problem in master branch.