[BUG] test suite failures on various not-so-common architectures

[florianernst]

Dear SRT maintainers,

when attempting to build srt 1.4.4 for Debian GNU/Linux I noticed that the test suite fails on various not-so-common architectures.

These architectures are:

• ppc64el (official Debian release architecture)
• s390x (official Debian release architecture)
• hppa (no Debian release architecture)
• powerpc (no Debian release architecture)
• ppc64 (no Debian release architecture)
• sparc64 (no rDebian elease architecture)
• kfreebsd-amd64 (no Debian release architecture, non-Linux port)
• kfreebsd-i386 (no Debian release architecture, non-Linux port)
• hurd-i386 (no Debian release architecture, non-Linux port)

It is my understanding that SRT is not focusing on these architectures, and hence I'm simply skipping the test suite on these architectures now, but I thought I'd inform you nonetheless. I'm reporting all these in a single bugreport now as I deem this report more informational, but feel free to split out / tell me to split out individual parts if you want to go into more detail.

Now follows a list of individual test failures for each architecture, with an excerpt listing the actual errors and with a link to the complete logs. The linked logs each contain a detailed description of the build environment in which the tests were run.

ppc64el log

99% tests passed, 1 tests failed out of 203

Total Test time (real) =  55.05 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	186 - SyncEvent.WaitForTwoNotifyOne (Failed)
Errors while running CTest

In fact, this one might more indicate that the test suite can be somewhat fragile, especially when being exposed to noisy neighbors, as the very same failure occasionally occurs during build reproducibility testing on architectures where in a controlled environment the test suite normally succeeds, cf. an i386 sample and an armhf sample. Also an occasional test timeout has been experienced, cf. an amd64 sample, which also hints in this direction.

s390x log

96% tests passed, 8 tests failed out of 203

Total Test time (real) =  40.87 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	 10 - CIPAddress.IPv6_pton (Failed)
	 75 - TestFEC.Connection (Failed)
	 76 - TestFEC.ConnectionReorder (Failed)
	 77 - TestFEC.ConnectionFull1 (Failed)
	 78 - TestFEC.ConnectionFull2 (Failed)
	 79 - TestFEC.ConnectionMess (Failed)
	 80 - TestFEC.ConnectionForced (Failed)
	 91 - TestIPv6.v6_calls_v4 (Failed)
Errors while running CTest

hppa log (not a Debian release architecture anymore)

96% tests passed, 9 tests failed out of 203

Total Test time (real) = 1729.44 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	 10 - CIPAddress.IPv6_pton (Failed)
	 75 - TestFEC.Connection (Failed)
	 76 - TestFEC.ConnectionReorder (Failed)
	 77 - TestFEC.ConnectionFull1 (Failed)
	 78 - TestFEC.ConnectionFull2 (Failed)
	 79 - TestFEC.ConnectionMess (Failed)
	 80 - TestFEC.ConnectionForced (Failed)
	 87 - Transmission.FileUpload (Failed)
	 91 - TestIPv6.v6_calls_v4 (Failed)
Errors while running CTest

powerpc log (not a Debian release architecture anymore)

96% tests passed, 8 tests failed out of 203

Total Test time (real) =  57.74 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	 10 - CIPAddress.IPv6_pton (Failed)
	 75 - TestFEC.Connection (Failed)
	 76 - TestFEC.ConnectionReorder (Failed)
	 77 - TestFEC.ConnectionFull1 (Failed)
	 78 - TestFEC.ConnectionFull2 (Failed)
	 79 - TestFEC.ConnectionMess (Failed)
	 80 - TestFEC.ConnectionForced (Failed)
	 91 - TestIPv6.v6_calls_v4 (Failed)
Errors while running CTest

ppc64 log (not a Debian release architecture anymore)

96% tests passed, 8 tests failed out of 203

Total Test time (real) =  59.26 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	 10 - CIPAddress.IPv6_pton (Failed)
	 75 - TestFEC.Connection (Failed)
	 76 - TestFEC.ConnectionReorder (Failed)
	 77 - TestFEC.ConnectionFull1 (Failed)
	 78 - TestFEC.ConnectionFull2 (Failed)
	 79 - TestFEC.ConnectionMess (Failed)
	 80 - TestFEC.ConnectionForced (Failed)
	 91 - TestIPv6.v6_calls_v4 (Failed)
Errors while running CTest

sparc64 log (not a Debian release architecture anymore)

96% tests passed, 8 tests failed out of 203

Total Test time (real) =  62.31 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	 10 - CIPAddress.IPv6_pton (Failed)
	 75 - TestFEC.Connection (Failed)
	 76 - TestFEC.ConnectionReorder (Failed)
	 77 - TestFEC.ConnectionFull1 (Failed)
	 78 - TestFEC.ConnectionFull2 (Failed)
	 79 - TestFEC.ConnectionMess (Failed)
	 80 - TestFEC.ConnectionForced (Failed)
	 91 - TestIPv6.v6_calls_v4 (Failed)
Errors while running CTest

kfreebsd-amd64 log (not a Debian release architecture, not a Linux port)

98% tests passed, 4 tests failed out of 203

Total Test time (real) =  35.03 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	192 - ThreadName.GetSet (Failed)
	193 - ThreadName.AutoReset (Failed)
	201 - ReuseAddr.SameAddr2 (Failed)
	202 - ReuseAddr.DiffAddr (Failed)
Errors while running CTest

kfreebsd-i386 log (not a Debian release architecture, not a Linux port)

99% tests passed, 2 tests failed out of 203

Total Test time (real) =  38.27 sec

The following tests did not run:
	194 - CTimer.SleeptoAccuracy (Disabled)

The following tests FAILED:
	192 - ThreadName.GetSet (Failed)
	193 - ThreadName.AutoReset (Failed)
Errors while running CTest

hurd-i386 log (not a Debian release architecture, not a Linux port)

test 72
        Start  72: TestEPoll.SimpleAsync

72: Test command: /<<PKGBUILDDIR>>/build-openssl/test-srt "--gtest_filter=TestEPoll.SimpleAsync"
72: Test timeout computed to be: 10000000
72: Running main() from ./googletest/src/gtest_main.cc
72: Note: Google Test filter = TestEPoll.SimpleAsync
72: [==========] Running 1 test from 1 test suite.
72: [----------] Global test environment set-up.
72: [----------] 1 test from TestEPoll
72: [ RUN      ] TestEPoll.SimpleAsync
72: ./test/test_epoll.cpp:668: Failure
72: Expected: (srt_bind(servsock, psa, sizeof sa)) != (SRT_ERROR), actual: -1 vs -1
72: ./test/test_epoll.cpp:596: Failure
72: Expected: (srt_connect(m_client_sock, psa, sizeof sa)) != (SRT_ERROR), actual: -1 vs -1
E: Build killed with signal TERM after 180 minutes of inactivity

Cheers,
Flo