significantly increase test coverage #3290

andrewrk · 2019-09-22T04:13:39Z

add zig build option -Dskip-libc to skip tests that build libc
(e.g. if you don't want to wait for musl to build)
add -Denable-wine option which uses wine to run cross compiled
windows tests on non-windows hosts
add -Denable-qemu option which uses qemu to run cross compiled
foreign architecture tests
add -Denable-foreign-glibc=path option which combined with
-Denable-qemu enables running cross compiled tests that link
against glibc. See
https://github.com/ziglang/zig/wiki/Updating-libc#glibc for how to
produce this directory.
the test matrix is done manually. release test builds are only
enabled by default for the native target. this should save us some CI
time, while still providing decent coverage of release builds.
- add test coverage for x86_64-linux-musl -lc (building musl libc)
- add test coverage for x86_64-linux-gnu -lc (building glibc)
- add test coverage for aarch64v8_5a-linux-none
- add test coverage for aarch64v8_5a-linux-musl -lc (building musl libc)
- add test coverage for aarch64v8_5a-linux-gnu -lc (building glibc)
- add test coverage for arm-linux-none
- test coverage for arm-linux-musleabihf -lc (building musl libc) is
  disabled due to unable to build musl for target armv8-linux-musleabihf #3286
- test coverage for arm-linux-gnueabihf -lc (building glibc) is disabled
  due to unable to build glibc for target armv8-linux-gnueabihf #3287
- test coverage for x86_64-windows-gnu -lc (building mingw-w64) is
  disabled due to mingw-w64 libc regressed #3285
enable qemu testing on the Linux CI job. There's not really a good
reason to enable wine, since we have a Windows CI job as well.
remove the no longer needed --build-file ../build.zig from CI
scripts
fix bug in glibc compilation where it wasn't properly reading the abi
list txt files, resulting in "key not found" error.
std.build.Target gains:
- isNetBSD
- isLinux
- osRequiresLibC
- getArchPtrBitWidth
- getExternalExecutor
zig build system gains support for enabling wine and enabling qemu.
artifact.enable_wine = true;, artifact.enable_qemu = true;. This
communicates that the system has these tools installed and the build
system will use them to run tests.
zig build system gains support for overriding the dynamic linker of
an executable artifact.
fix std.c.lseek prototype. makes behavior tests for
arm-linux-musleabihf pass.
disable std lib tests that are failing on ARM. See std lib tests failing on aarch64 #3288, std lib tests failing on arm 32-bit #3289
provide std.os.off_t.
disable some of the compiler_rt symbols for arm 32 bit. Fixes
compiler_rt tests for arm 32 bit
add __stack_chk_guard when linking against glibc. Fixes std lib tests
for aarch64-linux-gnu
workaround for "unable to inline function" using @inlineCall. Fixes
compiler_rt tests for arm 32 bit.

LemonBoy · 2019-09-22T09:37:15Z

std/build.zig

+        return self.isDarwin() or self.isFreeBSD() or self.isNetBSD();
+    }
+
+    pub fn getArchPtrBitWidth(self: Target) u32 {


This is right and wrong at the same time, the effective pointer size is given by the architecture and the ABI: x32 means the full 64bit registers are used but the pointer size if trimmed to 4 bytes instead of 8.

mikdusan · 2019-09-22T11:50:21Z

#3292 might fix freebsd error: unknown argument: '-faddrsig'

euantorano

I'm not sure disabling this check for S_ISCHR is the best way to fix an issue on a particular platform. This check is common practice when dealing with /dev/urandom (I've included a list below of some examples of other languages and libraries making use of it for example). I'm going to do some testing on some different platforms so I can see what's going on.

euantorano · 2019-09-22T12:44:17Z

std/os.zig

@@ -145,11 +145,6 @@ fn getRandomBytesDevURandom(buf: []u8) !void {
    const fd = try openC(c"/dev/urandom", O_RDONLY | O_CLOEXEC, 0);
    defer close(fd);

-    const st = try fstat(fd);


I've just spun up an aarch64 system on Scaleway:

Linux test-aarch64-urandom 4.9.93-mainline-rev1 #1 SMP Tue Apr 10 09:54:46 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Using file /dev/urandom returns:

/dev/urandom: character special (1/9)

Which is as expected.

I have then spun up an AMD64 system and ran an aarch64 image in QEMU using these instructions (though using Ubuntu 18.04 instead of 16.04):

Linux ubuntu 4.15.0-64-generic #73-Ubuntu SMP Thu Sep 12 13:18:31 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux

Connecting to this and running file /dev/urandom results in:

/dev/urandom: character special (1/9)

As expected.

I'm not sure exactly what the build/test script is doing to test the aarch64 stuff, but it must be doing something weird for /dev/urandom to not be a character device...

I have then spun up an AMD64 system and ran an aarch64 image in QEMU using these instructions (though using Ubuntu 18.04 instead of 16.04):

The tests are done using QEMU's userspace emulation instead of the classic full-system approach. All you have to do is cross-compile the test exercising getRandomBytesDevURandom and run it with qemu-aarch64.

I'll give that a go and see what happens then, thanks.

* add zig build option `-Dskip-libc` to skip tests that build libc (e.g. if you don't want to wait for musl to build) * add `-Denable-wine` option which uses wine to run cross compiled windows tests on non-windows hosts * add `-Denable-qemu` option which uses qemu to run cross compiled foreign architecture tests * add `-Denable-foreign-glibc=path` option which combined with `-Denable-qemu` enables running cross compiled tests that link against glibc. See https://github.com/ziglang/zig/wiki/Updating-libc#glibc for how to produce this directory. * the test matrix is done manually. release test builds are only enabled by default for the native target. this should save us some CI time, while still providing decent coverage of release builds. - add test coverage for `x86_64-linux-musl -lc` (building musl libc) - add test coverage for `x86_64-linux-gnu -lc` (building glibc) - add test coverage for `aarch64v8_5a-linux-none` - add test coverage for `aarch64v8_5a-linux-musl -lc` (building musl libc) - add test coverage for `aarch64v8_5a-linux-gnu -lc` (building glibc) - add test coverage for `arm-linux-none` - test coverage for `arm-linux-musleabihf -lc` (building musl libc) is disabled due to #3286 - test coverage for `arm-linux-gnueabihf -lc` (building glibc) is disabled due to #3287 - test coverage for `x86_64-windows-gnu -lc` (building mingw-w64) is disabled due to #3285 * enable qemu testing on the Linux CI job. There's not really a good reason to enable wine, since we have a Windows CI job as well. * remove the no longer needed `--build-file ../build.zig` from CI scripts * fix bug in glibc compilation where it wasn't properly reading the abi list txt files, resulting in "key not found" error. * std.build.Target gains: - isNetBSD - isLinux - osRequiresLibC - getArchPtrBitWidth - getExternalExecutor * zig build system gains support for enabling wine and enabling qemu. `artifact.enable_wine = true;`, `artifact.enable_qemu = true;`. This communicates that the system has these tools installed and the build system will use them to run tests. * zig build system gains support for overriding the dynamic linker of an executable artifact. * fix std.c.lseek prototype. makes behavior tests for arm-linux-musleabihf pass. * disable std lib tests that are failing on ARM. See #3288, #3289 * provide `std.os.off_t`. * disable some of the compiler_rt symbols for arm 32 bit. Fixes compiler_rt tests for arm 32 bit * add __stack_chk_guard when linking against glibc. Fixes std lib tests for aarch64-linux-gnu * workaround for "unable to inline function" using `@inlineCall`. Fixes compiler_rt tests for arm 32 bit.

The ABI was incorrect.

due to #3295

See #3024

After upgrading to LLVM 9 we now have the emscripten OS available. Fix the compile error in this function by specifying the default ABI for emscripten.

It was using __GNUC__ < 8 to check if _xgetbv intrinsic is available. Since we always use zig cc with this header, we know we have clang 9, which does have this intrinsic.

closes #3285

Previously the CMAKE_BUILD_TYPE=Release was conflicting with the "distribution" directory `release`. I renamed this to `dist` so that it won't conflict with any build types.

LemonBoy reviewed Sep 22, 2019

View reviewed changes

euantorano reviewed Sep 22, 2019

View reviewed changes

andrewrk force-pushed the more-test-coverage branch from 273244b to 4dff8c6 Compare September 22, 2019 15:12

andrewrk added 3 commits September 22, 2019 11:22

windows ci: make the case of file names agree

dc2463d

fix linux stat struct on aarch64

2510ede

The ABI was incorrect.

andrewrk force-pushed the more-test-coverage branch from 4dff8c6 to 2510ede Compare September 22, 2019 15:22

andrewrk added 15 commits September 22, 2019 14:00

clean up compiler_rt with some enum literals

9627a75

disable running cross compiled macos tests

786052c

due to #3295

do the release tests last because they take longer

cf4cc9b

ci: update windows image to MSVC 2019

12b195f

remove redundant assert

1d6d6d5

windows CI: use MinSizeRel to work around MSVC bug

65b495a

See #3024

more info in assertion failures

31b72da

more helpful error message when failing to parse glibc abi.txt

a239c74

git, I love you, but please stop mangling my files

2c8495b

tolerate windows newlines in glibc text files

f14ba90

silence nonportable include path warnings when building glibc on windows

d6f7b57

add emscripten to std.build.Target.defaultAbi

23f339f

After upgrading to LLVM 9 we now have the emscripten OS available. Fix the compile error in this function by specifying the default ABI for emscripten.

mingw-w64 libc: fix incorrect detection of _xgetbv

c161ce3

It was using __GNUC__ < 8 to check if _xgetbv intrinsic is available. Since we always use zig cc with this header, we know we have clang 9, which does have this intrinsic.

enable test coverage for cross compiling windows mingw-w64

96affd6

closes #3285

windows CI: fix name collision

55925b6

Previously the CMAKE_BUILD_TYPE=Release was conflicting with the "distribution" directory `release`. I renamed this to `dist` so that it won't conflict with any build types.

andrewrk merged commit 35c1d8c into master Sep 23, 2019

andrewrk deleted the more-test-coverage branch September 23, 2019 01:18

This was referenced Sep 23, 2019

Tier 1 Support for ARM 64-bit (aarch64) Linux #2443

Open

Tier 1 Support for ARM 32-bit Linux #3174

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

significantly increase test coverage #3290

significantly increase test coverage #3290

andrewrk commented Sep 22, 2019

LemonBoy Sep 22, 2019

mikdusan commented Sep 22, 2019

euantorano left a comment •

edited

Loading

euantorano Sep 22, 2019

LemonBoy Sep 22, 2019

euantorano Sep 22, 2019

significantly increase test coverage #3290

significantly increase test coverage #3290

Conversation

andrewrk commented Sep 22, 2019

LemonBoy Sep 22, 2019

Choose a reason for hiding this comment

mikdusan commented Sep 22, 2019

euantorano left a comment • edited Loading

Choose a reason for hiding this comment

euantorano Sep 22, 2019

Choose a reason for hiding this comment

LemonBoy Sep 22, 2019

Choose a reason for hiding this comment

euantorano Sep 22, 2019

Choose a reason for hiding this comment

euantorano left a comment •

edited

Loading