Add CI test harness for `thumb*` targets. [IRR-2018-embedded]

[sekineh]

This pull request will do the following (rather trivial) changes:

• Fix #52163. In other words, we enabled ./x.py test src/test/run-make for no_std targets.
• Modify dist-various-1 Dockerfile.
  • CI now performs run-make test run on the targets below:
    • thumbv6m-none-eabi
    • thumbv7m-none-eabi
    • thumbv7em-none-eabi
    • thumbv7em-none-eabihf.
• Add thumb-none Dockerfile.
  • Initially, thumbv7m-none-eabi, thumbv7em-none-eabi and thumbv7em-none-eabihf are included as the tested target. thumbv6m-none-eabi is disabled for now because LLVM support is not certain.
• Add thumb-none to .travis.yml

Note:

• run-make tests are not implemented yet. This PR is test harness only.

The amount of change is very small, but I'd like to open the pull request while the change is trivial.
Because I'm not very used to pull request process, I want to make a small progress first. This PR will be a foundation for later additions.

CC @kennytm @jamesmunns @nerdyvaishali

[TimNN]

Thanks for the PR! We’ll periodically check in on it to make sure that @alexcrichton or someone else from the team reviews it soon.

[alexcrichton]

Thanks for the PR @sekineh! All the content here looks good to me! This currently adds a new builder to .travis.yml though which we unfortunately don't have a lot of spare capacity for. Could this instead modify the existing src/ci/docker/dist-various-1/Dockerfile instead? That already doesn't take too too long and is compiling most of these targets already.

I think the SCRIPT there can be changed to basically run tests and then do the dist it's already doing today like musl is doing today

[sekineh]

Self review:

• PR #51966 did not actually disabled thumbv6m-none-eabi target in the end. I need to re-include that target in this PR.

[sekineh]

I've completed:

• Follow the @alexcrichton 's suggestion. (0b3b8bf)
• Follow my own suggestion. (7630090)

After that, I updated the PR description.

[alexcrichton]

@bors: r+

[bors]

📌 Commit 7630090 has been approved by alexcrichton

[bors]

⌛️ Testing commit 7630090 with merge 70f282b...

[bors]

💔 Test failed - status-travis

[rust-highfive]

The job mingw-check of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:06da1183:start=1532076755567489028,finish=1532076755574119348,duration=6630320
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:2fadcfbf
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:2ba375ca
travis_time:start:2ba375ca
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:04867d00
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[sekineh]

mingw-check failed due to network error.

[00:03:13]  Downloading openssl v0.10.10
[00:03:44] warning: spurious network error (2 tries remaining): [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)
[00:04:14] warning: spurious network error (1 tries remaining): [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)
[00:04:44] error: unable to get packages from source
[00:04:44] 
[00:04:44] Caused by:
[00:04:44]   [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)
[00:04:44] thread 'main' panicked at 'command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "metadata" "--format-version" "1" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml"
[00:04:44] expected success, got: exit code: 101', build_helper/lib.rs:125:9
[00:04:44] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:04:44] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
[00:04:44] Build completed unsuccessfully in 0:03:17

[sekineh]

@alexcrichton
Maybe we need to run retry.

[alexcrichton]

@bors: retry

[bors]

⌛️ Testing commit 7630090 with merge 5bc0673...

[bors]

💔 Test failed - status-travis

[rust-highfive]

The job wasm32-unknown of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:50:59] status: exit code: 2
[00:50:59] command: "make"
[00:50:59] stdout:
[00:50:59] ------------------------------------------
[00:50:59] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-panic-small/wasm-panic-small  foo.rs -C lto -O --target wasm32-unknown-unknown --cfg a
[00:50:59] Makefile:5: recipe for target 'all' failed
[00:50:59] ------------------------------------------
[00:50:59] stderr:
[00:50:59] ------------------------------------------
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59]   |
[00:50:59]   = note: the `wasm32-unknown-unknown` target may not be installed
[00:50:59] error: aborting due to previous error
[00:50:59] 
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] make: *** [all] Error 1
[00:50:59] ------------------------------------------
[00:50:59] 
[00:50:59] 
[00:50:59] thread '[run-make] run-make/wasm-panic-small' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3137:9
[00:50:59] 
[00:50:59] ---- [run-make] run-make/wasm-custom-section stdout ----
[00:50:59] 
[00:50:59] error: make failed
[00:50:59] error: make failed
[00:50:59] status: exit code: 2
[00:50:59] command: "make"
[00:50:59] stdout:
[00:50:59] ------------------------------------------
[00:50:59] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-custom-section/wasm-custom-section:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-custom-section/wasm-custom-section -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-custom-section/wasm-custom-section  foo.rs --target wasm32-unknown-unknown
[00:50:59] Makefile:5: recipe for target 'all' failed
[00:50:59] ------------------------------------------
[00:50:59] stderr:
[00:50:59] ------------------------------------------
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59]   |
[00:50:59]   = note: the `wasm32-unknown-unknown` target may not be installed
[00:50:59] error: aborting due to previous error
[00:50:59] 
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] make: *** [all] Error 1
[00:50:59] ------------------------------------------
[00:50:59] 
[00:50:59] thread '[run-make] run-make/wasm-custom-section' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3137:9
[00:50:59] 
[00:50:59] 
[00:50:59] ---- [run-make] run-make/wasm-import-module stdout ----
[00:50:59] 
[00:50:59] error: make failed
[00:50:59] status: exit code: 2
[00:50:59] command: "make"
[00:50:59] stdout:
[00:50:59] ------------------------------------------
[00:50:59] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-import-module/wasm-import-module:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-import-module/wasm-import-module -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/wasm-import-module/wasm-import-module  foo.rs --target wasm32-unknown-unknown
[00:50:59] Makefile:5: recipe for target 'all' failed
[00:50:59] ------------------------------------------
[00:50:59] stderr:
[00:50:59] ------------------------------------------
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59] error[E0463]: can't find crate for `std`
[00:50:59]   |
[00:50:59]   = note: the `wasm32-unknown-unknown` target may not be installed
[00:50:59] error: aborting due to previous error
[00:50:59] 
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] For more information about this error, try `rustc --explain E0463`.
[00:50:59] make: *** [all] Error 1
[00:50:59] ------------------------------------------
[00:50:59] 
[00:50:59] 
[00:50:59] thread '[run-make] run-make/wasm-import-module' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:3137:9
[00:50:59] 
[00:50:59] failures:
[00:50:59]     [run-make] run-make/wasm-custom-section
[00:50:59]     [run-make] run-make/wasm-import-module
[00:50:59]     [run-make] run-make/wasm-import-module
[00:50:59]     [run-make] run-make/wasm-panic-small
[00:50:59] 
[00:50:59] test result: FAILED. 0 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out
[00:50:59] 
[00:50:59] 
[00:50:59] 
[00:50:59] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/wasm32-unknown-unknown/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-make" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make" "--stage-id" "stage2-wasm32-unknown-unknown" "--mode" "run-make" "--target" "wasm32-unknown-unknown" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--nodejs" "/node-v9.2.0-linux-x64/bin/node" "--host-rustcflags" "-Crpath -O -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/wasm32-unknown-unknown/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--llvm-version" "7.0.0svn\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[00:50:59] 
[00:50:59] 
[00:50:59] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target wasm32-unknown-unknown src/test/run-make src/test/ui src/test/run-pass src/test/compile-fail src/test/parse-fail src/test/mir-opt src/test/codegen-units src/libcore
[00:50:59] Build completed unsuccessfully in 0:48:06
---
travis_time:end:1c8e9a9c:start=1532120432616565425,finish=1532120432626441103,duration=9875678
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:036fcdbf
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1347c2dd
travis_time:start:1347c2dd
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:01005e06
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[sekineh]

I confirmed that wasm32-unknown image works locally.
(At first, I forgot to run git checkout add-ci-thumb. Re-run and succeeded)

sekineh@sekineh-VirtualBox:~/rustme6$ src/ci/docker/run.sh wasm32-unknown 2>&1 | tee run_wasm32.log
(snip)
Build completed successfully in 0:26:17

[kennytm]

@bors r=alexcrichton

[bors]

📌 Commit 3f00b1c has been approved by alexcrichton

[sekineh]

Thanks, @kennytm

I also confirmed dist-various-1 docker image run successfully.
(At first, I forgot to run git checkout add-ci-thumb. Re-ran and succeeded again)

sekineh@sekineh-VirtualBox:~/rustme6$ src/ci/docker/run.sh dist-various-1 2>&1 | tee run_dist-various-1.log
(snip)
[TIMING] Std { compiler: Compiler { stage: 2, host: "x86_64-unknown-linux-gnu" }, target: "thumbv7em-none-eabihf" } -- 9.324
Build completed successfully in 0:09:04

Now I'm more confident bors run will succeeds!

[sekineh]

At first attempt, I forgot to run git checkout add-ci-thumb.
Now I re-ran with 3f00b1c and both docker run (wasm32-unknown and dist-various-1) are confirmed to succeeds.
I also edited each comments I wrote.