Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow cross-compiling doctests #60387

Open
wants to merge 4 commits into
base: master
from

Conversation

@Goirad
Copy link

commented Apr 29, 2019

This PR allows doctest to receive a --runtool argument, as well as possibly many --runtool-arg arguments, which are then used to run cross compiled doctests.
Also, functionality has been added to rustdoc to allow it to skip testing doctests on a per-target basis, in the same way that compiletest does it. For example, tagging the doctest with "ignore-sgx" disables testing on any targets that contain "sgx". A plain "ignore" still skips testing on all targets.

See here for the companion PR in the cargo project that extends functionality in Cargo so that it passes the appropriate parameters to rustdoc when cross compiling and testing doctests.

Part of #6460

@Goirad Goirad changed the title Test expansion Improvements for cross-compiling and cross-running tests Apr 29, 2019

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Apr 29, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:051df8ca:start=1556580223883284172,finish=1556580309914148978,duration=86030864806
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:03:22] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:03:23] tidy error: /checkout/src/librustdoc/lib.rs:363: line longer than 100 chars
[00:03:24] some tidy checks failed
[00:03:24] 
[00:03:24] 
[00:03:24] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:03:24] 
[00:03:24] 
[00:03:24] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:03:24] Build completed unsuccessfully in 0:00:46
[00:03:24] Build completed unsuccessfully in 0:00:46
[00:03:24] make: *** [tidy] Error 1
[00:03:24] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0332c1d2
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Mon Apr 29 23:28:44 UTC 2019
---
travis_time:end:1742d564:start=1556580525064195317,finish=1556580525068786623,duration=4591306
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:220d36c7
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:01fd3a34
travis_time:start:01fd3a34
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:055ccee8
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Show resolved Hide resolved src/bootstrap/bootstrap.py Outdated
Show resolved Hide resolved src/librustdoc/html/markdown.rs Outdated
@Centril

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

@rust-highfive rust-highfive assigned kennytm and unassigned ollie27 Apr 30, 2019

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Apr 30, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:1486f39e:start=1556581849530958173,finish=1556581935702154831,duration=86171196658
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:22:50] 
[01:22:50] running 9 tests
[01:22:50] iiiiiiiii
[01:22:50] 
[01:22:50]  finished in 0.155
[01:22:50] travis_fold:end:test_assembly

---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:23:07] 
[01:23:07] running 121 tests
[01:23:32] .iiiii...i.....i..i...i..i.i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i. 100/121
[01:23:37] i.i......iii.i.....ii
[01:23:37] 
[01:23:37]  finished in 30.550
[01:23:37] travis_fold:end:test_debuginfo

---
travis_time:start:test_rustdoc
Check compiletest suite=rustdoc mode=rustdoc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:27:45] 
[01:27:45] running 305 tests
[01:28:56] ............F...............i...F....FF............................................................. 100/305
[01:29:57] ................F................F..F..F...i........F.....................F.....F................F.. 200/305
[01:31:00] ...F.................................F.........................................FF................... 300/305
[01:31:03] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:31:03] .....
[01:31:03] failures:
[01:31:03] 
---
[01:31:03] ---- [rustdoc] rustdoc/doctest-manual-crate-name.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/doctest-manual-crate-name/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/doctest-manual-crate-name" "/checkout/src/test/rustdoc/doctest-manual-crate-name.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/edition-flag.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/edition-flag/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/edition-flag" "/checkout/src/test/rustdoc/edition-flag.rs" "--test" "-Z" "unstable-options" "--edition=2018"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-18199.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-18199/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-18199" "/checkout/src/test/rustdoc/issue-18199.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-23106.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23106/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23106" "/checkout/src/test/rustdoc/issue-23106.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-23744.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23744/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23744" "/checkout/src/test/rustdoc/issue-23744.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-25944.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-25944/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-25944" "/checkout/src/test/rustdoc/issue-25944.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-30252.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-30252/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-30252" "/checkout/src/test/rustdoc/issue-30252.rs" "--test" "--cfg" "feature=\"bar\""
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-38129.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-38129/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-38129" "/checkout/src/test/rustdoc/issue-38129.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-43153.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-43153/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-43153" "/checkout/src/test/rustdoc/issue-43153.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/issue-48377.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-48377/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-48377" "/checkout/src/test/rustdoc/issue-48377.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/process-termination.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/process-termination/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/process-termination" "/checkout/src/test/rustdoc/process-termination.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] ---- [rustdoc] rustdoc/test_option_check/bar.rs stdout ----
[01:31:03] 
[01:31:03] error: rustdoc failed!
[01:31:03] status: exit code: 1
[01:31:03] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/test_option_check/bar/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/test_option_check/bar" "/checkout/src/test/rustdoc/test_option_check/bar.rs" "--test"
[01:31:03] ------------------------------------------
[01:31:03] 
[01:31:03] ------------------------------------------
[01:31:03] stderr:
---
[01:31:03] test result: FAILED. 287 passed; 16 failed; 2 ignored; 0 measured; 0 filtered out
[01:31:03] 
[01:31:03] 
[01:31:03] 
[01:31:03] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/rustdoc" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:31:03] 
[01:31:03] 
[01:31:03] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:31:03] Build completed unsuccessfully in 0:20:22
[01:31:03] Build completed unsuccessfully in 0:20:22
[01:31:03] make: *** [check] Error 1
[01:31:03] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:2e5380e8
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Apr 30 01:23:28 UTC 2019
---
travis_time:end:02952ba6:start=1556587410721652229,finish=1556587410788724794,duration=67072565
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:060b781c
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0755277b
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Apr 30, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:15839499:start=1556595333876738337,finish=1556595419273925913,duration=85397187576
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:20:11] 
[01:20:11] running 9 tests
[01:20:11] iiiiiiiii
[01:20:11] 
[01:20:11]  finished in 0.154
[01:20:11] travis_fold:end:test_assembly

---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:20:27] 
[01:20:27] running 121 tests
[01:20:52] .iiiii...i.....i..i...i..i.i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i. 100/121
[01:20:56] i.i......iii.i.....ii
[01:20:56] 
[01:20:56]  finished in 29.948
[01:20:56] travis_fold:end:test_debuginfo

---
travis_time:start:test_rustdoc
Check compiletest suite=rustdoc mode=rustdoc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:24:59] 
[01:24:59] running 305 tests
[01:26:09] ............F...............i...F.....FF............................................................ 100/305
[01:27:08] ................F................F...F.F...i.........F....................F.....F...............F... 200/305
[01:28:09] ...F.................................F.........................................FF................... 300/305
[01:28:12] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:28:12] .....
[01:28:12] failures:
[01:28:12] 
---
[01:28:12] ---- [rustdoc] rustdoc/doctest-manual-crate-name.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/doctest-manual-crate-name/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/doctest-manual-crate-name" "/checkout/src/test/rustdoc/doctest-manual-crate-name.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/edition-flag.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/edition-flag/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/edition-flag" "/checkout/src/test/rustdoc/edition-flag.rs" "--test" "-Z" "unstable-options" "--edition=2018"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-18199.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-18199/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-18199" "/checkout/src/test/rustdoc/issue-18199.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-23106.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23106/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23106" "/checkout/src/test/rustdoc/issue-23106.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-23744.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23744/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-23744" "/checkout/src/test/rustdoc/issue-23744.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-25944.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-25944/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-25944" "/checkout/src/test/rustdoc/issue-25944.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-30252.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-30252/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-30252" "/checkout/src/test/rustdoc/issue-30252.rs" "--test" "--cfg" "feature=\"bar\""
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-38129.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-38129/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-38129" "/checkout/src/test/rustdoc/issue-38129.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-43153.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-43153/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-43153" "/checkout/src/test/rustdoc/issue-43153.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/issue-48377.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-48377/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-48377" "/checkout/src/test/rustdoc/issue-48377.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/process-termination.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/process-termination/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/process-termination" "/checkout/src/test/rustdoc/process-termination.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] ---- [rustdoc] rustdoc/test_option_check/bar.rs stdout ----
[01:28:12] 
[01:28:12] error: rustdoc failed!
[01:28:12] status: exit code: 1
[01:28:12] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/test_option_check/bar/auxiliary" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/test_option_check/bar" "/checkout/src/test/rustdoc/test_option_check/bar.rs" "--test"
[01:28:12] ------------------------------------------
[01:28:12] 
[01:28:12] ------------------------------------------
[01:28:12] stderr:
---
[01:28:12] test result: FAILED. 287 passed; 16 failed; 2 ignored; 0 measured; 0 filtered out
[01:28:12] 
[01:28:12] 
[01:28:12] 
[01:28:12] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/rustdoc" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:28:12] 
[01:28:12] 
[01:28:12] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:28:12] Build completed unsuccessfully in 0:19:43
[01:28:12] Build completed unsuccessfully in 0:19:43
[01:28:12] Makefile:48: recipe for target 'check' failed
[01:28:12] make: *** [check] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0dab7055
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Apr 30 05:05:21 UTC 2019
---
travis_time:end:0d88f700:start=1556600723557060129,finish=1556600723561944083,duration=4883954
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:2afe235a
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:2ccfbdad
travis_time:start:2ccfbdad
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:30eeb2b2
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Apr 30, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:1e6997b4:start=1556649926925201123,finish=1556650043983212622,duration=117058011499
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:20:56] 
[01:20:56] running 9 tests
[01:20:56] iiiiiiiii
[01:20:56] 
[01:20:56]  finished in 0.152
[01:20:56] travis_fold:end:test_assembly

---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:21:12] 
[01:21:12] running 121 tests
[01:21:38] .iiiii...i.....i..i...i..i.i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i. 100/121
[01:21:43] i.i......iii.i.....ii
[01:21:43] 
[01:21:43]  finished in 31.227
[01:21:43] travis_fold:end:test_debuginfo

---
[01:45:45] 
[01:45:45] running 25 tests
[01:45:58] thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:517:22
[01:45:58] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:45:58] ............F............
[01:45:58] 
[01:45:58] ---- [ui] rustdoc-ui/failed-doctest-output.rs stdout ----
[01:45:58] diff of stdout:
[01:45:58] 
[01:45:58] 
[01:45:58] 15 error: aborting due to previous error
[01:45:58] 16 
[01:45:58] 17 For more information about this error, try `rustc --explain E0425`.
[01:45:58] - thread '$DIR/failed-doctest-output.rs - OtherStruct (line 17)' panicked at 'couldn't compile the test', src/librustdoc/test.rs:310:13
[01:45:58] + thread '$DIR/failed-doctest-output.rs - OtherStruct (line 17)' panicked at 'couldn't compile the test', src/librustdoc/test.rs:317:13
[01:45:58] 20 
[01:45:58] 20 
[01:45:58] 21 ---- $DIR/failed-doctest-output.rs - SomeStruct (line 11) stdout ----
[01:45:58] 
[01:45:58] 24 thread 'main' panicked at 'oh no', $DIR/failed-doctest-output.rs:3:1
[01:45:58] 26 
[01:45:58] - ', src/librustdoc/test.rs:332:17
[01:45:58] + ', src/librustdoc/test.rs:346:17
[01:45:58] 28 
[01:45:58] 28 
[01:45:58] 29 
[01:45:58] 30 failures:
[01:45:58] 
[01:45:58] 
[01:45:58] The actual stdout differed from the expected stdout.
[01:45:58] Actual stdout saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/failed-doctest-output/failed-doctest-output.stdout
[01:45:58] To update references, rerun the tests and pass the `--bless` flag
[01:45:58] To only update this specific test, also pass `--test-args failed-doctest-output.rs`
[01:45:58] error: 1 errors occurred comparing output.
[01:45:58] status: exit code: 101
[01:45:58] status: exit code: 101
[01:45:58] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "/checkout/src/test/rustdoc-ui/failed-doctest-output.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/failed-doctest-output/a" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--test" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/failed-doctest-output/auxiliary"
[01:45:58] ------------------------------------------
[01:45:58] 
[01:45:58] running 2 tests
[01:45:58] test /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - OtherStruct (line 17) ... FAILED
[01:45:58] test /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - OtherStruct (line 17) ... FAILED
[01:45:58] test /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - SomeStruct (line 11) ... FAILED
[01:45:58] failures:
[01:45:58] 
[01:45:58] ---- /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - OtherStruct (line 17) stdout ----
[01:45:58] error[E0425]: cannot find value `no` in this scope
---
[01:45:58] For more information about this error, try `rustc --explain E0425`.
[01:45:58] thread '/checkout/src/test/rustdoc-ui/failed-doctest-output.rs - OtherStruct (line 17)' panicked at 'couldn't compile the test', src/librustdoc/test.rs:317:13
[01:45:58] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:45:58] 
[01:45:58] ---- /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - SomeStruct (line 11) stdout ----
[01:45:58] thread '/checkout/src/test/rustdoc-ui/failed-doctest-output.rs - SomeStruct (line 11)' panicked at 'test executable failed:
[01:45:58] 
[01:45:58] thread 'main' panicked at 'oh no', /checkout/src/test/rustdoc-ui/failed-doctest-output.rs:3:1
[01:45:58] 
[01:45:58] ', src/librustdoc/test.rs:346:17
[01:45:58] 
[01:45:58] 
[01:45:58] 
[01:45:58] failures:
[01:45:58]     /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - OtherStruct (line 17)
[01:45:58]     /checkout/src/test/rustdoc-ui/failed-doctest-output.rs - SomeStruct (line 11)
[01:45:58] test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out
[01:45:58] 
[01:45:58] 
[01:45:58] ------------------------------------------
---
[01:45:58] 
[01:45:51:45:58] 
[01:45:58] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:45:58] Build completed unsuccessfully in 0:36:50
[01:45:58] Makefile:48: recipe for target 'check' failed
[01:45:58] make: *** [check] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:32677e6f
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Apr 30 20:33:31 UTC 2019

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Goirad Goirad force-pushed the Goirad:test-expansion branch from 3363db6 to df5eb19 May 8, 2019

@Goirad

This comment has been minimized.

Copy link
Author

commented May 14, 2019

Awaiting review, all checks pass now.

@jethrogb

This comment has been minimized.

Copy link
Contributor

commented May 15, 2019

compiletest does space-splitting on runtool, but rustdoc does not, currently. So if your runtool contains a space you'll get different behavior for rustdoc and compiletest.

@bors

This comment has been minimized.

Copy link
Contributor

commented May 22, 2019

☔️ The latest upstream changes (presumably #61027) made this pull request unmergeable. Please resolve the merge conflicts.

@Goirad Goirad closed this Jun 5, 2019

@Goirad Goirad reopened this Jun 5, 2019

@Goirad Goirad force-pushed the Goirad:test-expansion branch from df5eb19 to d9a1c2f Jun 5, 2019

@Goirad Goirad changed the title Improvements for cross-compiling and cross-running tests Allow cross-compiling doctests Jun 5, 2019

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jun 5, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:00ccd10c:start=1559775465663459616,finish=1559775553948126160,duration=88284666544
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:04:09] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:10] tidy error: /checkout/src/librustdoc/lib.rs:367: line longer than 100 chars
[00:04:15] some tidy checks failed
[00:04:15] 
[00:04:15] 
[00:04:15] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:15] 
[00:04:15] 
[00:04:15] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:15] Build completed unsuccessfully in 0:01:13
---
travis_time:end:233b8d5c:start=1559775819182289176,finish=1559775819187597253,duration=5308077
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:10f5e280
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:13ae8f84
travis_time:start:13ae8f84
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0df9d4ba
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Goirad

This comment has been minimized.

Copy link
Author

commented Jun 5, 2019

@kennytm I have restricted the scope of this PR to only adding runtool functionality to rustdoc, along with Cargo in the other PR.

@Goirad Goirad force-pushed the Goirad:test-expansion branch from d9a1c2f to c0b417b Jun 5, 2019

@kennytm
Copy link
Member

left a comment

Are those ignore changes still in the scope of this PR? They're still in the diff.

Show resolved Hide resolved src/librustdoc/lib.rs Outdated
Show resolved Hide resolved src/librustdoc/test.rs Outdated
@Goirad

This comment has been minimized.

Copy link
Author

commented Jun 6, 2019

Yes ignore is still a part of this PR, since being able to ignore on a per-target basis is relevant to using the crosscompilation feature.

@kennytm

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

🤔 It's better to delegate the review to @rust-lang/rustdoc then, as it is changing the behavior of ```rust,ignore-foo.

@Goirad

This comment has been minimized.

Copy link
Author

commented Jun 6, 2019

Sorry, how would that behavior change?
Edit: I get it, yea it changes what would have previously done nothing ("ignore-foo") to do something.

@Goirad Goirad force-pushed the Goirad:test-expansion branch from c0b417b to 33ceb2e Jun 6, 2019

@kennytm

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

@Goirad yeah that's what the "insta-stable" comment in #60387 (comment) is about.

Show resolved Hide resolved src/librustdoc/test.rs Outdated
@GuillaumeGomez

This comment has been minimized.

Copy link
Member

commented Jun 8, 2019

Ok, just one nit and then it's good for me.

Can you take a look as well @ollie27 please? :)

@Goirad Goirad force-pushed the Goirad:test-expansion branch from c588d39 to 9b5543d Jun 10, 2019

@Centril

This comment has been minimized.

Copy link
Member

commented Jun 10, 2019

@rust-highfive rust-highfive assigned ollie27 and unassigned kennytm Jun 10, 2019

@Goirad Goirad force-pushed the Goirad:test-expansion branch from c30fc2c to 063ad83 Jun 11, 2019

@QuietMisdreavus
Copy link
Member

left a comment

Thanks so much for writing this up! I have a couple comments, but it seems like the basic principle is sound. I may need to come back later on and give another read to be 100% sure of the complete implementation. Regardless, this is big! I'm excited to see this working.

Regarding bikeshedding the ignore- filters: One consequence of changing how we parse the lang strings like this is that when people start using new lang string options, older versions of rustdoc may no longer recognize those doctests, since they'll show up as an unknown lang string. However, in this case, if you write rust,ignore-something instead, then current versions of rustdoc will still see (and try to execute) the doctest. It may be worth allowing ignore-something to override ignore so that crates can still pass tests on older versions of rustdoc while the feature is still new. The lang string rust,ignore,ignore-something is a little verbose and seemingly contradictory, but hopefully once the feature lands it'll only be necessary as long as stable doesn't recognize it.

One thing i'm curious about is whether we should also support only running a specific doctest on a given filter, like compiletest's only- filters. I can imagine wanting to display some platform-specific behavior in a doctest, and adding ignore-linux,ignore-windows,ignore-bsd to your test to ensure it only runs on macOS (for example) quickly becomes way too cumbersome to properly ensure.

I'm also amused about how the ignore- feature overlaps somewhat with #[doc(cfg)]. That feature already restricts running doctests unless the target matches the given cfg string. In some sense, this is even better, since you can filter individual doctests, but (as of the current state of the PR) there's no visibility on the generated docs about where a doctest is intended to run. (With #[doc(cfg)], there's a banner printed on the item in question stating "this item is only available on Windows" or the like.)

It would be nice to see tests that exercise that doctests are run (or not run) on certain platforms. That may have to be a collection of tests in src/test/rustdoc-ui that filter the platform they run on and have some combination of flags that let it run the doctest (running a 32-bit doctest on its 64-bit counterpart and asserting that the bit width is correct, maybe?) combined with other tests that ensure that a doctest isn't run on the wrong platform (by ensuring that the console output states that the test was ignored).

I would also like to see some documentation in the Rustdoc Book (src/doc/rustdoc/src/unstable-features.md) that mention what these flags do. This is a fairly major feature, and relying on word-of-mouth or external documentation is less-than-ideal in making sure people try it out.

Show resolved Hide resolved src/librustdoc/test.rs Outdated
Show resolved Hide resolved src/librustdoc/test.rs Outdated
@@ -476,6 +487,9 @@ impl Options {
let generate_search_filter = !matches.opt_present("disable-per-crate-search");
let persist_doctests = matches.opt_str("persist-doctests").map(PathBuf::from);
let generate_redirect_pages = matches.opt_present("generate-redirect-pages");
let runtool = matches.opt_str("runtool");
let runtool_args = matches.opt_strs("runtool-arg");

This comment has been minimized.

Copy link
@QuietMisdreavus

QuietMisdreavus Jun 11, 2019

Member

This is what @jethrogb was talking about with the behavior of the runtool arguments. This just takes arguments as-is without splitting or reparsing them, which (apparently) is what compiletest does. A more robust solution would split the strings we get from opt_strs to fill out a separate Vec, rather than taking the list directly.

...however, after reviewing your companion PR on Cargo, it seems like you're going to lean on Cargo pre-parsing the runner arguments, so i'm not too concerned, in the end. I'd like to see whether/how this is going to be integrated with bootstrap - do we cross-compile other tests already? How is that orchestrated? (Who is the best person to ask these questions to? 😅)

This comment has been minimized.

Copy link
@Goirad

Goirad Jun 11, 2019

Author

I'm a bit confused by your first paragraph, it is my understanding that opt_strs returns a vec of all arguments passed to (possibly many) --runtool-arg's. The intention being that arguments get passed one at a time, one per --runtool-arg.

As for the question of integration with bootstrap, I am working on that now, but wanted to get the ball rolling here in rustdoc. My plan is to add an unstable flag to bootstrap that makes it pass --Zdoctest-xcompile to cargo test invocations. The current state is that compiletests do get cross-compiled, and it already has a --runtool option. My change to compiletest removes that option and instead uses Cargo to create the merged .cargo/config for a directory and use the runner specified there, with the idea being that it will ensure consistent testing across rustdoc and compiltest.

This comment has been minimized.

Copy link
@Goirad

Goirad Jun 11, 2019

Author

I see now what you meant in your first paragraph. You're right it would be more robust, I just omitted any error checking there, since worst case scenario one string that is in reality several instructions would just get passed wholesale to the runtool (which I think would not in the end be any different from adding them one at a time).
Also as far as who to ask, I don't know, but I welcome any suggestions since this seems to be pretty central to how the whole rust compiler building and testing process works.

This comment has been minimized.

Copy link
@QuietMisdreavus

QuietMisdreavus Jun 12, 2019

Member

(which I think would not in the end be any different from adding them one at a time)

Since we would be building up a Command instance with these args, the space-splitting wouldn't happen - the recipient would receive the blob in one "argument", as if it had all been quoted together.

One option would be to look at how many --runtool-arg arguments there are and treat it differently based on whether there are one or more than one. If there's only one argument, we could do space-splitting on that, and if there is more than one argument, we can leave them as-is. This somewhat mirrors Cargo's own parsing of the relevant configuration item, which whitespace-splits it is it's just a string, and leaves them alone if given a list. (This can backfire if someone tried to give a single argument with spaces in it, though... 🤔)

This comment has been minimized.

Copy link
@jethrogb

jethrogb Jun 12, 2019

Contributor

It's not possible to correctly space-split command-line arguments based on some heuristic if you also want to support arguments with spaces in them. Just let the cargo/the user take care of this. Most people won't be running rustdoc directly this way anyway.

Show resolved Hide resolved src/librustdoc/html/markdown.rs Outdated
Ignore::All => true,
Ignore::None => false,
Ignore::Some(ref ignores) => {
ignores.iter().any(|s| target_str.contains(s))

This comment has been minimized.

Copy link
@QuietMisdreavus

QuietMisdreavus Jun 11, 2019

Member

Is there a better method of testing the ignore- string than this? I was hoping we could reuse compiletest's filters, which don't seem like they could work with a substring search.

This comment has been minimized.

Copy link
@Goirad

Goirad Jun 11, 2019

Author

I have not investigated leveraging any existing machinery from compiltest's filters, although in the worst case it would not be too much trouble to extend the flags to handle only and some other target details.

This comment has been minimized.

Copy link
@jethrogb

jethrogb Jun 11, 2019

Contributor

I don't think the compiletest filters are particularly well-suited to stabilizing. They just define a custom list of keywords for matching platforms. Since compiletest is for internal use and this is not, I think we are going to have to put a little more thought into this.

I do agree that substring matching on the target name is a little strange. It would be nice to somehow hook into Rust's existing conditional compilation story, which is cfg.

This comment has been minimized.

Copy link
@Goirad

Goirad Jun 11, 2019

Author

In the end I think this is sufficient for an initial implementation of an unstable feature, and we can revisit it before stabilization.

@Goirad

This comment has been minimized.

Copy link
Author

commented Jun 11, 2019

First, thanks for the review @QuietMisdreavus ! I really appreciate the input. To address the rest of your comments, I can definitely add some tests, at least for the functionality I have added, as well as update the book.
As far as allowing ignore-something to override ignore, that seems like a fine idea, and fairly simple to implement, I can add that, assuming we don't leave this conversation with a substantial rework planned.
Also you mentioned the output of these filters when creating the rustdoc, and to be honest I hadn't given it much thought. I will say that I think I made it so that ignore != Ignore::None makes rustdoc output that the test will be ignored, although obviously there are more nuanced way to tackle that problem.

@bors

This comment has been minimized.

Copy link
Contributor

commented Jun 16, 2019

☔️ The latest upstream changes (presumably #60730) made this pull request unmergeable. Please resolve the merge conflicts.

@Goirad Goirad force-pushed the Goirad:test-expansion branch from 30586b6 to 0f4eff3 Jun 19, 2019

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jun 19, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:00788d36:start=1560965053027262726,finish=1560965055226862871,duration=2199600145
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:07:26] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:07:26] tidy error: /checkout/src/librustdoc/html/markdown.rs:628: line longer than 100 chars
[00:07:31] some tidy checks failed
[00:07:31] 
[00:07:31] 
[00:07:31] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:07:31] 
[00:07:31] 
[00:07:31] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:07:31] Build completed unsuccessfully in 0:01:14
---
travis_time:end:297cd26e:start=1560965517473985534,finish=1560965517478942316,duration=4956782
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0cf1f5c0
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0163075e
travis_time:start:0163075e
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0b712ad1
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Dario Gonzalez added some commits Jun 6, 2019

Dario Gonzalez
added feature gate enable-per-target-ignores
updated and augmented tests in html/markdown.rs
Dario Gonzalez
changed target from option to plain target, populated with host tripl…
…e at argument parsing time if no --target arguments

@Goirad Goirad force-pushed the Goirad:test-expansion branch from 0f4eff3 to 5b597b2 Jun 19, 2019

@Goirad

This comment has been minimized.

Copy link
Author

commented Jun 26, 2019

@ollie27 review ping

@Goirad

This comment has been minimized.

Copy link
Author

commented Jul 10, 2019

@ollie27 @QuietMisdreavus @GuillaumeGomez Can I get some review please?

@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 11, 2019

☔️ The latest upstream changes (presumably #63471) made this pull request unmergeable. Please resolve the merge conflicts.

@JohnCSimon

This comment has been minimized.

Copy link

commented Aug 17, 2019

ping from triage -
@Goirad @ollie27 @QuietMisdreavus @GuillaumeGomez
This PR has a merge conflict, but as soon as it's resolved it's ready for review.
Thank you for your contributions!

@ollie27 @QuietMisdreavus @GuillaumeGomez Can I get some review please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.