Allow checking of run-pass execution output in compiletest #63825

nathanwhit · 2019-08-23T00:28:38Z

Closes #63751
Adds a check-run-results flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.

Moved into ```fn load_compare_outputs(&self, proc_res: &ProcRes, explicit_format: bool) -> usize```

Addresses rust-lang#63751

rust-highfive · 2019-08-23T00:28:48Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

nathanwhit · 2019-08-23T00:29:10Z

r? @Mark-Simulacrum

Mark-Simulacrum

Thanks! A few comments, but this looks good overall.

src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs

src/tools/compiletest/src/runtest.rs

rust-highfive · 2019-08-25T23:46:08Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-25T23:26:58.9031804Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-25T23:26:58.9238611Z ##[command]git config gc.auto 0
2019-08-25T23:26:58.9324471Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-25T23:26:58.9375356Z ##[command]git config --get-all http.proxy
2019-08-25T23:26:58.9518002Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63825/merge:refs/remotes/pull/63825/merge
---
2019-08-25T23:27:34.0738372Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-25T23:27:34.0738404Z 
2019-08-25T23:27:34.0738605Z   git checkout -b <new-branch-name>
2019-08-25T23:27:34.0738635Z 
2019-08-25T23:27:34.0738705Z HEAD is now at 1132f1b32 Merge 3dc3cf30a45108c45df0492900224515d03b79ea into 521d78407471cb78e9bbf47160f6aa23047ac499
2019-08-25T23:27:34.0910922Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-25T23:27:34.0914541Z ==============================================================================
2019-08-25T23:27:34.0914601Z Task         : Bash
2019-08-25T23:27:34.0914650Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-25T23:33:54.7659866Z    Compiling serde_json v1.0.40
2019-08-25T23:33:56.5941106Z    Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-08-25T23:34:07.6290144Z     Finished release [optimized] target(s) in 1m 32s
2019-08-25T23:34:07.6364208Z tidy check
2019-08-25T23:34:08.1676169Z tidy error: /checkout/src/tools/compiletest/src/runtest.rs:4: line longer than 100 chars
2019-08-25T23:34:08.1677953Z tidy error: /checkout/src/tools/compiletest/src/runtest.rs:2942: line longer than 100 chars
2019-08-25T23:34:09.6212119Z some tidy checks failed
2019-08-25T23:34:09.6213546Z 
2019-08-25T23:34:09.6213546Z 
2019-08-25T23:34:09.6214853Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-08-25T23:34:09.6215467Z 
2019-08-25T23:34:09.6215653Z 
2019-08-25T23:34:09.6220405Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-08-25T23:34:09.6220891Z Build completed unsuccessfully in 0:01:35
2019-08-25T23:34:09.6220891Z Build completed unsuccessfully in 0:01:35
2019-08-25T23:34:09.6269335Z == clock drift check ==
2019-08-25T23:34:09.6283445Z   local time: Sun Aug 25 23:34:09 UTC 2019
2019-08-25T23:34:09.7768637Z   network time: Sun, 25 Aug 2019 23:34:09 GMT
2019-08-25T23:34:09.7776388Z == end clock drift check ==
2019-08-25T23:34:11.1112099Z ##[error]Bash exited with code '1'.
2019-08-25T23:34:11.1145239Z ##[section]Starting: Checkout
2019-08-25T23:34:11.1146981Z ==============================================================================
2019-08-25T23:34:11.1147033Z Task         : Get sources
2019-08-25T23:34:11.1147094Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-26T01:04:09Z

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-26T00:03:26.6088551Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-26T00:03:26.6292091Z ##[command]git config gc.auto 0
2019-08-26T00:03:26.6355363Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-26T00:03:26.6407770Z ##[command]git config --get-all http.proxy
2019-08-26T00:03:26.6533914Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63825/merge:refs/remotes/pull/63825/merge
---
2019-08-26T00:04:01.6262190Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-26T00:04:01.6262489Z 
2019-08-26T00:04:01.6262951Z   git checkout -b <new-branch-name>
2019-08-26T00:04:01.6263261Z 
2019-08-26T00:04:01.6263521Z HEAD is now at ce9430cbf Merge 1c6b98647b9e7cf44d2bbea5497e7ec1bae329f9 into 521d78407471cb78e9bbf47160f6aa23047ac499
2019-08-26T00:04:01.6429693Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-26T00:04:01.6432562Z ==============================================================================
2019-08-26T00:04:01.6432617Z Task         : Bash
2019-08-26T00:04:01.6432658Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-26T00:13:38.1510567Z configure: build.locked-deps    := True
2019-08-26T00:13:38.1510635Z configure: llvm.ccache          := sccache
2019-08-26T00:13:38.1510893Z configure: build.cargo-native-static := True
2019-08-26T00:13:38.1511129Z configure: dist.missing-tools   := True
2019-08-26T00:13:38.1511409Z configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
2019-08-26T00:13:38.1511534Z configure: writing `config.toml` in current directory
2019-08-26T00:13:38.1511582Z configure: 
2019-08-26T00:13:38.1511820Z configure: run `python /checkout/x.py --help`
2019-08-26T00:13:38.1511888Z configure: 
---
2019-08-26T00:22:15.1772440Z    Compiling serde v1.0.99
2019-08-26T00:22:24.1612047Z    Compiling serde_json v1.0.40
2019-08-26T00:22:25.9333716Z    Compiling rustfix v0.4.6
2019-08-26T00:22:29.6875956Z    Compiling compiletest v0.0.0 (/checkout/src/tools/compiletest)
2019-08-26T00:22:29.7226465Z error: expected `;`, found keyword `use`
2019-08-26T00:22:29.7227087Z   |
2019-08-26T00:22:29.7227087Z   |
2019-08-26T00:22:29.7227372Z 4 | use crate::common::{expected_output_path, UI_EXTENSIONS, UI_FIXED, UI_STDERR, UI_STDOUT}
2019-08-26T00:22:29.7227728Z   |                                                                                         - expected `;`
2019-08-26T00:22:29.7227990Z 5 | use crate::{UI_RUN_STDERR, UI_RUN_STDOUT};
2019-08-26T00:22:29.7228333Z 
2019-08-26T00:22:29.7229184Z error: aborting due to previous error
2019-08-26T00:22:29.7229229Z 
2019-08-26T00:22:29.7289733Z error: Could not compile `compiletest`.
---
2019-08-26T00:22:29.7365784Z == clock drift check ==
2019-08-26T00:22:29.7381807Z   local time: Mon Aug 26 00:22:29 UTC 2019
2019-08-26T00:22:30.0165878Z   network time: Mon, 26 Aug 2019 00:22:30 GMT
2019-08-26T00:22:30.0170539Z == end clock drift check ==
2019-08-26T00:22:31.4914643Z ##[error]Bash exited with code '1'.
2019-08-26T00:22:31.4957212Z ##[section]Starting: Checkout
2019-08-26T00:22:31.4958893Z ==============================================================================
2019-08-26T00:22:31.4958943Z Task         : Get sources
2019-08-26T00:22:31.4958986Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-26T02:19:45Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-26T01:11:42.2304245Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-26T01:11:42.2485061Z ##[command]git config gc.auto 0
2019-08-26T01:11:42.2532536Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-26T01:11:42.2582171Z ##[command]git config --get-all http.proxy
2019-08-26T01:11:42.2707133Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63825/merge:refs/remotes/pull/63825/merge
---
2019-08-26T01:12:16.2442000Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-26T01:12:16.2442023Z 
2019-08-26T01:12:16.2442184Z   git checkout -b <new-branch-name>
2019-08-26T01:12:16.2442207Z 
2019-08-26T01:12:16.2442261Z HEAD is now at 258f84984 Merge 29cc305e936d9637f12f37e0fb6e38e04009d610 into 521d78407471cb78e9bbf47160f6aa23047ac499
2019-08-26T01:12:16.2581600Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-26T01:12:16.2584017Z ==============================================================================
2019-08-26T01:12:16.2584060Z Task         : Bash
2019-08-26T01:12:16.2584110Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-26T02:09:48.6788006Z .................................................................................................... 1500/8953
2019-08-26T02:09:53.9076357Z .................................................................................................... 1600/8953
2019-08-26T02:10:05.5501478Z .............................................i...............i...................................... 1700/8953
2019-08-26T02:10:13.0023819Z .................................................................................................... 1800/8953
2019-08-26T02:10:25.9442778Z .....................................iiiii.......................................................... 1900/8953
2019-08-26T02:10:35.4629534Z .................................................................................................... 2100/8953
2019-08-26T02:10:37.7014429Z .................................................................................................... 2200/8953
2019-08-26T02:10:41.5578349Z .................................................................................................... 2300/8953
2019-08-26T02:10:48.1215673Z .................................................................................................... 2400/8953
---
2019-08-26T02:13:29.9595274Z .........................i...............i.......................................................... 4700/8953
2019-08-26T02:13:40.6811563Z .................................................................................................... 4800/8953
2019-08-26T02:13:46.6843206Z .................................................................................................... 4900/8953
2019-08-26T02:13:56.3073241Z .................................................................................................... 5000/8953
2019-08-26T02:14:01.3981268Z ......ii.ii......................................................................................... 5100/8953
2019-08-26T02:14:14.4263835Z .................................................................................................... 5300/8953
2019-08-26T02:14:20.8232614Z ..............................................................i..................................... 5400/8953
2019-08-26T02:14:27.2372657Z .................................................................................................... 5500/8953
2019-08-26T02:14:34.3069123Z .................................................................................................... 5600/8953
2019-08-26T02:14:34.3069123Z .................................................................................................... 5600/8953
2019-08-26T02:14:44.2388744Z ........................................................ii...i..ii...........i...................... 5700/8953
2019-08-26T02:15:05.3834533Z .................................................................................................... 5900/8953
2019-08-26T02:15:09.8345842Z .................................................................................................... 6000/8953
2019-08-26T02:15:09.8345842Z .................................................................................................... 6000/8953
2019-08-26T02:15:16.2958225Z .........................................................i..ii...................................... 6100/8953
2019-08-26T02:15:41.1398436Z .................................................................................................... 6300/8953
2019-08-26T02:15:43.0168886Z ...i................................................................................................ 6400/8953
2019-08-26T02:15:44.9037814Z ...........................................................................i........................ 6500/8953
2019-08-26T02:15:47.2749594Z .................................................................................................... 6600/8953
---
2019-08-26T02:19:20.9718377Z 
2019-08-26T02:19:20.9719844Z ---- [ui] ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs stdout ----
2019-08-26T02:19:20.9720100Z diff of run.stderr:
2019-08-26T02:19:20.9720255Z 
2019-08-26T02:19:20.9720724Z - [$DIR/dbg-macro-expected-behavior.rs:20] Unit = Unit
2019-08-26T02:19:20.9721232Z - [$DIR/dbg-macro-expected-behavior.rs:21] a = Unit
2019-08-26T02:19:20.9721691Z - [$DIR/dbg-macro-expected-behavior.rs:27] Point{x: 42, y: 24,} = Point {
2019-08-26T02:19:20.9722464Z + [$DIR/dbg-macro-expected-behavior.rs:23] Unit = Unit
2019-08-26T02:19:20.9723086Z + [$DIR/dbg-macro-expected-behavior.rs:24] a = Unit
2019-08-26T02:19:20.9723678Z + [$DIR/dbg-macro-expected-behavior.rs:30] Point{x: 42, y: 24,} = Point {
2019-08-26T02:19:20.9723868Z 4     x: 42,
2019-08-26T02:19:20.9724181Z 5     y: 24,
2019-08-26T02:19:20.9724625Z 
2019-08-26T02:19:20.9724625Z 
2019-08-26T02:19:20.9725153Z - [$DIR/dbg-macro-expected-behavior.rs:28] b = Point {
2019-08-26T02:19:20.9725708Z + [$DIR/dbg-macro-expected-behavior.rs:31] b = Point {
2019-08-26T02:19:20.9761533Z 8     x: 42,
2019-08-26T02:19:20.9813326Z 9     y: 24,
2019-08-26T02:19:20.9813823Z 
2019-08-26T02:19:20.9813823Z 
2019-08-26T02:19:20.9814230Z - [$DIR/dbg-macro-expected-behavior.rs:36]
2019-08-26T02:19:20.9814450Z - [$DIR/dbg-macro-expected-behavior.rs:40] &a = NoCopy(
2019-08-26T02:19:20.9814828Z + [$DIR/dbg-macro-expected-behavior.rs:39]
2019-08-26T02:19:20.9815166Z + [$DIR/dbg-macro-expected-behavior.rs:43] &a = NoCopy(
2019-08-26T02:19:20.9815474Z 14 )
2019-08-26T02:19:20.9815474Z 14 )
2019-08-26T02:19:20.9815717Z - [$DIR/dbg-macro-expected-behavior.rs:40] dbg!(& a) = NoCopy(
2019-08-26T02:19:20.9816092Z + [$DIR/dbg-macro-expected-behavior.rs:43] dbg!(& a) = NoCopy(
2019-08-26T02:19:20.9816397Z 17 )
2019-08-26T02:19:20.9816397Z 17 )
2019-08-26T02:19:20.9816657Z - [$DIR/dbg-macro-expected-behavior.rs:45] f(&42) = 42
2019-08-26T02:19:20.9816999Z + [$DIR/dbg-macro-expected-behavior.rs:48] f(&42) = 42
2019-08-26T02:19:20.9817151Z 19 before
2019-08-26T02:19:20.9817508Z - [$DIR/dbg-macro-expected-behavior.rs:50] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-08-26T02:19:20.9817704Z - [$DIR/dbg-macro-expected-behavior.rs:58] ("Yeah",) = (
2019-08-26T02:19:20.9818089Z + [$DIR/dbg-macro-expected-behavior.rs:53] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-08-26T02:19:20.9818433Z + [$DIR/dbg-macro-expected-behavior.rs:61] ("Yeah",) = (
2019-08-26T02:19:20.9818584Z 22     "Yeah",
2019-08-26T02:19:20.9818723Z 23 )
2019-08-26T02:19:20.9819378Z - [$DIR/dbg-macro-expected-behavior.rs:61] 1 = 1
2019-08-26T02:19:20.9819840Z - [$DIR/dbg-macro-expected-behavior.rs:61] 2 = 2
2019-08-26T02:19:20.9821410Z - [$DIR/dbg-macro-expected-behavior.rs:65] 1u8 = 1
2019-08-26T02:19:20.9821878Z - [$DIR/dbg-macro-expected-behavior.rs:65] 2u32 = 2
2019-08-26T02:19:20.9824258Z - [$DIR/dbg-macro-expected-behavior.rs:65] "Yeah" = "Yeah"
2019-08-26T02:19:20.9825482Z + [$DIR/dbg-macro-expected-behavior.rs:64] 1 = 1
2019-08-26T02:19:20.9825867Z + [$DIR/dbg-macro-expected-behavior.rs:64] 2 = 2
2019-08-26T02:19:20.9826234Z + [$DIR/dbg-macro-expected-behavior.rs:68] 1u8 = 1
2019-08-26T02:19:20.9826594Z + [$DIR/dbg-macro-expected-behavior.rs:68] 2u32 = 2
2019-08-26T02:19:20.9826953Z + [$DIR/dbg-macro-expected-behavior.rs:68] "Yeah" = "Yeah"
2019-08-26T02:19:20.9827390Z 
2019-08-26T02:19:20.9827414Z 
2019-08-26T02:19:20.9827479Z The actual run.stderr differed from the expected run.stderr.
2019-08-26T02:19:20.9827479Z The actual run.stderr differed from the expected run.stderr.
2019-08-26T02:19:20.9828019Z Actual run.stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/dbg-macro-expected-behavior.run.stderr
2019-08-26T02:19:20.9875378Z error: 1 errors occured comparing run output.
2019-08-26T02:19:20.9875516Z status: exit code: 0
2019-08-26T02:19:20.9875994Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a"
2019-08-26T02:19:20.9876046Z stdout:
2019-08-26T02:19:20.9876046Z stdout:
2019-08-26T02:19:20.9876262Z ------------------------------------------
2019-08-26T02:19:20.9876909Z 
2019-08-26T02:19:20.9877176Z ------------------------------------------
2019-08-26T02:19:20.9877237Z stderr:
2019-08-26T02:19:20.9877422Z ------------------------------------------
2019-08-26T02:19:20.9877646Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:23] Unit = Unit
2019-08-26T02:19:20.9877882Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:24] a = Unit
2019-08-26T02:19:20.9878298Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:30] Point{x: 42, y: 24,} = Point {
2019-08-26T02:19:20.9878346Z     x: 42,
2019-08-26T02:19:20.9878392Z     y: 24,
2019-08-26T02:19:20.9878427Z }
2019-08-26T02:19:20.9878686Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:31] b = Point {
2019-08-26T02:19:20.9878742Z     x: 42,
2019-08-26T02:19:20.9878777Z     y: 24,
2019-08-26T02:19:20.9880478Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:39]
2019-08-26T02:19:20.9880478Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:39]
2019-08-26T02:19:20.9881330Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:43] &a = NoCopy(
2019-08-26T02:19:20.9881434Z )
2019-08-26T02:19:20.9881434Z )
2019-08-26T02:19:20.9881744Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:43] dbg!(& a) = NoCopy(
2019-08-26T02:19:20.9882206Z )
2019-08-26T02:19:20.9882206Z )
2019-08-26T02:19:20.9882743Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:48] f(&42) = 42
2019-08-26T02:19:20.9882787Z before
2019-08-26T02:19:20.9883468Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:53] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-08-26T02:19:20.9884034Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] ("Yeah",) = (
2019-08-26T02:19:20.9884090Z     "Yeah",
2019-08-26T02:19:20.9884123Z )
2019-08-26T02:19:20.9884373Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:64] 1 = 1
2019-08-26T02:19:20.9884605Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:64] 2 = 2
2019-08-26T02:19:20.9884833Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] 1u8 = 1
2019-08-26T02:19:20.9885073Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] 2u32 = 2
2019-08-26T02:19:20.9885306Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] "Yeah" = "Yeah"
2019-08-26T02:19:20.9885532Z ------------------------------------------
2019-08-26T02:19:20.9885570Z 
2019-08-26T02:19:20.9885592Z 
2019-08-26T02:19:20.9885613Z 
---
2019-08-26T02:19:20.9886715Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-08-26T02:19:20.9886763Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-26T02:19:20.9886789Z 
2019-08-26T02:19:20.9886809Z 
2019-08-26T02:19:20.9888002Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-26T02:19:20.9888281Z 
2019-08-26T02:19:20.9888306Z 
2019-08-26T02:19:20.9888359Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-26T02:19:20.9888406Z Build completed unsuccessfully in 1:00:54
2019-08-26T02:19:20.9888406Z Build completed unsuccessfully in 1:00:54
2019-08-26T02:19:20.9888490Z == clock drift check ==
2019-08-26T02:19:20.9898371Z   local time: Mon Aug 26 02:19:20 UTC 2019
2019-08-26T02:19:21.1387764Z   network time: Mon, 26 Aug 2019 02:19:21 GMT
2019-08-26T02:19:21.1387926Z == end clock drift check ==
2019-08-26T02:19:22.0396707Z ##[error]Bash exited with code '1'.
2019-08-26T02:19:22.0436057Z ##[section]Starting: Checkout
2019-08-26T02:19:22.0437718Z ==============================================================================
2019-08-26T02:19:22.0437777Z Task         : Get sources
2019-08-26T02:19:22.0437833Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Mark-Simulacrum · 2019-08-26T20:30:55Z

I've filed #63924 to follow up on my comment about the ignores, but this looks good to go. Thanks so much for getting it done!

@bors r+

bors · 2019-08-26T20:30:56Z

📌 Commit cdf3beb178dd6ae8eabb7519b6b7007e8d30e19a has been approved by Mark-Simulacrum

src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs

bors · 2019-08-28T11:06:56Z

⌛ Testing commit cdf3beb178dd6ae8eabb7519b6b7007e8d30e19a with merge f970d3d042fcb718c2ab908190bf0cbbb47a8271...

bors · 2019-08-28T13:42:59Z

💔 Test failed - checks-azure

rust-highfive · 2019-08-28T13:43:01Z

The job armhf-gnu of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-28T11:08:19.1766441Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-28T11:08:19.1766503Z 
2019-08-28T11:08:19.1767303Z   git checkout -b <new-branch-name>
2019-08-28T11:08:19.1767394Z 
2019-08-28T11:08:19.1767803Z HEAD is now at f970d3d04 Auto merge of #63825 - nathanwhit:check-run-results, r=Mark-Simulacrum
2019-08-28T11:08:19.1990385Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-28T11:08:19.1994216Z ==============================================================================
2019-08-28T11:08:19.1994342Z Task         : Bash
2019-08-28T11:08:19.1994423Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-28T13:40:02.0775712Z 
2019-08-28T13:40:02.0775896Z failures:
2019-08-28T13:40:02.0823701Z 
2019-08-28T13:40:02.0824877Z ---- [ui] ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs stdout ----
2019-08-28T13:40:02.0825319Z normalized run.stdout:
2019-08-28T13:40:02.0829161Z uploaded "$TEST_BUILD_DIR/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-08-28T13:40:02.0829597Z 
2019-08-28T13:40:02.0829662Z 
2019-08-28T13:40:02.0829773Z The actual run.stdout differed from the expected run.stdout.
2019-08-28T13:40:02.0829773Z The actual run.stdout differed from the expected run.stdout.
2019-08-28T13:40:02.0830543Z Actual run.stdout saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/dbg-macro-expected-behavior.run.stdout
2019-08-28T13:40:02.0830819Z error: 1 errors occured comparing run output.
2019-08-28T13:40:02.0830920Z status: exit code: 0
2019-08-28T13:40:02.0830920Z status: exit code: 0
2019-08-28T13:40:02.0831434Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "run" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a"
2019-08-28T13:40:02.0831885Z ------------------------------------------
2019-08-28T13:40:02.0831885Z ------------------------------------------
2019-08-28T13:40:02.0832317Z uploaded "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-08-28T13:40:02.0832709Z ------------------------------------------
2019-08-28T13:40:02.0832791Z stderr:
2019-08-28T13:40:02.0833081Z ------------------------------------------
2019-08-28T13:40:02.0833081Z ------------------------------------------
2019-08-28T13:40:02.0833429Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:23] Unit = Unit
2019-08-28T13:40:02.0833806Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:24] a = Unit
2019-08-28T13:40:02.0834195Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:30] Point{x: 42, y: 24,} = Point {
2019-08-28T13:40:02.0834317Z     x: 42,
2019-08-28T13:40:02.0834385Z     y: 24,
2019-08-28T13:40:02.0834478Z }
2019-08-28T13:40:02.0834817Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:31] b = Point {
2019-08-28T13:40:02.0834933Z     x: 42,
2019-08-28T13:40:02.0835001Z     y: 24,
2019-08-28T13:40:02.0835672Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:39]
2019-08-28T13:40:02.0835672Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:39]
2019-08-28T13:40:02.0836149Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:43] &a = NoCopy(
2019-08-28T13:40:02.0836329Z )
2019-08-28T13:40:02.0836329Z )
2019-08-28T13:40:02.0836681Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:43] dbg!(& a) = NoCopy(
2019-08-28T13:40:02.0836866Z )
2019-08-28T13:40:02.0836866Z )
2019-08-28T13:40:02.0837579Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:48] f(&42) = 42
2019-08-28T13:40:02.0837683Z before
2019-08-28T13:40:02.0838079Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:53] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-08-28T13:40:02.0838696Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] ("Yeah",) = (
2019-08-28T13:40:02.0838812Z     "Yeah",
2019-08-28T13:40:02.0838877Z )
2019-08-28T13:40:02.0839200Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:64] 1 = 1
2019-08-28T13:40:02.0839533Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:64] 2 = 2
2019-08-28T13:40:02.0839885Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] 1u8 = 1
2019-08-28T13:40:02.0840232Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] 2u32 = 2
2019-08-28T13:40:02.0840569Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:68] "Yeah" = "Yeah"
2019-08-28T13:40:02.0840910Z ------------------------------------------
2019-08-28T13:40:02.0840962Z 
2019-08-28T13:40:02.0840999Z 
2019-08-28T13:40:02.0841055Z 
---
2019-08-28T13:40:02.0891443Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-08-28T13:40:02.0891994Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-28T13:40:02.0918096Z 
2019-08-28T13:40:02.0918540Z 
2019-08-28T13:40:02.0921216Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-arm-unknown-linux-gnueabihf" "--mode" "ui" "--target" "arm-unknown-linux-gnueabihf" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--linker" "arm-linux-gnueabihf-gcc" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/arm-unknown-linux-gnueabihf/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--llvm-version" "9.0.0-rust-1.39.0-dev\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--remote-test-client" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-28T13:40:02.0922057Z 
2019-08-28T13:40:02.0922122Z 
2019-08-28T13:40:02.0932223Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target arm-unknown-linux-gnueabihf
2019-08-28T13:40:02.0932823Z Build completed unsuccessfully in 2:26:45
2019-08-28T13:40:02.0932823Z Build completed unsuccessfully in 2:26:45
2019-08-28T13:40:02.1000476Z == clock drift check ==
2019-08-28T13:40:02.1019075Z   local time: Wed Aug 28 13:40:02 UTC 2019
2019-08-28T13:40:02.1751209Z   network time: Wed, 28 Aug 2019 13:40:02 GMT
2019-08-28T13:40:02.1755768Z == end clock drift check ==
2019-08-28T13:40:02.9521818Z ##[error]Bash exited with code '1'.
2019-08-28T13:40:02.9564433Z ##[section]Starting: Upload CPU usage statistics
2019-08-28T13:40:02.9571751Z ==============================================================================
2019-08-28T13:40:02.9571835Z Task         : Bash
2019-08-28T13:40:02.9571913Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-09-02T00:26:53Z

The job arm-android of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-02T00:24:47.0255789Z 
2019-09-02T00:24:47.0255879Z failures:
2019-09-02T00:24:47.0289739Z 
2019-09-02T00:24:47.0290473Z ---- [ui] ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs stdout ----
2019-09-02T00:24:47.0290591Z normalized run.stdout:
2019-09-02T00:24:47.0291077Z uploaded "$TEST_BUILD_DIR/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-09-02T00:24:47.0292371Z 
2019-09-02T00:24:47.0292405Z 
2019-09-02T00:24:47.0292503Z The actual run.stdout differed from the expected run.stdout.
2019-09-02T00:24:47.0292503Z The actual run.stdout differed from the expected run.stdout.
2019-09-02T00:24:47.0293042Z Actual run.stdout saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/dbg-macro-expected-behavior.run.stdout
2019-09-02T00:24:47.0293230Z error: 1 errors occured comparing run output.
2019-09-02T00:24:47.0293387Z status: exit code: 0
2019-09-02T00:24:47.0293387Z status: exit code: 0
2019-09-02T00:24:47.0293868Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "run" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a"
2019-09-02T00:24:47.0294435Z ------------------------------------------
2019-09-02T00:24:47.0294435Z ------------------------------------------
2019-09-02T00:24:47.0294866Z uploaded "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-09-02T00:24:47.0295493Z ------------------------------------------
2019-09-02T00:24:47.0295556Z stderr:
2019-09-02T00:24:47.0295783Z ------------------------------------------
2019-09-02T00:24:47.0295783Z ------------------------------------------
2019-09-02T00:24:47.0296048Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:20] Unit = Unit
2019-09-02T00:24:47.0296343Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:21] a = Unit
2019-09-02T00:24:47.0296829Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:27] Point{x: 42, y: 24,} = Point {
2019-09-02T00:24:47.0296910Z     x: 42,
2019-09-02T00:24:47.0296981Z     y: 24,
2019-09-02T00:24:47.0297035Z }
2019-09-02T00:24:47.0297327Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:28] b = Point {
2019-09-02T00:24:47.0297401Z     x: 42,
2019-09-02T00:24:47.0297472Z     y: 24,
2019-09-02T00:24:47.0297795Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:36]
2019-09-02T00:24:47.0297795Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:36]
2019-09-02T00:24:47.0298086Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:40] &a = NoCopy(
2019-09-02T00:24:47.0298230Z )
2019-09-02T00:24:47.0298230Z )
2019-09-02T00:24:47.0298524Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:40] dbg!(& a) = NoCopy(
2019-09-02T00:24:47.0298671Z )
2019-09-02T00:24:47.0298671Z )
2019-09-02T00:24:47.0298936Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:45] f(&42) = 42
2019-09-02T00:24:47.0299027Z before
2019-09-02T00:24:47.0299333Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:50] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-09-02T00:24:47.0299657Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:58] ("Yeah",) = (
2019-09-02T00:24:47.0299731Z     "Yeah",
2019-09-02T00:24:47.0299800Z )
2019-09-02T00:24:47.0300061Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] 1 = 1
2019-09-02T00:24:47.0300355Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] 2 = 2
2019-09-02T00:24:47.0300641Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] 1u8 = 1
2019-09-02T00:24:47.0300940Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] 2u32 = 2
2019-09-02T00:24:47.0301855Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] "Yeah" = "Yeah"
2019-09-02T00:24:47.0302204Z ------------------------------------------
2019-09-02T00:24:47.0302271Z 
2019-09-02T00:24:47.0302305Z 
2019-09-02T00:24:47.0302338Z 
---
2019-09-02T00:24:47.0344865Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-09-02T00:24:47.0345162Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-09-02T00:24:47.0363412Z 
2019-09-02T00:24:47.0363552Z 
2019-09-02T00:24:47.0366255Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-linux-androideabi/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-arm-linux-androideabi" "--mode" "ui" "--target" "arm-linux-androideabi" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--linker" "/android/ndk/arm-14/bin/arm-linux-androideabi-clang" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/arm-linux-androideabi/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--llvm-version" "9.0.0-rust-1.39.0-dev\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--remote-test-client" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "/android/ndk/arm-14" "--color" "always"
2019-09-02T00:24:47.0367393Z 
2019-09-02T00:24:47.0367530Z 
2019-09-02T00:24:47.0375124Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target arm-linux-androideabi
2019-09-02T00:24:47.0375958Z Build completed unsuccessfully in 2:09:12
2019-09-02T00:24:47.0375958Z Build completed unsuccessfully in 2:09:12
2019-09-02T00:24:47.0434074Z == clock drift check ==
2019-09-02T00:24:47.0455085Z   local time: Mon Sep  2 00:24:47 UTC 2019
2019-09-02T00:24:47.3176369Z   network time: Mon, 02 Sep 2019 00:24:47 GMT
2019-09-02T00:24:47.3182244Z == end clock drift check ==
2019-09-02T00:24:48.4924561Z ##[error]Bash exited with code '1'.
2019-09-02T00:24:48.4959129Z ##[section]Starting: Upload CPU usage statistics
2019-09-02T00:24:48.4963693Z ==============================================================================
2019-09-02T00:24:48.4963807Z Task         : Bash
2019-09-02T00:24:48.4963879Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-09-02T00:27:08Z

💔 Test failed - checks-azure

RalfJung · 2019-09-02T11:18:47Z

Hm, this makes them "pass" on that platform though, instead of being marked as "ignored" as it probably should.
@Mark-Simulacrum should know if that is acceptable.

Mark-Simulacrum · 2019-09-02T12:43:17Z

Yeah, I'd rather we didn't mark them as having passed.

nathanwhit · 2019-09-02T15:59:49Z

Okay, I'll go ahead and make that change.
My thinking behind running instead of ignoring was that even though we can't check the output of execution for that target, we can still check the compiler output and pass/fail based on that, but I can see how that could be undesirable.

Ignores run-pass tests with the `check-run-results` flag enabled for the wasm32-bare ("wasm32-unknown-unknown") target, as it does not support printing to stdout/stderr.

Mark-Simulacrum · 2019-09-02T17:21:01Z

Yeah, I think it's a bit ambiguous as to what the correct behavior there is but overall I'd prefer to ignore the test entirely I think. We can of course change that at a later point.

@bors r+

bors · 2019-09-02T17:21:02Z

📌 Commit bf44f12 has been approved by Mark-Simulacrum

bors · 2019-09-03T02:00:28Z

⌛ Testing commit bf44f12 with merge c11b9bb...

Allow checking of run-pass execution output in compiletest Closes #63751 Adds a `check-run-results` flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.

rust-highfive · 2019-09-03T04:32:51Z

The job armhf-gnu of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-03T04:31:13.4366953Z 
2019-09-03T04:31:13.4367018Z failures:
2019-09-03T04:31:13.4403774Z 
2019-09-03T04:31:13.4404771Z ---- [ui] ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs stdout ----
2019-09-03T04:31:13.4405092Z normalized run.stdout:
2019-09-03T04:31:13.4405523Z uploaded "$TEST_BUILD_DIR/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-09-03T04:31:13.4405831Z 
2019-09-03T04:31:13.4406167Z 
2019-09-03T04:31:13.4406429Z The actual run.stdout differed from the expected run.stdout.
2019-09-03T04:31:13.4406429Z The actual run.stdout differed from the expected run.stdout.
2019-09-03T04:31:13.4406973Z Actual run.stdout saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/dbg-macro-expected-behavior.run.stdout
2019-09-03T04:31:13.4407372Z error: 1 errors occured comparing run output.
2019-09-03T04:31:13.4407486Z status: exit code: 0
2019-09-03T04:31:13.4407486Z status: exit code: 0
2019-09-03T04:31:13.4408035Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "run" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a"
2019-09-03T04:31:13.4408639Z ------------------------------------------
2019-09-03T04:31:13.4408639Z ------------------------------------------
2019-09-03T04:31:13.4409191Z uploaded "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior/a", waiting for result
2019-09-03T04:31:13.4409773Z ------------------------------------------
2019-09-03T04:31:13.4410011Z stderr:
2019-09-03T04:31:13.4410668Z ------------------------------------------
2019-09-03T04:31:13.4410668Z ------------------------------------------
2019-09-03T04:31:13.4411283Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:20] Unit = Unit
2019-09-03T04:31:13.4411855Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:21] a = Unit
2019-09-03T04:31:13.4412408Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:27] Point{x: 42, y: 24,} = Point {
2019-09-03T04:31:13.4412672Z     x: 42,
2019-09-03T04:31:13.4412766Z     y: 24,
2019-09-03T04:31:13.4412862Z }
2019-09-03T04:31:13.4413225Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:28] b = Point {
2019-09-03T04:31:13.4413489Z     x: 42,
2019-09-03T04:31:13.4413597Z     y: 24,
2019-09-03T04:31:13.4414031Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:36]
2019-09-03T04:31:13.4414031Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:36]
2019-09-03T04:31:13.4414544Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:40] &a = NoCopy(
2019-09-03T04:31:13.4414925Z )
2019-09-03T04:31:13.4414925Z )
2019-09-03T04:31:13.4415297Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:40] dbg!(& a) = NoCopy(
2019-09-03T04:31:13.4415466Z )
2019-09-03T04:31:13.4415466Z )
2019-09-03T04:31:13.4415809Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:45] f(&42) = 42
2019-09-03T04:31:13.4415915Z before
2019-09-03T04:31:13.4416279Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:50] { foo += 1; eprintln!("before"); 7331 } = 7331
2019-09-03T04:31:13.4416901Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:58] ("Yeah",) = (
2019-09-03T04:31:13.4417174Z     "Yeah",
2019-09-03T04:31:13.4417269Z )
2019-09-03T04:31:13.4417692Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] 1 = 1
2019-09-03T04:31:13.4418022Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:61] 2 = 2
2019-09-03T04:31:13.4418384Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] 1u8 = 1
2019-09-03T04:31:13.4418714Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] 2u32 = 2
2019-09-03T04:31:13.4419071Z [/checkout/src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs:65] "Yeah" = "Yeah"
2019-09-03T04:31:13.4419660Z ------------------------------------------
2019-09-03T04:31:13.4419710Z 
2019-09-03T04:31:13.4419747Z 
2019-09-03T04:31:13.4419800Z 
---
2019-09-03T04:31:13.4460057Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-09-03T04:31:13.4460994Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-09-03T04:31:13.4483321Z 
2019-09-03T04:31:13.4483438Z 
2019-09-03T04:31:13.4485696Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-arm-unknown-linux-gnueabihf" "--mode" "ui" "--target" "arm-unknown-linux-gnueabihf" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--linker" "arm-linux-gnueabihf-gcc" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/arm-unknown-linux-gnueabihf/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--llvm-version" "9.0.0-rust-1.39.0-dev\n" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--remote-test-client" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-09-03T04:31:13.4486400Z 
2019-09-03T04:31:13.4486459Z 
2019-09-03T04:31:13.4494185Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --target arm-unknown-linux-gnueabihf
2019-09-03T04:31:13.4494329Z Build completed unsuccessfully in 2:24:24
2019-09-03T04:31:13.4494329Z Build completed unsuccessfully in 2:24:24
2019-09-03T04:31:13.4553509Z == clock drift check ==
2019-09-03T04:31:13.4573275Z   local time: Tue Sep  3 04:31:13 UTC 2019
2019-09-03T04:31:13.5301592Z   network time: Tue, 03 Sep 2019 04:31:13 GMT
2019-09-03T04:31:13.5301737Z == end clock drift check ==
2019-09-03T04:31:14.2526513Z ##[error]Bash exited with code '1'.
2019-09-03T04:31:14.2565956Z ##[section]Starting: Upload CPU usage statistics
2019-09-03T04:31:14.2572640Z ==============================================================================
2019-09-03T04:31:14.2572767Z Task         : Bash
2019-09-03T04:31:14.2572845Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-09-03T04:33:03Z

💔 Test failed - checks-azure

The remote-test-client outputs a message of the form "uploaded "<build_dir>/<executable_path>", waiting for result" onto stdout when executing a test, which is then captured in the process result. This needs to be removed when comparing the results of the run-pass test execution.

nathanwhit · 2019-09-03T21:16:53Z

The most recent commit should take care of the failing test, and allow for the PR to merge.

Mark-Simulacrum · 2019-09-03T21:20:35Z

@bors r+

bors · 2019-09-03T21:20:36Z

📌 Commit 12adc39 has been approved by Mark-Simulacrum

bors · 2019-09-04T11:37:44Z

⌛ Testing commit 12adc39 with merge 6c18a3d...

Allow checking of run-pass execution output in compiletest Closes #63751 Adds a `check-run-results` flag to compiletest headers, which if enabled checks the output of the execution of a run-pass test's binary against expected output.

bors · 2019-09-04T15:28:06Z

☀️ Test successful - checks-azure
Approved by: Mark-Simulacrum
Pushing 6c18a3d to master...

nathanwhit added 3 commits August 22, 2019 18:53

Add header flag to check run-pass test output

54f80a5

Extract loading and comparison of compiletest outputs to new fn

0c379df

Moved into ```fn load_compare_outputs(&self, proc_res: &ProcRes, explicit_format: bool) -> usize```

Compare run-pass outputs if check-run-results flag enabled

b9ba8f9

Addresses rust-lang#63751

rust-highfive assigned alexcrichton Aug 23, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 23, 2019

rust-highfive assigned Mark-Simulacrum and unassigned alexcrichton Aug 23, 2019

Mark-Simulacrum reviewed Aug 25, 2019

View reviewed changes

src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs Show resolved Hide resolved

src/tools/compiletest/src/runtest.rs Outdated Show resolved Hide resolved

nathanwhit force-pushed the check-run-results branch from 457bfce to 3dc3cf3 Compare August 25, 2019 23:26

nathanwhit force-pushed the check-run-results branch from 3dc3cf3 to 1c6b986 Compare August 26, 2019 00:02

Allow checking of both compile & run output for run-pass tests

0e3bc62

nathanwhit force-pushed the check-run-results branch from 1c6b986 to 29cc305 Compare August 26, 2019 01:11

nathanwhit force-pushed the check-run-results branch from 29cc305 to cdf3beb Compare August 26, 2019 03:48

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 26, 2019

RalfJung mentioned this pull request Aug 27, 2019

Migrate test suite to use run-pass stdout/stderr testing #63924

Closed

RalfJung reviewed Aug 28, 2019

View reviewed changes

src/test/ui/rfc-2361-dbg-macro/dbg-macro-expected-behavior.rs Outdated Show resolved Hide resolved

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 28, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 2, 2019

Ignore check-run-results tests for wasm32-bare

bf44f12

Ignores run-pass tests with the `check-run-results` flag enabled for the wasm32-bare ("wasm32-unknown-unknown") target, as it does not support printing to stdout/stderr.

nathanwhit force-pushed the check-run-results branch from e86d112 to bf44f12 Compare September 2, 2019 16:05

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 2, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 3, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 3, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 4, 2019

bors merged commit 12adc39 into rust-lang:master Sep 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow checking of run-pass execution output in compiletest #63825

Allow checking of run-pass execution output in compiletest #63825

nathanwhit commented Aug 23, 2019

rust-highfive commented Aug 23, 2019

nathanwhit commented Aug 23, 2019

Mark-Simulacrum left a comment

rust-highfive commented Aug 25, 2019

rust-highfive commented Aug 26, 2019

rust-highfive commented Aug 26, 2019

Mark-Simulacrum commented Aug 26, 2019

bors commented Aug 26, 2019

bors commented Aug 28, 2019

bors commented Aug 28, 2019

rust-highfive commented Aug 28, 2019

rust-highfive commented Sep 2, 2019

bors commented Sep 2, 2019

RalfJung commented Sep 2, 2019

Mark-Simulacrum commented Sep 2, 2019

nathanwhit commented Sep 2, 2019 •

edited

Mark-Simulacrum commented Sep 2, 2019

bors commented Sep 2, 2019

bors commented Sep 3, 2019

rust-highfive commented Sep 3, 2019

bors commented Sep 3, 2019

nathanwhit commented Sep 3, 2019

Mark-Simulacrum commented Sep 3, 2019

bors commented Sep 3, 2019

bors commented Sep 4, 2019

bors commented Sep 4, 2019

Allow checking of run-pass execution output in compiletest #63825

Allow checking of run-pass execution output in compiletest #63825

Conversation

nathanwhit commented Aug 23, 2019

rust-highfive commented Aug 23, 2019

nathanwhit commented Aug 23, 2019

Mark-Simulacrum left a comment

Choose a reason for hiding this comment

rust-highfive commented Aug 25, 2019

rust-highfive commented Aug 26, 2019

rust-highfive commented Aug 26, 2019

Mark-Simulacrum commented Aug 26, 2019

bors commented Aug 26, 2019

bors commented Aug 28, 2019

bors commented Aug 28, 2019

rust-highfive commented Aug 28, 2019

rust-highfive commented Sep 2, 2019

bors commented Sep 2, 2019

RalfJung commented Sep 2, 2019

Mark-Simulacrum commented Sep 2, 2019

nathanwhit commented Sep 2, 2019 • edited

Mark-Simulacrum commented Sep 2, 2019

bors commented Sep 2, 2019

bors commented Sep 3, 2019

rust-highfive commented Sep 3, 2019

bors commented Sep 3, 2019

nathanwhit commented Sep 3, 2019

Mark-Simulacrum commented Sep 3, 2019

bors commented Sep 3, 2019

bors commented Sep 4, 2019

bors commented Sep 4, 2019

nathanwhit commented Sep 2, 2019 •

edited