Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc: implement argsfiles for command line #63175

Open
wants to merge 6 commits into
base: master
from

Conversation

@jsgf
Copy link
Contributor

commented Jul 31, 2019

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line. This causes file to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \-escaped. Args files may recursively include other args files via @file2.

See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters.

This is useful for very large command lines, or when command lines are being generated into files by other tooling.

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 31, 2019

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

@petrochenkov

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2019

I clearly remember I've seen an issue about this, but I can't find it right now.

MSVC supports them as well, with the same syntax.

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Jul 31, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-31T21:44:39.1595484Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-31T21:44:39.1777303Z ##[command]git config gc.auto 0
2019-07-31T21:44:39.1851197Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-31T21:44:39.1899141Z ##[command]git config --get-all http.proxy
2019-07-31T21:44:39.2050902Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-07-31T21:45:16.9271457Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-31T21:45:16.9271488Z 
2019-07-31T21:45:16.9271720Z   git checkout -b <new-branch-name>
2019-07-31T21:45:16.9271751Z 
2019-07-31T21:45:16.9271799Z HEAD is now at a33b6907c Merge d52a989c71a8e3dc644a873f3835144bd6040349 into e3976fff44e6ce14c2f92252e6a807800b9aa7c0
2019-07-31T21:45:16.9426443Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-31T21:45:16.9429887Z ==============================================================================
2019-07-31T21:45:16.9429951Z Task         : Bash
2019-07-31T21:45:16.9430015Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-31T21:51:28.6266415Z    Compiling serde_json v1.0.40
2019-07-31T21:51:33.0119672Z    Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-07-31T21:51:41.8241199Z     Finished release [optimized] target(s) in 1m 32s
2019-07-31T21:51:41.8306234Z tidy check
2019-07-31T21:51:42.1205930Z tidy error: /checkout/src/librustc_driver/args.rs:8: line longer than 100 chars
2019-07-31T21:51:42.1206032Z tidy error: /checkout/src/librustc_driver/args.rs:9: line longer than 100 chars
2019-07-31T21:51:42.1206078Z tidy error: /checkout/src/librustc_driver/args.rs:11: line longer than 100 chars
2019-07-31T21:51:42.1206141Z tidy error: /checkout/src/librustc_driver/args.rs:12: line longer than 100 chars
2019-07-31T21:51:42.1206188Z tidy error: /checkout/src/librustc_driver/args.rs:13: line longer than 100 chars
2019-07-31T21:51:42.1206261Z tidy error: /checkout/src/librustc_driver/args.rs:120: line longer than 100 chars
2019-07-31T21:51:42.1206324Z tidy error: /checkout/src/librustc_driver/args.rs:121: line longer than 100 chars
2019-07-31T21:51:43.3678344Z tidy error: `/checkout/src/librustc_driver/args.rs:166` contains `#[test]`; unit tests and benchmarks must be placed into separate files or directories named `tests.rs`, `benches.rs`, `tests` or `benches`
2019-07-31T21:51:43.6953609Z some tidy checks failed
2019-07-31T21:51:43.6956172Z 
2019-07-31T21:51:43.6956172Z 
2019-07-31T21:51:43.6962388Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-07-31T21:51:43.6962597Z 
2019-07-31T21:51:43.6962625Z 
2019-07-31T21:51:43.6973516Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-07-31T21:51:43.6978358Z Build completed unsuccessfully in 0:01:35
2019-07-31T21:51:43.6978358Z Build completed unsuccessfully in 0:01:35
2019-07-31T21:51:45.0073125Z ##[error]Bash exited with code '1'.
2019-07-31T21:51:45.0109403Z ##[section]Starting: Checkout
2019-07-31T21:51:45.0111254Z ==============================================================================
2019-07-31T21:51:45.0111343Z Task         : Get sources
2019-07-31T21:51:45.0111389Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@jsgf jsgf force-pushed the jsgf:argsfile branch 2 times, most recently from cf1bc47 to fbcaacf Jul 31, 2019

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

cc @alexcrichton since I believe you added our support for calling linkers with this and are probably generally interested due to Cargo concerns

@Centril Centril added the relnotes label Jul 31, 2019

Some(FileArg::Arg(arg)) => return Some(arg),
Some(FileArg::Path(path)) => {
let file =
if self.stack.len() >= MAX_RECURSION { None } else { fs::read(&path).ok() };

This comment has been minimized.

Copy link
@Mark-Simulacrum

Mark-Simulacrum Jul 31, 2019

Member

It feels a bit error-prone to simply ignore @files that are "too deep". Maybe we can at least eprintln or something here? There's probably no good way to give a good error message here...

One possible "solution" is to just panic and bump MAX_RECURSION to some very large number like 1000 which would be relatively unreasonable.

This comment has been minimized.

Copy link
@jsgf

jsgf Jul 31, 2019

Author Contributor

The documented error handling is pretty awkward - it also just includes the literal @file string as an arg if it can't open/read the file. The recursion handling is at least consistent with that low standard.

I'm happy to scatter some eprintlns (if that's the best that's available here) around to make noise about things going wrong.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

Though this matches the ld spec, it is worth noting that this is the first place in rustc/cargo/etc where we're splitting by whitespace and implementing quoting rules around that. I haven't reviewed the implementation here closely but maybe we could avoid doing that somehow?

@petrochenkov

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2019

@cramertj

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

Hey, I was just wanting this! Wonderful :)

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2019

@Mark-Simulacrum I was following this as a convention rather than novelty for the sake of it. I chose gnu-ld as a model because it was the first even slightly detailed description of the file syntax which made sense (tho there's still a fair amount of ambiguity).

I could go with something simpler like "each arg is on its own line", though that raises questions like:

  1. What if an arg contains a newline? Doesn't seem completely unreasonable.
  2. How to define newlines on Windows? Insist on full \r\n, or also handle Unix \n line breaks? What does \r on its own mean?

The this implementation:

  1. consumes leading and trailing whitespace in an unsurprising way
  2. allows for empty args by quoting ""
  3. handles space, tab, \r and \n uniformly as whitespace which eliminates line-ending ambiguity
  4. Handles '...' and "..." in a shell-like way, as well as \ quoting.

This has some amount of superfluous complexity, but I think that's offset by making it compatible with typical shell quoting rules (so, for example, Python's shlex module can be used to quote args in this form - with some care).

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

I completely agree that this is a viable and maybe best implementation; just wanted to raise the concern that we've elsewhere always tried to shy away from shlex or the like.

One solution might be to use a null byte as a separator, since iirc they're not valid utf-8 (and rustc generally doesn't work with anything requiring them, anyway). Since these files are presumably mostly programmatically generated that doesn't seem too bad.

@jsgf jsgf force-pushed the jsgf:argsfile branch from fbcaacf to 9291a1c Jul 31, 2019

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2019

@Mark-Simulacrum \0-separated is an option, but its a bit painful to generate from a shell script and would make the files be treated as binary and unmergable by a source control system (eg, tests in rustc itself).

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

BTW, where does $DIR expansion happen? It doesn't seem to be happening for @$DIR/.... Does the $ need to be the first non-whitespace character?

@jsgf jsgf force-pushed the jsgf:argsfile branch from 9291a1c to 94f3d58 Aug 1, 2019

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

$DIR among other expansions happens in normalize_output.

I think my personal preference is \0 as the separator over implementing shlex-alikes but I don't care too much.

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented Aug 1, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-01T00:04:18.9735769Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-01T00:04:18.9957847Z ##[command]git config gc.auto 0
2019-08-01T00:04:19.0039949Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-01T00:04:19.0103492Z ##[command]git config --get-all http.proxy
2019-08-01T00:04:19.0249955Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-01T00:04:55.0434035Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-01T00:04:55.0434868Z 
2019-08-01T00:04:55.0435567Z   git checkout -b <new-branch-name>
2019-08-01T00:04:55.0435723Z 
2019-08-01T00:04:55.0435859Z HEAD is now at 972f666ee Merge 94f3d582447a100ad6d263f783711ed5eb6fd20d into 8a58268b5ad9c4a240be349a633069d48991eb0c
2019-08-01T00:04:55.0594681Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-01T00:04:55.0597736Z ==============================================================================
2019-08-01T00:04:55.0597803Z Task         : Bash
2019-08-01T00:04:55.0597841Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-01T01:05:24.3546057Z .................................................................................................... 1400/8820
2019-08-01T01:05:30.3618493Z .................................................................................................... 1500/8820
2019-08-01T01:05:43.0373725Z ..................................................................i...............i................. 1600/8820
2019-08-01T01:05:50.6920553Z .................................................................................................... 1700/8820
2019-08-01T01:06:05.6569129Z ....................................................iiiii........................................... 1800/8820
2019-08-01T01:06:17.1320088Z .................................................................................................... 2000/8820
2019-08-01T01:06:19.7035573Z .................................................................................................... 2100/8820
2019-08-01T01:06:23.3414523Z .................................................................................................... 2200/8820
2019-08-01T01:06:30.0341411Z .................................................................................................... 2300/8820
---
2019-08-01T01:10:28.9691799Z .................................................................................................... 5300/8820
2019-08-01T01:10:36.8113532Z ...............i.................................................................................... 5400/8820
2019-08-01T01:10:42.4991372Z .................................................................................................... 5500/8820
2019-08-01T01:10:55.0037881Z .................................................................................................... 5600/8820
2019-08-01T01:11:08.6460303Z .........ii...i..ii...........i..................................................................... 5700/8820
2019-08-01T01:11:26.3030927Z .................................................................................................... 5900/8820
2019-08-01T01:11:31.2592605Z .................................................................................................... 6000/8820
2019-08-01T01:11:31.2592605Z .................................................................................................... 6000/8820
2019-08-01T01:11:45.2277197Z .........i..ii...................................................................................... 6100/8820
2019-08-01T01:12:04.5741858Z ....................................................i............................................... 6300/8820
2019-08-01T01:12:06.8047406Z .................................................................................................... 6400/8820
2019-08-01T01:12:09.2735322Z ......................i............................................................................. 6500/8820
2019-08-01T01:12:13.9382428Z .................................................................................................... 6600/8820
---
2019-08-01T01:16:12.9798247Z failures:
2019-08-01T01:16:12.9830862Z 
2019-08-01T01:16:12.9831365Z ---- [ui] ui/commandline-argfile.rs stdout ----
2019-08-01T01:16:12.9831428Z 
2019-08-01T01:16:12.9831809Z error: test compilation failed although it shouldn't!
2019-08-01T01:16:12.9831858Z status: exit code: 1
2019-08-01T01:16:12.9832808Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/commandline-argfile.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--cfg" "cmdline_set" "@$DIR/commandline-argfile.args" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile/auxiliary" "-A" "unused"
2019-08-01T01:16:12.9833692Z ------------------------------------------
2019-08-01T01:16:12.9833728Z 
2019-08-01T01:16:12.9833941Z ------------------------------------------
2019-08-01T01:16:12.9833987Z stderr:
2019-08-01T01:16:12.9833987Z stderr:
2019-08-01T01:16:12.9834215Z ------------------------------------------
2019-08-01T01:16:12.9834480Z Warning: Failed to open $DIR/commandline-argfile.args: No such file or directory (os error 2)
2019-08-01T01:16:12.9835036Z error: multiple input filenames provided (first two filenames are `/checkout/src/test/ui/commandline-argfile.rs` and `@$DIR/commandline-argfile.args`)
2019-08-01T01:16:12.9835136Z 
2019-08-01T01:16:12.9835351Z ------------------------------------------
2019-08-01T01:16:12.9835382Z 
2019-08-01T01:16:12.9835424Z 
---
2019-08-01T01:16:12.9871295Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:533:22
2019-08-01T01:16:12.9871404Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-01T01:16:12.9885121Z 
2019-08-01T01:16:12.9885246Z 
2019-08-01T01:16:12.9890592Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-01T01:16:12.9890895Z 
2019-08-01T01:16:12.9890941Z 
2019-08-01T01:16:12.9904256Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-01T01:16:12.9904604Z Build completed unsuccessfully in 1:05:02
2019-08-01T01:16:12.9904604Z Build completed unsuccessfully in 1:05:02
2019-08-01T01:16:13.7548722Z ##[error]Bash exited with code '1'.
2019-08-01T01:16:13.7588057Z ##[section]Starting: Checkout
2019-08-01T01:16:13.7589772Z ==============================================================================
2019-08-01T01:16:13.7589823Z Task         : Get sources
2019-08-01T01:16:13.7589869Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@jsgf jsgf force-pushed the jsgf:argsfile branch from 94f3d58 to c46e0a4 Aug 1, 2019

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

@Mark-Simulacrum Ah thanks, I had thought that it could be used in the .rs file as an expansion. Test fixed (I hope).

@rust-highfive

This comment was marked as resolved.

Copy link
Collaborator

commented Aug 1, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-01T01:21:31.6088024Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-01T01:21:31.6289533Z ##[command]git config gc.auto 0
2019-08-01T01:21:31.6363465Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-01T01:21:31.6421099Z ##[command]git config --get-all http.proxy
2019-08-01T01:21:32.1828638Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-01T01:22:06.7915032Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-01T01:22:06.7915066Z 
2019-08-01T01:22:06.7915292Z   git checkout -b <new-branch-name>
2019-08-01T01:22:06.7915344Z 
2019-08-01T01:22:06.7915399Z HEAD is now at bcc209efb Merge c46e0a425d5c57e242c77ca8bfc43eaf0fe5843f into 8a58268b5ad9c4a240be349a633069d48991eb0c
2019-08-01T01:22:06.8092778Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-01T01:22:06.8095924Z ==============================================================================
2019-08-01T01:22:06.8096007Z Task         : Bash
2019-08-01T01:22:06.8096059Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-01T02:24:36.3649874Z .................................................................................................... 1400/8820
2019-08-01T02:24:42.6015099Z .................................................................................................... 1500/8820
2019-08-01T02:24:55.6139378Z ..................................................................i...............i................. 1600/8820
2019-08-01T02:25:03.3192117Z .................................................................................................... 1700/8820
2019-08-01T02:25:19.0101227Z ....................................................iiiii........................................... 1800/8820
2019-08-01T02:25:30.5298418Z .................................................................................................... 2000/8820
2019-08-01T02:25:33.1730548Z .................................................................................................... 2100/8820
2019-08-01T02:25:36.8757734Z .................................................................................................... 2200/8820
2019-08-01T02:25:43.5720215Z .................................................................................................... 2300/8820
---
2019-08-01T02:29:44.7322948Z .................................................................................................... 5300/8820
2019-08-01T02:29:52.5081301Z ...............i.................................................................................... 5400/8820
2019-08-01T02:29:58.2691244Z .................................................................................................... 5500/8820
2019-08-01T02:30:10.8739413Z .................................................................................................... 5600/8820
2019-08-01T02:30:24.5827135Z .........ii...i..ii...........i..................................................................... 5700/8820
2019-08-01T02:30:42.6148486Z .................................................................................................... 5900/8820
2019-08-01T02:30:47.6624464Z .................................................................................................... 6000/8820
2019-08-01T02:30:47.6624464Z .................................................................................................... 6000/8820
2019-08-01T02:31:01.9743591Z .........i..ii...................................................................................... 6100/8820
2019-08-01T02:31:21.6219448Z ....................................................i............................................... 6300/8820
2019-08-01T02:31:23.8200412Z .................................................................................................... 6400/8820
2019-08-01T02:31:26.3388541Z ......................i............................................................................. 6500/8820
2019-08-01T02:31:31.0030694Z .................................................................................................... 6600/8820
---
2019-08-01T02:35:32.7029176Z failures:
2019-08-01T02:35:32.7068847Z 
2019-08-01T02:35:32.7069352Z ---- [ui] ui/commandline-argfile.rs stdout ----
2019-08-01T02:35:32.7069421Z 
2019-08-01T02:35:32.7069669Z error: test compilation failed although it shouldn't!
2019-08-01T02:35:32.7069722Z status: exit code: 1
2019-08-01T02:35:32.7070532Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/commandline-argfile.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--cfg" "cmdline_set" "@src/test/ui/commandline-argfile.args" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile/auxiliary" "-A" "unused"
2019-08-01T02:35:32.7071118Z ------------------------------------------
2019-08-01T02:35:32.7071154Z 
2019-08-01T02:35:32.7071369Z ------------------------------------------
2019-08-01T02:35:32.7071433Z stderr:
2019-08-01T02:35:32.7071433Z stderr:
2019-08-01T02:35:32.7071645Z ------------------------------------------
2019-08-01T02:35:32.7071937Z Warning: Failed to open src/test/ui/commandline-argfile.args: No such file or directory (os error 2) (cwd: /checkout/obj)
2019-08-01T02:35:32.7072431Z error: multiple input filenames provided (first two filenames are `/checkout/src/test/ui/commandline-argfile.rs` and `@src/test/ui/commandline-argfile.args`)
2019-08-01T02:35:32.7072519Z 
2019-08-01T02:35:32.7073316Z ------------------------------------------
2019-08-01T02:35:32.7073382Z 
2019-08-01T02:35:32.7073407Z 
---
2019-08-01T02:35:32.7123171Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:533:22
2019-08-01T02:35:32.7123262Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-01T02:35:32.7139680Z 
2019-08-01T02:35:32.7140384Z 
2019-08-01T02:35:32.7148538Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-01T02:35:32.7149674Z 
2019-08-01T02:35:32.7149736Z 
2019-08-01T02:35:32.7156683Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-01T02:35:32.7157076Z Build completed unsuccessfully in 1:06:53
2019-08-01T02:35:32.7157076Z Build completed unsuccessfully in 1:06:53
2019-08-01T02:35:33.5089571Z ##[error]Bash exited with code '1'.
2019-08-01T02:35:33.5153681Z ##[section]Starting: Checkout
2019-08-01T02:35:33.5156115Z ==============================================================================
2019-08-01T02:35:33.5156203Z Task         : Get sources
2019-08-01T02:35:33.5156251Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@jsgf jsgf force-pushed the jsgf:argsfile branch 2 times, most recently from 746623c to f3620d5 Aug 1, 2019

bors added a commit that referenced this pull request Aug 14, 2019

Auto merge of #63557 - Centril:rollup-r1c65e7, r=Centril
Rollup of 10 pull requests

Successful merges:

 - #62984 (Add lint for excess trailing semicolons)
 - #63075 (Miri: Check that a ptr is aligned and inbounds already when evaluating `*`)
 - #63175 (rustc: implement argsfiles for command line)
 - #63490 (libsyntax: cleanup and refactor `pat.rs`)
 - #63495 ( Remove redundant `ty` fields from `mir::Constant` and `hair::pattern::PatternRange`.)
 - #63509 (Point at the right enclosing scope when using `await` in non-async fn)
 - #63528 (syntax: Remove `DummyResult::expr_only`)
 - #63534 (Bump to 1.39)
 - #63537 (expand: Unimplement `MutVisitor` on `MacroExpander`)
 - #63542 (Add NodeId for Arm, Field and FieldPat)

Failed merges:

r? @ghost
@Centril

This comment has been minimized.

Copy link
Member

commented Aug 14, 2019

Failed in #63557 (comment), @bors r-

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 14, 2019

Issue #63576

It looks like the tests have some platform variations. Is there a way to wildcard/patternmatch/normalize .stderr files to ignore the variations? Otherwise I can make the tests platform specific.

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

I believe there are normalize-stderr-test directives in tests to help doctor the output

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

@bors: r+

@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2019

📌 Commit 6589dce has been approved by alexcrichton

@Centril

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

@bors p=1

@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 16, 2019

⌛️ Testing commit 6589dce with merge 010299f...

bors added a commit that referenced this pull request Aug 16, 2019

Auto merge of #63175 - jsgf:argsfile, r=alexcrichton
rustc: implement argsfiles for command line

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line.  This causes `file` to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \\-escaped. Args files may recursively include other args files via `@file2`.

See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters.

This is useful for very large command lines, or when command lines are being generated into files by other tooling.

Centril added a commit to Centril/rust that referenced this pull request Aug 16, 2019

Rollup merge of rust-lang#63175 - jsgf:argsfile, r=alexcrichton
rustc: implement argsfiles for command line

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line.  This causes `file` to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \\-escaped. Args files may recursively include other args files via `@file2`.

See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters.

This is useful for very large command lines, or when command lines are being generated into files by other tooling.
@Centril

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

@bors retry rolled up.

bors added a commit that referenced this pull request Aug 16, 2019

Auto merge of #63644 - Centril:rollup-kapx76n, r=Centril
Rollup of 7 pull requests

Successful merges:

 - #63149 (resolve: Populate external modules in more automatic and lazy way)
 - #63175 (rustc: implement argsfiles for command line)
 - #63545 (Feature gate 'yield $expr?' pre-expansion)
 - #63548 (Update rustc-demangle to 0.1.16.)
 - #63558 (Remap paths for proc-macro crates.)
 - #63641 (add git keyword to submodule comments in config.example.toml)
 - #63642 (Rename overflowing_{add,sub,mul} intrinsics to wrapping_{add,sub,mul}.)

Failed merges:

r? @ghost
@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2019

⌛️ Testing commit 6589dce with merge 5addb4b...

bors added a commit that referenced this pull request Aug 17, 2019

Auto merge of #63175 - jsgf:argsfile, r=alexcrichton
rustc: implement argsfiles for command line

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line.  This causes `file` to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \\-escaped. Args files may recursively include other args files via `@file2`.

See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters.

This is useful for very large command lines, or when command lines are being generated into files by other tooling.
@Centril

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

Failed in #63644 (comment), @bors r- rollup=never

@Centril

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

@bors retry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.