Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BufRead::read_while #70772

Closed
wants to merge 1 commit into from

Conversation

yoshuawuyts
Copy link
Member

@yoshuawuyts yoshuawuyts commented Apr 4, 2020

This patch implements BufRead::read_while, a counterpart to BufRead::read_until that takes a predicate instead of a single byte. This allows delimiting byte streams using more refined patterns, which in particularly works well with the newly stabilized matches macro.

Usage

use std::io::{self, BufRead};

let mut cursor = io::Cursor::new(b"lorem-ipsum");
let mut buf = vec![];

cursor.read_while(&mut buf, |b| b != b'-')?;
assert_eq!(buf, b"lorem");

Motivation

The Read and BufRead traits provide far fewer adapters than Iterator does. The idea is that in most cases it should be possible to convert a Read into a BufRead, and subsequently call lines or split to convert it into an Iterator.

However both BufRead::split and BufRead::lines consume self which is problematic for streams that want to parse only a section of a stream but leave the rest of the stream intact. In such cases it can be incredibly convenient to be able to directly operate on the stream (a common example are "headers" sections). And adapters such as BufRead::read_while make such operations faster to author and less prone to errors.

Differences with BufRead::read_until

BufRead::read_until provides an inclusive read, BufRead::read_while provides a non-inclusive read. When the predicate no longer returns true, the operation finishes and the buffer becomes available with all bytes read up until that point.

References

@rust-highfive
Copy link
Collaborator

rust-highfive commented Apr 4, 2020

r? @sfackler

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review label Apr 4, 2020
@yoshuawuyts yoshuawuyts force-pushed the bufread-read_while branch 4 times, most recently from f8d4fdb to d5f1a70 Compare Apr 4, 2020
@yoshuawuyts
Copy link
Member Author

yoshuawuyts commented Apr 4, 2020

I am confused why CI is failing. Perhaps someone with more knowledge on Rust's CI could point me in the right direction?

@Xanewok
Copy link
Member

Xanewok commented Apr 4, 2020

#69898 will hopefully unblock the CI, right now mingw-check fails on outdated toolstate script that doesn't take renamed rustc-guide into account properly

@rust-highfive
Copy link
Collaborator

rust-highfive commented Apr 4, 2020

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-04-04T18:26:07.7611716Z ========================== Starting Command Output ===========================
2020-04-04T18:26:07.7614751Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/151bfbbf-9a32-4e82-93e8-719f5001fae8.sh
2020-04-04T18:26:07.7614985Z 
2020-04-04T18:26:07.7619382Z ##[section]Finishing: Disable git automatic line ending conversion
2020-04-04T18:26:07.7637053Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-04T18:26:07.7640824Z Task         : Get sources
2020-04-04T18:26:07.7641117Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-04T18:26:07.7641355Z Version      : 1.0.0
2020-04-04T18:26:07.7641515Z Author       : Microsoft
---
2020-04-04T18:26:08.7545008Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-04-04T18:26:08.7554948Z ##[command]git config gc.auto 0
2020-04-04T18:26:08.7561724Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-04-04T18:26:08.7568351Z ##[command]git config --get-all http.proxy
2020-04-04T18:26:08.7574774Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/70772/merge:refs/remotes/pull/70772/merge
---
2020-04-04T18:28:28.1049533Z  ---> 3fc1b512c57b
2020-04-04T18:28:28.1049798Z Step 6/7 : ENV RUN_CHECK_WITH_PARALLEL_QUERIES 1
2020-04-04T18:28:28.1050235Z  ---> Using cache
2020-04-04T18:28:28.1050602Z  ---> 5ee4295733f4
2020-04-04T18:28:28.1054934Z Step 7/7 : ENV SCRIPT python2.7 ../x.py test src/tools/expand-yaml-anchors &&            python2.7 ../x.py check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu &&            python2.7 ../x.py build --stage 0 src/tools/build-manifest &&            python2.7 ../x.py test --stage 0 src/tools/compiletest &&            python2.7 ../x.py test src/tools/tidy &&            /scripts/validate-toolstate.sh
2020-04-04T18:28:28.1056387Z  ---> 3d07a0fa42fe
2020-04-04T18:28:28.1090797Z Successfully built 3d07a0fa42fe
2020-04-04T18:28:28.1116134Z Successfully tagged rust-ci:latest
2020-04-04T18:28:28.2007217Z Built container sha256:3d07a0fa42feb5754fc13bb2f7010ebe13e4b8b8cdbebed0c75d8da320c8c8ad
2020-04-04T18:28:28.2007217Z Built container sha256:3d07a0fa42feb5754fc13bb2f7010ebe13e4b8b8cdbebed0c75d8da320c8c8ad
2020-04-04T18:28:28.2023200Z Looks like docker image is the same as before, not uploading
2020-04-04T18:28:34.9671849Z [CI_JOB_NAME=mingw-check]
2020-04-04T18:28:34.9938593Z [CI_JOB_NAME=mingw-check]
2020-04-04T18:28:34.9967080Z == clock drift check ==
2020-04-04T18:28:34.9975948Z   local time: Sat Apr  4 18:28:34 UTC 2020
2020-04-04T18:28:35.1594782Z   network time: Sat, 04 Apr 2020 18:28:35 GMT
2020-04-04T18:28:35.1602418Z Starting sccache server...
2020-04-04T18:28:35.2493290Z configure: processing command line
2020-04-04T18:28:35.2494266Z configure: 
2020-04-04T18:28:35.2495401Z configure: rust.parallel-compiler := True
---
2020-04-04T18:32:15.9232221Z     Checking rustc_span v0.0.0 (/checkout/src/librustc_span)
2020-04-04T18:32:20.8367462Z     Checking rustc_errors v0.0.0 (/checkout/src/librustc_errors)
2020-04-04T18:32:22.1418738Z     Checking rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-04T18:32:22.1842065Z     Checking fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-04T18:32:22.4025033Z     Checking rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-04T18:32:23.2200901Z     Checking rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-04T18:32:23.2920413Z     Checking rustc_session v0.0.0 (/checkout/src/librustc_session)
2020-04-04T18:32:24.9003501Z     Checking rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-04T18:32:25.4204724Z     Checking rustc_parse v0.0.0 (/checkout/src/librustc_parse)
---
2020-04-04T18:34:24.4649198Z configure: build.locked-deps    := True
2020-04-04T18:34:24.4649680Z configure: llvm.ccache          := sccache
2020-04-04T18:34:24.4650632Z configure: build.cargo-native-static := True
2020-04-04T18:34:24.4651815Z configure: dist.missing-tools   := True
2020-04-04T18:34:24.4652945Z configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
2020-04-04T18:34:24.4653782Z configure: writing `config.toml` in current directory
2020-04-04T18:34:24.4655147Z configure: 
2020-04-04T18:34:24.4655929Z configure: run `python /checkout/x.py --help`
2020-04-04T18:34:24.4656288Z configure: 
---
2020-04-04T18:35:53.4296837Z Hugepagesize:       2048 kB
2020-04-04T18:35:53.4297060Z DirectMap4k:      145344 kB
2020-04-04T18:35:53.4297282Z DirectMap2M:     4048896 kB
2020-04-04T18:35:53.4297523Z DirectMap1G:     5242880 kB
2020-04-04T18:35:53.4298085Z + python2.7 ../x.py test src/tools/expand-yaml-anchors
2020-04-04T18:35:54.1152910Z Ensuring the YAML anchors in the GitHub Actions config were expanded
2020-04-04T18:35:54.1152910Z Ensuring the YAML anchors in the GitHub Actions config were expanded
2020-04-04T18:35:54.1160512Z Building stage0 tool expand-yaml-anchors (x86_64-unknown-linux-gnu)
2020-04-04T18:35:54.3484992Z    Compiling unicode-xid v0.2.0
2020-04-04T18:35:54.4932279Z    Compiling syn v1.0.11
2020-04-04T18:35:55.3939043Z    Compiling linked-hash-map v0.5.2
2020-04-04T18:35:55.4021922Z    Compiling lazy_static v1.4.0
2020-04-04T18:35:55.4021922Z    Compiling lazy_static v1.4.0
2020-04-04T18:35:55.6243833Z    Compiling yaml-rust v0.4.3
2020-04-04T18:36:00.1773036Z    Compiling quote v1.0.2
2020-04-04T18:36:15.8685730Z    Compiling thiserror-impl v1.0.5
2020-04-04T18:36:21.0825124Z    Compiling thiserror v1.0.5
2020-04-04T18:36:21.1500457Z    Compiling yaml-merge-keys v0.4.0
2020-04-04T18:36:22.4575408Z    Compiling expand-yaml-anchors v0.1.0 (/checkout/src/tools/expand-yaml-anchors)
2020-04-04T18:36:25.1194550Z Build completed successfully in 0:00:32
2020-04-04T18:36:25.1206417Z + python2.7 ../x.py check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu
2020-04-04T18:36:25.3792110Z     Finished dev [unoptimized] target(s) in 0.18s
2020-04-04T18:36:26.5581578Z Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> i686-pc-windows-gnu)
---
2020-04-04T18:38:42.5641760Z     Checking rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-04T18:38:42.6605878Z     Checking fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-04T18:38:42.8993750Z     Checking rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
2020-04-04T18:38:43.0146227Z     Checking rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-04T18:38:43.5623842Z     Checking rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-04T18:38:46.0785145Z     Checking rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-04T18:38:46.6118499Z     Checking rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-04-04T18:38:48.8861553Z     Checking rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
2020-04-04T18:38:49.3793209Z     Checking rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
---
2020-04-04T18:43:13.8462319Z Build completed successfully in 0:00:44
2020-04-04T18:43:13.8466196Z + /scripts/validate-toolstate.sh
2020-04-04T18:43:13.8537273Z Cloning into 'rust-toolstate'...
2020-04-04T18:43:14.5164782Z Traceback (most recent call last):
2020-04-04T18:43:14.5166190Z   File "../../src/tools/publish_toolstate.py", line 305, in <module>
2020-04-04T18:43:14.5166717Z     cur_datetime
2020-04-04T18:43:14.5167052Z   File "../../src/tools/publish_toolstate.py", line 205, in update_latest
2020-04-04T18:43:14.5168026Z     maintainers = ' '.join('@'+name for name in MAINTAINERS[tool])
2020-04-04T18:43:14.5168557Z KeyError: u'rustc-dev-guide'
2020-04-04T18:43:14.5218106Z   local time: Sat Apr  4 18:43:14 UTC 2020
2020-04-04T18:43:14.5218106Z   local time: Sat Apr  4 18:43:14 UTC 2020
2020-04-04T18:43:14.7286553Z   network time: Sat, 04 Apr 2020 18:43:14 GMT
2020-04-04T18:43:16.0732029Z 
2020-04-04T18:43:16.0732029Z 
2020-04-04T18:43:16.0809417Z ##[error]Bash exited with code '1'.
2020-04-04T18:43:16.0825597Z ##[section]Finishing: Run build
2020-04-04T18:43:16.0900131Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-04T18:43:16.0907132Z Task         : Get sources
2020-04-04T18:43:16.0907511Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-04T18:43:16.0907988Z Version      : 1.0.0
2020-04-04T18:43:16.0908208Z Author       : Microsoft
2020-04-04T18:43:16.0908208Z Author       : Microsoft
2020-04-04T18:43:16.0908750Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-04-04T18:43:16.0909499Z ==============================================================================
2020-04-04T18:43:16.4435462Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-04-04T18:43:16.4479035Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-04T18:43:16.4577072Z Cleaning up task key
2020-04-04T18:43:16.4578460Z Start cleaning up orphan processes.
2020-04-04T18:43:16.4770036Z Terminate orphan process: pid (3930) (python)
2020-04-04T18:43:16.4944850Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

src/libstd/io/mod.rs Show resolved Hide resolved
/// assert_eq!(buf, b"-ipsum");
/// ```
#[unstable(feature = "buf_read_read_while", issue = "none")]
fn read_while<P>(&mut self, buf: &mut Vec<u8>, predicate: P) -> Result<usize>
Copy link
Member

@sfackler sfackler Apr 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth updating read_until to delegate to this method.

Copy link
Member Author

@yoshuawuyts yoshuawuyts Apr 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

read_until uses memchr to scan for the token; it seems likely this operation is vectorized in many cases, and I'm afraid moving to read_while's linear scan could inadvertently cause a slowdown.

@sfackler
Copy link
Member

sfackler commented Apr 4, 2020

I'm a bit worried that BufRead will have both read_until and read_while with different arguments but the naming doesn't make it super clear which is which.

@yoshuawuyts
Copy link
Member Author

yoshuawuyts commented Apr 4, 2020

@sfackler That's a fair concern. My reasoning for naming this method read_while is because the _while suffix is used for several other methods in std already to describe something that: "While a predicate holds true, do a thing":

I think read_while fits well into this lineup: "While a predicate holds true, read bytes into a buffer". The existing read_until method does not have any other counterparts in std, but has been around since Rust 1.0.0.

I think you're right that the naming of these methods could confuse people who aren't already familiar with Rust. However both methods seem useful and reasonably named. I think the best resolution here would be to acknowledge that both methods indeed have a close resemblance, and document how they differ. For example we could add something like following to read_while:

read_while will read bytes into the buffer until the predicate returns false or the underlying byte stream reaches EOF. When the function returns the buffer will contain all bytes inspected excluding the last byte seen (the byte that caused the predicate to return false).

If you're looking to read bytes including the last byte seen consider using read_until.

@yoshuawuyts yoshuawuyts force-pushed the bufread-read_while branch 2 times, most recently from 1694935 to 1239ba9 Compare Apr 5, 2020
src/libstd/io/mod.rs Outdated Show resolved Hide resolved
@rust-highfive
Copy link
Collaborator

rust-highfive commented Apr 5, 2020

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-04-05T00:10:11.4164352Z ========================== Starting Command Output ===========================
2020-04-05T00:10:11.4168543Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/9add5743-b602-4f53-9c73-08cca991e1a1.sh
2020-04-05T00:10:11.4168984Z 
2020-04-05T00:10:11.4173762Z ##[section]Finishing: Disable git automatic line ending conversion
2020-04-05T00:10:11.4192789Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T00:10:11.4196216Z Task         : Get sources
2020-04-05T00:10:11.4196514Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-05T00:10:11.4196816Z Version      : 1.0.0
2020-04-05T00:10:11.4197223Z Author       : Microsoft
---
2020-04-05T00:10:12.9182031Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-04-05T00:10:12.9194427Z ##[command]git config gc.auto 0
2020-04-05T00:10:12.9200050Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-04-05T00:10:13.9050835Z ##[command]git config --get-all http.proxy
2020-04-05T00:10:13.9064542Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/70772/merge:refs/remotes/pull/70772/merge
---
2020-04-05T00:13:52.3500013Z Looks like docker image is the same as before, not uploading
2020-04-05T00:14:01.0723626Z [CI_JOB_NAME=x86_64-gnu-llvm-7]
2020-04-05T00:14:01.1056954Z [CI_JOB_NAME=x86_64-gnu-llvm-7]
2020-04-05T00:14:01.1073478Z == clock drift check ==
2020-04-05T00:14:01.1083965Z   local time: Sun Apr  5 00:14:01 UTC 2020
2020-04-05T00:14:01.2701905Z   network time: Sun, 05 Apr 2020 00:14:01 GMT
2020-04-05T00:14:01.2725956Z Starting sccache server...
2020-04-05T00:14:01.3646920Z configure: processing command line
2020-04-05T00:14:01.3647293Z configure: 
2020-04-05T00:14:01.3652395Z configure: rust.dist-src        := False
---
2020-04-05T00:19:44.2622839Z    Compiling rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-05T00:19:45.8855500Z    Compiling fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-05T00:19:47.5771674Z    Compiling rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
2020-04-05T00:19:49.7676263Z    Compiling rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-05T00:19:58.4747621Z    Compiling rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-05T00:20:02.3298855Z    Compiling rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
2020-04-05T00:20:07.2497613Z    Compiling rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-05T00:20:11.7537547Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-04-05T00:20:21.2811953Z    Compiling rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
---
2020-04-05T00:45:09.9867748Z    Compiling rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-05T00:45:11.9682631Z    Compiling fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-05T00:45:14.2332119Z    Compiling rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
2020-04-05T00:45:16.8497770Z    Compiling rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-05T00:45:28.0615852Z    Compiling rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-05T00:45:32.0060909Z    Compiling rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
2020-04-05T00:45:37.9568627Z    Compiling rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-05T00:45:43.8889545Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-04-05T00:45:55.3141596Z    Compiling rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
---
2020-04-05T01:14:02.4276218Z .................................................................................................... 1700/9875
2020-04-05T01:14:06.4656575Z .................................................................................................... 1800/9875
2020-04-05T01:14:15.8922228Z ................................................................................................i... 1900/9875
2020-04-05T01:14:24.1514297Z .................................................................................................... 2000/9875
2020-04-05T01:14:30.9079314Z ......................................................................................iiiii......... 2100/9875
2020-04-05T01:14:53.3049037Z .................................................................................................... 2300/9875
2020-04-05T01:14:55.5405490Z .................................................................................................... 2400/9875
2020-04-05T01:14:57.8225817Z .................................................................................................... 2500/9875
2020-04-05T01:15:04.1083687Z .................................................................................................... 2600/9875
---
2020-04-05T01:18:06.9258447Z ............................................................i...............i....................... 5000/9875
2020-04-05T01:18:14.4678305Z .................................................................................................... 5100/9875
2020-04-05T01:18:22.8138830Z .................................................................................................... 5200/9875
2020-04-05T01:18:28.1214882Z .....i.............................................................................................. 5300/9875
2020-04-05T01:18:38.6807388Z ..............................................................................................ii.ii. 5400/9875
2020-04-05T01:18:43.5427025Z .......i...i........................................................................................ 5500/9875
2020-04-05T01:18:52.5803080Z .......................................i............................................................ 5700/9875
2020-04-05T01:18:52.5803080Z .......................................i............................................................ 5700/9875
2020-04-05T01:19:02.6658277Z ...........................................................ii....................................i.. 5800/9875
2020-04-05T01:19:15.7023204Z .................................................................................................... 6000/9875
2020-04-05T01:19:15.7023204Z .................................................................................................... 6000/9875
2020-04-05T01:19:25.6012583Z ...........................................................................................ii...i..i 6100/9875
2020-04-05T01:19:38.1023329Z i...........i....................................................................................... 6200/9875
2020-04-05T01:19:54.6090032Z .................................................................................................... 6400/9875
2020-04-05T01:20:00.5928457Z .................................................................................................... 6500/9875
2020-04-05T01:20:00.5928457Z .................................................................................................... 6500/9875
2020-04-05T01:20:25.8944933Z .....................i..ii.......................................................................... 6600/9875
2020-04-05T01:20:46.8974379Z .................................................................................................... 6800/9875
2020-04-05T01:20:49.0257346Z .....................i.............................................................................. 6900/9875
2020-04-05T01:20:51.1411487Z .................................................................................................... 7000/9875
2020-04-05T01:20:53.4559044Z ............................................................i....................................... 7100/9875
---
2020-04-05T01:22:39.0081399Z .................................................................................................... 7800/9875
2020-04-05T01:22:43.6947763Z .................................................................................................... 7900/9875
2020-04-05T01:22:49.7157663Z .................................................................................................... 8000/9875
2020-04-05T01:22:58.0027647Z ........................i........................................................................... 8100/9875
2020-04-05T01:23:06.4020914Z .........................................................................iiiiiiiiii.i............... 8200/9875
2020-04-05T01:23:22.9477579Z ..................i.....i........................................................................... 8400/9875
2020-04-05T01:23:27.9353155Z .................................................................................................... 8500/9875
2020-04-05T01:23:39.7484100Z .................................................................................................... 8600/9875
2020-04-05T01:23:52.4506754Z .................................................................................................... 8700/9875
---
2020-04-05T01:26:25.7872248Z Suite("src/test/codegen") not skipped for "bootstrap::test::Codegen" -- not in ["src/tools/tidy"]
2020-04-05T01:26:25.8063307Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T01:26:26.0277376Z 
2020-04-05T01:26:26.0280578Z running 185 tests
2020-04-05T01:26:28.8163708Z iiii......i............ii.i..iiii....i....i...........i............i..i..................i....i..... 100/185
2020-04-05T01:26:31.8348033Z .......i.i.i...iii..iiiiiiiiiiiiiiii.......................iii...............ii......
2020-04-05T01:26:31.8352758Z 
2020-04-05T01:26:31.8360955Z  finished in 5.837
2020-04-05T01:26:31.8362182Z Suite("src/test/codegen-units") not skipped for "bootstrap::test::CodegenUnits" -- not in ["src/tools/tidy"]
2020-04-05T01:26:31.8363348Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T01:26:33.8171313Z Suite("src/test/assembly") not skipped for "bootstrap::test::Assembly" -- not in ["src/tools/tidy"]
2020-04-05T01:26:33.8358432Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T01:26:33.9949084Z 
2020-04-05T01:26:33.9949528Z running 9 tests
2020-04-05T01:26:33.9950679Z iiiiiiiii
2020-04-05T01:26:33.9951866Z 
2020-04-05T01:26:33.9952308Z  finished in 0.158
2020-04-05T01:26:33.9953086Z Suite("src/test/incremental") not skipped for "bootstrap::test::Incremental" -- not in ["src/tools/tidy"]
2020-04-05T01:26:34.0139878Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T01:26:54.7775986Z Suite("src/test/debuginfo") not skipped for "bootstrap::test::Debuginfo" -- not in ["src/tools/tidy"]
2020-04-05T01:26:54.8000197Z Check compiletest suite=debuginfo mode=debuginfo (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T01:26:54.9989273Z 
2020-04-05T01:26:54.9989605Z running 115 tests
2020-04-05T01:27:09.5923282Z iiiii..i.....i..i...i..i.i.i..i..i..ii....i.i....ii..........iiii.........i.....i..i.......ii.i.ii.. 100/115
2020-04-05T01:27:11.2845509Z ...iiii.....ii.
2020-04-05T01:27:11.2850833Z 
2020-04-05T01:27:11.2851746Z  finished in 16.484
2020-04-05T01:27:11.2884562Z Suite("src/test/ui-fulldeps") not skipped for "bootstrap::test::UiFullDeps" -- not in ["src/tools/tidy"]
2020-04-05T01:27:11.2885399Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T01:41:03.0367687Z 
2020-04-05T01:41:03.0369158Z    Doc-tests core
2020-04-05T01:41:08.1853793Z 
2020-04-05T01:41:08.1854249Z running 2489 tests
2020-04-05T01:41:17.6590622Z ......iiiii......................................................................................... 100/2489
2020-04-05T01:41:26.9250696Z .....................................................................................ii............. 200/2489
2020-04-05T01:41:48.5288887Z ....................i............................................................................... 400/2489
2020-04-05T01:41:48.5288887Z ....................i............................................................................... 400/2489
2020-04-05T01:41:58.8296863Z ..........................................................................i..i..................iiii 500/2489
2020-04-05T01:42:15.7560596Z .................................................................................................... 700/2489
2020-04-05T01:42:24.5868977Z .................................................................................................... 800/2489
2020-04-05T01:42:33.2098095Z .................................................................................................... 900/2489
2020-04-05T01:42:41.9414087Z .................................................................................................... 1000/2489
---
2020-04-05T01:46:10.9559236Z .................................................thread '<unnamed>' panicked at 'explicit panic', src/libstd/io/stdio.rs:888:13
2020-04-05T01:46:10.9566326Z .. 300/761
2020-04-05T01:46:11.0603878Z .................................................................................................... 400/761
2020-04-05T01:46:13.1515714Z .................................................................................................... 500/761
2020-04-05T01:46:13.1793021Z ...................thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:2741:22
2020-04-05T01:46:13.1813704Z ....thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libstd/sync/mpsc/mod.rs:2766:17
2020-04-05T01:46:13.1829127Z .thread '<unnamed>.' panicked at '..called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:2778:21
2020-04-05T01:46:13.1840919Z ...thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:2645:13
2020-04-05T01:46:13.5097179Z ..........................................thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:1997:22
2020-04-05T01:46:13.5126088Z .....thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:2034:21
2020-04-05T01:46:13.5147664Z .......thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libstd/sync/mpsc/mod.rs:1916:13
2020-04-05T01:46:13.5241496Z ................ 600/761
2020-04-05T01:46:15.5458577Z ......................thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:633:13
2020-04-05T01:46:15.5467597Z ..thread '<unnamed>' panicked at 'test panic in inner thread to poison mutex', src/libstd/sync/mutex.rs:587:13
---
2020-04-05T01:46:24.6531423Z 
2020-04-05T01:46:24.6550469Z running 1019 tests
2020-04-05T01:46:43.2306800Z i................................................................................................... 100/1019
2020-04-05T01:46:53.6676062Z .................................................................................................... 200/1019
2020-04-05T01:47:01.4006433Z ..................iii......i......i...i......i...................................................... 300/1019
2020-04-05T01:47:13.1130126Z ...................................................i....i......................................ii... 500/1019
2020-04-05T01:47:20.8693017Z .................................................................................................... 600/1019
2020-04-05T01:47:25.9459833Z .................................................................................................... 700/1019
2020-04-05T01:47:25.9459833Z .................................................................................................... 700/1019
2020-04-05T01:47:33.2199489Z .............................................iiii................................................... 800/1019
2020-04-05T01:47:47.7224179Z .................................................................................................... 900/1019
2020-04-05T01:47:54.0350846Z ...................................................................iiii............................. 1000/1019
2020-04-05T01:47:55.3532516Z failures:
2020-04-05T01:47:55.3532651Z 
2020-04-05T01:47:55.3533664Z ---- io/mod.rs - io::BufRead::read_while (line 1899) stdout ----
2020-04-05T01:47:55.3534569Z error[E0658]: use of unstable library feature 'buf_read_read_while'
2020-04-05T01:47:55.3534569Z error[E0658]: use of unstable library feature 'buf_read_read_while'
2020-04-05T01:47:55.3537062Z   --> io/mod.rs:1907:8
2020-04-05T01:47:55.3537270Z    |
2020-04-05T01:47:55.3537768Z 11 | cursor.read_while(&mut buf, |b| b != b'-')
2020-04-05T01:47:55.3538229Z    |
2020-04-05T01:47:55.3538229Z    |
2020-04-05T01:47:55.3538570Z    = help: add `#![feature(buf_read_read_while)]` to the crate attributes to enable
2020-04-05T01:47:55.3539067Z error: aborting due to previous error
2020-04-05T01:47:55.3539257Z 
2020-04-05T01:47:55.3539793Z For more information about this error, try `rustc --explain E0658`.
2020-04-05T01:47:55.3540299Z Couldn't compile the test.
---
2020-04-05T01:47:55.3660582Z 
2020-04-05T01:47:55.3661299Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --exclude src/tools/tidy
2020-04-05T01:47:55.3661748Z Build completed unsuccessfully in 1:32:08
2020-04-05T01:47:55.3717768Z == clock drift check ==
2020-04-05T01:47:55.3732872Z   local time: Sun Apr  5 01:47:55 UTC 2020
2020-04-05T01:47:55.4747816Z   network time: Sun, 05 Apr 2020 01:47:55 GMT
2020-04-05T01:47:56.1365975Z 
2020-04-05T01:47:56.1365975Z 
2020-04-05T01:47:56.1454497Z ##[error]Bash exited with code '1'.
2020-04-05T01:47:56.1469332Z ##[section]Finishing: Run build
2020-04-05T01:47:56.1546250Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T01:47:56.1551874Z Task         : Get sources
2020-04-05T01:47:56.1552225Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-05T01:47:56.1552546Z Version      : 1.0.0
2020-04-05T01:47:56.1552788Z Author       : Microsoft
2020-04-05T01:47:56.1552788Z Author       : Microsoft
2020-04-05T01:47:56.1553145Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-04-05T01:47:56.1553757Z ==============================================================================
2020-04-05T01:47:56.5355887Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-04-05T01:47:56.5405577Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T01:47:56.5504586Z Cleaning up task key
2020-04-05T01:47:56.5505854Z Start cleaning up orphan processes.
2020-04-05T01:47:56.5703217Z Terminate orphan process: pid (3818) (python)
2020-04-05T01:47:56.9187360Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@rust-highfive
Copy link
Collaborator

rust-highfive commented Apr 5, 2020

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-04-05T15:12:53.5507836Z ========================== Starting Command Output ===========================
2020-04-05T15:12:53.5510473Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/79744b60-4e93-454b-8c32-2f312c1af284.sh
2020-04-05T15:12:53.5510785Z 
2020-04-05T15:12:53.5514480Z ##[section]Finishing: Disable git automatic line ending conversion
2020-04-05T15:12:53.5528584Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T15:12:53.5530914Z Task         : Get sources
2020-04-05T15:12:53.5531141Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-05T15:12:53.5531355Z Version      : 1.0.0
2020-04-05T15:12:53.5531495Z Author       : Microsoft
---
2020-04-05T15:12:54.5422801Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-04-05T15:12:54.5426608Z ##[command]git config gc.auto 0
2020-04-05T15:12:54.5429085Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-04-05T15:12:54.5431384Z ##[command]git config --get-all http.proxy
2020-04-05T15:12:54.5435587Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/70772/merge:refs/remotes/pull/70772/merge
---
2020-04-05T15:14:48.3401106Z Looks like docker image is the same as before, not uploading
2020-04-05T15:14:55.7525462Z [CI_JOB_NAME=x86_64-gnu-llvm-7]
2020-04-05T15:14:55.7746032Z [CI_JOB_NAME=x86_64-gnu-llvm-7]
2020-04-05T15:14:55.7770361Z == clock drift check ==
2020-04-05T15:14:55.7781085Z   local time: Sun Apr  5 15:14:55 UTC 2020
2020-04-05T15:14:55.9394632Z   network time: Sun, 05 Apr 2020 15:14:55 GMT
2020-04-05T15:14:55.9418298Z Starting sccache server...
2020-04-05T15:14:56.0038512Z configure: processing command line
2020-04-05T15:14:56.0038839Z configure: 
2020-04-05T15:14:56.0039786Z configure: rust.dist-src        := False
---
2020-04-05T15:18:53.3776334Z    Compiling rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-05T15:18:54.4579102Z    Compiling fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-05T15:18:55.6503977Z    Compiling rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
2020-04-05T15:18:56.1991269Z    Compiling rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-05T15:19:03.1028746Z    Compiling rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-05T15:19:04.6351159Z    Compiling rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
2020-04-05T15:19:07.5958717Z    Compiling rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-05T15:19:10.6045337Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-04-05T15:19:18.0941568Z    Compiling rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
---
2020-04-05T15:36:03.9110716Z    Compiling rustc_feature v0.0.0 (/checkout/src/librustc_feature)
2020-04-05T15:36:05.0590875Z    Compiling fmt_macros v0.0.0 (/checkout/src/libfmt_macros)
2020-04-05T15:36:06.6528226Z    Compiling rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
2020-04-05T15:36:07.4812429Z    Compiling rustc_hir v0.0.0 (/checkout/src/librustc_hir)
2020-04-05T15:36:16.3110097Z    Compiling rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
2020-04-05T15:36:17.7961108Z    Compiling rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
2020-04-05T15:36:21.8328036Z    Compiling rustc_attr v0.0.0 (/checkout/src/librustc_attr)
2020-04-05T15:36:25.9398439Z    Compiling rustc_parse v0.0.0 (/checkout/src/librustc_parse)
2020-04-05T15:36:34.9816104Z    Compiling rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
---
2020-04-05T15:55:50.8230329Z .................................................................................................... 1700/9876
2020-04-05T15:55:53.8810256Z .................................................................................................... 1800/9876
2020-04-05T15:56:00.7560148Z ................................................................................................i... 1900/9876
2020-04-05T15:56:07.0099981Z .................................................................................................... 2000/9876
2020-04-05T15:56:12.2013840Z ......................................................................................iiiii......... 2100/9876
2020-04-05T15:56:28.6879845Z .................................................................................................... 2300/9876
2020-04-05T15:56:30.2340700Z .................................................................................................... 2400/9876
2020-04-05T15:56:32.0238811Z .................................................................................................... 2500/9876
2020-04-05T15:56:37.0216612Z .................................................................................................... 2600/9876
---
2020-04-05T15:58:58.0371756Z ............................................................i...............i....................... 5000/9876
2020-04-05T15:59:03.8727321Z .................................................................................................... 5100/9876
2020-04-05T15:59:09.9277896Z .................................................................................................... 5200/9876
2020-04-05T15:59:14.0920972Z .....i.............................................................................................. 5300/9876
2020-04-05T15:59:21.9904747Z ..............................................................................................ii.ii. 5400/9876
2020-04-05T15:59:25.7617247Z .......i...i........................................................................................ 5500/9876
2020-04-05T15:59:32.6319259Z .......................................i............................................................ 5700/9876
2020-04-05T15:59:32.6319259Z .......................................i............................................................ 5700/9876
2020-04-05T15:59:40.0908816Z ...........................................................ii.....................................i. 5800/9876
2020-04-05T15:59:49.7435165Z .................................................................................................... 6000/9876
2020-04-05T15:59:49.7435165Z .................................................................................................... 6000/9876
2020-04-05T15:59:57.2623985Z ............................................................................................ii...i.. 6100/9876
2020-04-05T16:00:06.8136135Z ii...........i...................................................................................... 6200/9876
2020-04-05T16:00:17.2236167Z .................................................................................................... 6400/9876
2020-04-05T16:00:19.3948749Z .................................................................................................... 6500/9876
2020-04-05T16:00:19.3948749Z .................................................................................................... 6500/9876
2020-04-05T16:00:29.0837716Z ......................i..ii......................................................................... 6600/9876
2020-04-05T16:00:45.0385008Z .................................................................................................... 6800/9876
2020-04-05T16:00:46.7313854Z ......................i............................................................................. 6900/9876
2020-04-05T16:00:48.3141692Z .................................................................................................... 7000/9876
2020-04-05T16:00:50.0175519Z .............................................................i...................................... 7100/9876
---
2020-04-05T16:02:04.6573839Z .................................................................................................... 7800/9876
2020-04-05T16:02:08.1288607Z .................................................................................................... 7900/9876
2020-04-05T16:02:12.5883966Z .................................................................................................... 8000/9876
2020-04-05T16:02:18.9692940Z .........................i.......................................................................... 8100/9876
2020-04-05T16:02:25.6400555Z ..........................................................................iiiiiiiiii.i.............. 8200/9876
2020-04-05T16:02:38.0839230Z ..................i......i.......................................................................... 8400/9876
2020-04-05T16:02:41.8925236Z .................................................................................................... 8500/9876
2020-04-05T16:02:50.7727972Z .................................................................................................... 8600/9876
2020-04-05T16:03:00.7065079Z .................................................................................................... 8700/9876
---
2020-04-05T16:04:51.5936615Z Suite("src/test/codegen") not skipped for "bootstrap::test::Codegen" -- not in ["src/tools/tidy"]
2020-04-05T16:04:51.6101550Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T16:04:51.7748067Z 
2020-04-05T16:04:51.7749077Z running 185 tests
2020-04-05T16:04:53.8603818Z iiii......i............ii.i..iiii....i....i...........i............i..i..................i....i..... 100/185
2020-04-05T16:04:55.8357064Z .......i.i.i...iii..iiiiiiiiiiiiiiii.......................iii...............ii......
2020-04-05T16:04:55.8359892Z 
2020-04-05T16:04:55.8361659Z  finished in 4.225
2020-04-05T16:04:55.8371237Z Suite("src/test/codegen-units") not skipped for "bootstrap::test::CodegenUnits" -- not in ["src/tools/tidy"]
2020-04-05T16:04:55.8518828Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T16:04:57.4333237Z Suite("src/test/assembly") not skipped for "bootstrap::test::Assembly" -- not in ["src/tools/tidy"]
2020-04-05T16:04:57.4482395Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T16:04:57.5658169Z 
2020-04-05T16:04:57.5658554Z running 9 tests
2020-04-05T16:04:57.5659630Z iiiiiiiii
2020-04-05T16:04:57.5661002Z 
2020-04-05T16:04:57.5661289Z  finished in 0.117
2020-04-05T16:04:57.5667127Z Suite("src/test/incremental") not skipped for "bootstrap::test::Incremental" -- not in ["src/tools/tidy"]
2020-04-05T16:04:57.5821493Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T16:05:12.7341465Z Suite("src/test/debuginfo") not skipped for "bootstrap::test::Debuginfo" -- not in ["src/tools/tidy"]
2020-04-05T16:05:12.7527706Z Check compiletest suite=debuginfo mode=debuginfo (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2020-04-05T16:05:12.8927842Z 
2020-04-05T16:05:12.8928167Z running 115 tests
2020-04-05T16:05:23.6351802Z iiiii..i.....i..i...i..i.i.i..i..i..ii....i.i....ii..........iiii.........i.....i..i.......ii.i.ii.. 100/115
2020-04-05T16:05:24.8335879Z ...iiii.....ii.
2020-04-05T16:05:24.8337162Z 
2020-04-05T16:05:24.8339515Z  finished in 12.081
2020-04-05T16:05:24.8344483Z Suite("src/test/ui-fulldeps") not skipped for "bootstrap::test::UiFullDeps" -- not in ["src/tools/tidy"]
2020-04-05T16:05:24.8347437Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2020-04-05T16:14:40.2996288Z 
2020-04-05T16:14:40.3004227Z    Doc-tests core
2020-04-05T16:14:43.7617633Z 
2020-04-05T16:14:43.7618589Z running 2490 tests
2020-04-05T16:14:50.6823796Z ......iiiii......................................................................................... 100/2490
2020-04-05T16:14:57.3877788Z .....................................................................................ii............. 200/2490
2020-04-05T16:15:12.8378197Z ....................i............................................................................... 400/2490
2020-04-05T16:15:12.8378197Z ....................i............................................................................... 400/2490
2020-04-05T16:15:20.2934084Z ..........................................................................i..i..................iiii 500/2490
2020-04-05T16:15:32.4205059Z .................................................................................................... 700/2490
2020-04-05T16:15:38.7471364Z .................................................................................................... 800/2490
2020-04-05T16:15:44.9503461Z .................................................................................................... 900/2490
2020-04-05T16:15:51.2825445Z .................................................................................................... 1000/2490
---
2020-04-05T16:18:32.0346388Z 
2020-04-05T16:18:32.0346812Z running 1019 tests
2020-04-05T16:18:44.9242005Z i................................................................................................... 100/1019
2020-04-05T16:18:52.9985526Z .................................................................................................... 200/1019
2020-04-05T16:18:58.2427967Z ..................iii......i......i...i......i...................................................... 300/1019
2020-04-05T16:19:06.7305061Z ...................................................i....i......................................ii... 500/1019
2020-04-05T16:19:12.8205229Z .................................................................................................... 600/1019
2020-04-05T16:19:16.7313246Z .................................................................................................... 700/1019
2020-04-05T16:19:16.7313246Z .................................................................................................... 700/1019
2020-04-05T16:19:22.2514393Z .............................................iiii................................................... 800/1019
2020-04-05T16:19:32.9020980Z .................................................................................................... 900/1019
2020-04-05T16:19:37.7109452Z ...................................................................iiii............................. 1000/1019
2020-04-05T16:19:38.6893742Z failures:
2020-04-05T16:19:38.6894059Z 
2020-04-05T16:19:38.6894999Z ---- io/mod.rs - io::BufRead::read_while (line 1886) stdout ----
2020-04-05T16:19:38.6895286Z error: unused `std::result::Result` that must be used
---
2020-04-05T16:19:38.6991655Z 
2020-04-05T16:19:38.6998230Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test --exclude src/tools/tidy
2020-04-05T16:19:38.6998715Z Build completed unsuccessfully in 1:03:24
2020-04-05T16:19:38.7048457Z == clock drift check ==
2020-04-05T16:19:38.7064142Z   local time: Sun Apr  5 16:19:38 UTC 2020
2020-04-05T16:19:38.7713605Z   network time: Sun, 05 Apr 2020 16:19:38 GMT
2020-04-05T16:19:39.1467806Z 
2020-04-05T16:19:39.1467806Z 
2020-04-05T16:19:39.1494858Z ##[error]Bash exited with code '1'.
2020-04-05T16:19:39.1512047Z ##[section]Finishing: Run build
2020-04-05T16:19:39.1552998Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T16:19:39.1558723Z Task         : Get sources
2020-04-05T16:19:39.1559069Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-04-05T16:19:39.1559386Z Version      : 1.0.0
2020-04-05T16:19:39.1559647Z Author       : Microsoft
2020-04-05T16:19:39.1559647Z Author       : Microsoft
2020-04-05T16:19:39.1559997Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-04-05T16:19:39.1560400Z ==============================================================================
2020-04-05T16:19:39.4203237Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-04-05T16:19:39.4244173Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/70772/merge to s
2020-04-05T16:19:39.4308360Z Cleaning up task key
2020-04-05T16:19:39.4309265Z Start cleaning up orphan processes.
2020-04-05T16:19:39.4444674Z Terminate orphan process: pid (3529) (python)
2020-04-05T16:19:39.4584234Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@HeroicKatora
Copy link
Contributor

HeroicKatora commented Apr 5, 2020

I don't quite follow the motivation. As for the second paragraph I don't see the problem of taking self by value as BufRead is implemented for &mut impl BufRead so a &mut self parameter could as well be written currently with self.by_ref().lines(). The first paragraph is about adaptors but the implementation is not an adaptor. Are there any problems with returning such a proxy implementation of BufRead which consumes bytes from the underlying reader until the predicate no longer holds, which follows the Iterator analogue more closely, and which potentially specializes read_to_end with the method provided here?

@yoshuawuyts
Copy link
Member Author

yoshuawuyts commented Apr 5, 2020

As for the second paragraph I don't see the problem of taking self by value as BufRead is implemented for &mut impl BufRead.

@HeroicKatora Ah yes, fair enough. I didn't realize that this was implemented for &mut as well. Spelling it out makes it seem very obvious in hindsight (playground).


I don't quite follow the motivation.

Perhaps adding more details could be helpful here. A while ago I was implementing a parser and encoder for the MIME spec and figured we could abstract it to take any Read input. One of the sections of the spec contains the following:

  • If position is not past the end of input, then:
    • If the code point at position within input is U+003B (;), then continue.
    • Advance position by 1. (This skips past U+003D (=).)

This means: "read until you encounter either ; or =, then match on that." We found the easiest way to write this was as follows (link):

// Get the param name.
//
// ```txt
// text/html; charset=utf-8;
//            ^^^^^^^
// ```
let mut param_name = vec![];
s.read_while(&mut param_name, |b| !matches!(b, b';' | b'='))?;

Neither BufReader::lines, BufReader::split or BufReader::read_until would be a good fit here. Though I'm sure there is a way to get this done with std-only types, there wasn't an obvious, convenient solution available. read_while feels like the right solution for these kinds of operations, and feels like a useful addition to other convenience methods such as read_until, read_exact, and read_to_end.

@HeroicKatora
Copy link
Contributor

HeroicKatora commented Apr 5, 2020

It seems clear that the existing methods are insufficient, adding this convenience would be good complement. Perhaps this was not very clear but I wasn't trying to questioning the need per-se, but the need of that solution in particular. Which alternatives were considered and why was this one chosen? The adaptor solution that I was trying to hint at would look like:

let mut param_name = vec![];
s
    .by_ref()
    .take_while(|b| !matches!(b, b';' | b'='))
    .read_to_end(&mut param_name)?;

which parallels Iterator usage more closely as well. The advantage here would be that even users that can't risk an unbounded allocation during reading would benefit, as the return value from take_while would combines with existing methods of BufRead, such as take or manual loops.

@yoshuawuyts
Copy link
Member Author

yoshuawuyts commented Apr 8, 2020

Which alternatives were considered and why was this one chosen?

@HeroicKatora Oh I really like that suggestion! That's a design I hadn't considered, and I'll be the first to admit that it ticks all the same boxes as what I had, but with fewer downsides (as @sfackler mentioned: argument ordering of BufRead::read_while vs BufRead::read_until can be confusing). Perhaps there is a place for read_while as a shorthand for take_while + read_to_end, but that's something that can be considered at a later time.

I think the right next step is to close this PR and file an issue to implement Read::take_while.

@yoshuawuyts yoshuawuyts closed this Apr 8, 2020
@yoshuawuyts yoshuawuyts deleted the bufread-read_while branch Apr 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants