Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix excessive initialization and reads beyond EOF in io::copy(_, Vec<u8>) specialization #117576

Merged
merged 2 commits into from Nov 6, 2023
Merged

Fix excessive initialization and reads beyond EOF in io::copy(_, Vec<u8>) specialization #117576

merged 2 commits into from Nov 6, 2023

Conversation

the8472
Copy link
Member

@the8472 the8472 commented Nov 4, 2023

@rustbot
Copy link
Collaborator

rustbot commented Nov 4, 2023

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 4, 2023
@the8472 the8472 mentioned this pull request Nov 4, 2023
Merged
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@the8472
avoid excessive initialization when copying to a Vec
8d8f06b 
It now keeps track of initialized bytes to avoid reinitialization.
It also keeps track of read sizes to avoid initializing more bytes
than the reader needs. This is important when passing a huge vector to a
Read that only has a few bytes to offer and doesn't implement read_buf().
@the8472
detect EOF earlier
78aa5e5 
The initial probe-for-empty-source by stack_buffer_copy only detected EOF
if the source was empty but not when it was merely small which lead to
additional calls to read() after Ok(0) had already been returned
in the stack copy routine
@Mark-Simulacrum
Copy link
Member

@bors r+

I wonder if the detection here should be publicly available on BorrowedBuf or similar - seems like it might be useful elsewhere too. But shouldn't block this.

@bors
Copy link
Contributor

bors commented Nov 4, 2023

📌 Commit 78aa5e5 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 4, 2023
@bors
Copy link
Contributor

bors commented Nov 6, 2023

⌛ Testing commit 78aa5e5 with merge 7a892ab...

@bors
Copy link
Contributor

bors commented Nov 6, 2023

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 7a892ab to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Nov 6, 2023
@bors bors merged commit 7a892ab into rust-lang:master Nov 6, 2023
12 checks passed
@rustbot rustbot added this to the 1.75.0 milestone Nov 6, 2023
@github-actions github-actions bot mentioned this pull request Nov 6, 2023
Merged
bors-ferrocene bot added a commit to ferrocene/ferrocene that referenced this pull request Nov 6, 2023
@bors-ferrocene @BenWiederhake
@SabrinaJewson @J-ZhengLi @koka831 @bjorn3 @joshlf @lengyijun @Zalathar @oli-obk @flip1995 @y21 @bors @bvanjoi
Merge #84
a01e81f 
84: Automated pull from upstream `master` r=Dajamante a=github-actions[bot]


This PR pulls the following changes from the upstream repository:

* rust-lang/rust#117585
* rust-lang/rust#117576
* rust-lang/rust#96979
* rust-lang/rust#117191
* rust-lang/rust#117179
* rust-lang/rust#117574
* rust-lang/rust#117537
* rust-lang/rust#117608
  * rust-lang/rust#117596
  * rust-lang/rust#117588
  * rust-lang/rust#117524
  * rust-lang/rust#116017
* rust-lang/rust#117504
* rust-lang/rust#117469
* rust-lang/rust#116218
* rust-lang/rust#117589
* rust-lang/rust#117581
* rust-lang/rust#117503
* rust-lang/rust#117590
  * rust-lang/rust#117583
  * rust-lang/rust#117570
  * rust-lang/rust#117562
  * rust-lang/rust#117534
  * rust-lang/rust#116894
  * rust-lang/rust#110340
* rust-lang/rust#113343
* rust-lang/rust#117579
* rust-lang/rust#117094
* rust-lang/rust#117566
* rust-lang/rust#117564
  * rust-lang/rust#117554
  * rust-lang/rust#117550
  * rust-lang/rust#117343
* rust-lang/rust#115274
* rust-lang/rust#117540
* rust-lang/rust#116412
* rust-lang/rust#115333
* rust-lang/rust#117507
* rust-lang/rust#117538
  * rust-lang/rust#117533
  * rust-lang/rust#117523
  * rust-lang/rust#117520
  * rust-lang/rust#117505
  * rust-lang/rust#117434
* rust-lang/rust#117535
* rust-lang/rust#117510
* rust-lang/rust#116439
* rust-lang/rust#117508



Co-authored-by: Ben Wiederhake <BenWiederhake.GitHub@gmx.de>
Co-authored-by: SabrinaJewson <sejewson@gmail.com>
Co-authored-by: J-ZhengLi <lizheng135@huawei.com>
Co-authored-by: koka <koka.code@gmail.com>
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
Co-authored-by: Joshua Liebow-Feeser <joshlf@users.noreply.github.com>
Co-authored-by: lengyijun <sjtu5140809011@gmail.com>
Co-authored-by: Zalathar <Zalathar@users.noreply.github.com>
Co-authored-by: Oli Scherer <git-spam-no-reply9815368754983@oli-obk.de>
Co-authored-by: Philipp Krones <hello@philkrones.com>
Co-authored-by: y21 <30553356+y21@users.noreply.github.com>
Co-authored-by: bors <bors@rust-lang.org>
Co-authored-by: bohan <bohan-zhang@foxmail.com>
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (7a892ab): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 - - 0
Regressions ❌
(secondary)		 - - 0
Improvements ✅
(primary)		 -1.0% [-1.0%, -1.0%] 1
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) -1.0% [-1.0%, -1.0%] 1

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 1.0% [0.7%, 1.4%] 2
Regressions ❌
(secondary)		 2.5% [2.5%, 2.5%] 1
Improvements ✅
(primary)		 -0.6% [-0.8%, -0.4%] 3
Improvements ✅
(secondary)		 - - 0
All ❌✅ (primary) 0.1% [-0.8%, 1.4%] 5

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 0.6% [0.5%, 0.7%] 3
Regressions ❌
(secondary)		 0.4% [0.4%, 0.4%] 1
Improvements ✅
(primary)		 -0.5% [-0.5%, -0.5%] 1
Improvements ✅
(secondary)		 -2.6% [-2.6%, -2.6%] 1
All ❌✅ (primary) 0.3% [-0.5%, 0.7%] 4

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 636.185s -> 637.484s (0.20%)
Artifact size: 304.50 MiB -> 304.49 MiB (-0.00%)

@bonsairobo bonsairobo mentioned this pull request Nov 8, 2023
Closed
@a1phyr
Copy link
Contributor

a1phyr commented Nov 9, 2023

Side question: is there a reason why this is not implemented as reader.read_to_end(self) ?

@the8472
Copy link
Member Author

the8472 commented Nov 9, 2023

Good point, they're quite similar though they contain some different optimizations. It probably make sense to unify them.

@the8472 the8472 added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Nov 9, 2023
@the8472
Copy link
Member Author

the8472 commented Nov 9, 2023

beta-nominated on behalf of this comment

@Mark-Simulacrum Mark-Simulacrum mentioned this pull request Nov 13, 2023
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 13, 2023
@bors
Auto merge of rust-lang#117843 - Mark-Simulacrum:stable-next, r=Mark-…
79e9716 
…Simulacrum

[stable] Prepare 1.74.0 release

https://forge.rust-lang.org/release/process.html#promote-branches-t-3-days-monday

Also backports:

* Disabling specialization as an alternative backport of "Fix excessive initialization and reads beyond EOF in io::copy(_, Vec<u8>) specialization rust-lang#117576"
*  coverage: Avoid creating malformed macro name spans rust-lang#117827

r? `@Mark-Simulacrum`
@Amanieu Amanieu added the beta-accepted Accepted for backporting to the compiler in the beta channel. label Nov 15, 2023
@cuviper
Copy link
Member

cuviper commented Nov 17, 2023

This was backported during the 1.74.0 stable promotion in #117843, and it already landed in what is now 1.75-beta, so there's nothing more to do here.

@rustbot label -beta-nominated

@rustbot rustbot removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Nov 17, 2023
@cuviper cuviper modified the milestones: 1.75.0, 1.74.0 Nov 17, 2023
@the8472 the8472 mentioned this pull request Nov 24, 2023
Merged
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Nov 28, 2023
@matthiaskrgr
Rollup merge of rust-lang#118222 - the8472:copy-use-vec-write, r=m-ou-se
97ef5a3 
unify read_to_end and io::copy impls for reading into a Vec

This ports over the initial probe (to avoid allocation) and the dynamic read sizing from the io::copy specialization to the `default_read_to_end` implementation which already had its own optimizations for different cases.

I think it should be a best-of-both now.

suggested by `@a1phyr` in rust-lang#117576 (comment)
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Nov 29, 2023
@rust-timer
Unrolled build for rust-lang#118222
98f60b3 
Rollup merge of rust-lang#118222 - the8472:copy-use-vec-write, r=m-ou-se

unify read_to_end and io::copy impls for reading into a Vec

This ports over the initial probe (to avoid allocation) and the dynamic read sizing from the io::copy specialization to the `default_read_to_end` implementation which already had its own optimizations for different cases.

I think it should be a best-of-both now.

suggested by `@a1phyr` in rust-lang#117576 (comment)
bors pushed a commit to rust-lang/miri that referenced this pull request Nov 30, 2023
@matthiaskrgr
Rollup merge of #118222 - the8472:copy-use-vec-write, r=m-ou-se
8ea3070 
unify read_to_end and io::copy impls for reading into a Vec

This ports over the initial probe (to avoid allocation) and the dynamic read sizing from the io::copy specialization to the `default_read_to_end` implementation which already had its own optimizations for different cases.

I think it should be a best-of-both now.

suggested by `@a1phyr` in rust-lang/rust#117576 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Mark-Simulacrum Mark-Simulacrum

Labels
beta-accepted Accepted for backporting to the compiler in the beta channel. merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.74.0
Development

Successfully merging this pull request may close these issues.

Exponential Performance Regression in Rust 1.73 Read API from BorrowedCursor::ensure_init
9 participants
@the8472 @rustbot @rust-log-analyzer @Mark-Simulacrum @bors @rust-timer @a1phyr @cuviper @Amanieu