Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have all methods of Filter and FilterMap use internal iteration #58730

Merged
merged 4 commits into from Mar 3, 2019

Conversation

Projects
None yet
4 participants
@timvermeulen
Copy link
Contributor

timvermeulen commented Feb 25, 2019

This PR changes Filter::{next, next_back, count} and FilterMap::{next, next_back} to all use internal iteration. The next and next_back methods are changed to directly forward to try_for_each and try_rfold respectively, using Result as the Try type. I think that's okay? Alternatively, I could change their implementations to use LoopState instead if there's any benefit in doing so.

r? @scottmcm

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Feb 25, 2019

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @scottmcm (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@scottmcm
Copy link
Member

scottmcm left a comment

Thanks for picking this up, @timvermeulen!

The code itself looks right, and it's nice that it's actually shorter, I just have a small side request before merging.

Show resolved Hide resolved src/libcore/iter/adapters/mod.rs
count
fn count(self) -> usize {
let mut predicate = self.predicate;
self.iter.map(|x| predicate(&x) as usize).sum()

This comment was marked as resolved.

@scottmcm

scottmcm Feb 26, 2019

Member

Since there's a big perf comment above this, I think this should include a benchmark and some results from you showing that it's no worse for simple cases, and better in some. Maybe something like the ones in https://github.com/rust-lang/rust/blob/master/src/libcore/benches/iter.rs#L161

@timvermeulen

This comment has been minimized.

Copy link
Contributor Author

timvermeulen commented Feb 27, 2019

I added a few benchmarks, of which only bench_filter_chain_count is noticeably faster, as expected:

bench_filter_count              1,240,390 ns/iter (+/- 196,022)  ->  1,255,115 ns/iter (+/- 240,559)
bench_filter_ref_count          1,320,505 ns/iter (+/- 166,123)  ->  1,208,644 ns/iter (+/- 97,721)
bench_filter_chain_count        3,869,760 ns/iter (+/- 676,929)  ->  2,554,825 ns/iter (+/- 395,721)
bench_filter_chain_ref_count    4,198,060 ns/iter (+/- 714,083)  ->  4,207,487 ns/iter (+/- 638,199)

And here are the results of the existing Filter and FilterMap benchmarks:

bench_filter_sum                1,035,545 ns/iter (+/- 273,430)    ->    829,121 ns/iter (+/- 123,512)
bench_filter_ref_sum            1,221,215 ns/iter (+/- 331,032)    ->  1,214,944 ns/iter (+/- 351,951)
bench_filter_chain_sum          1,989,356 ns/iter (+/- 358,755)    ->  2,101,564 ns/iter (+/- 372,251)
bench_filter_chain_ref_sum      5,924,146 ns/iter (+/- 1,753,172)  ->  3,223,435 ns/iter (+/- 594,346)

bench_filter_map_sum              649,279 ns/iter (+/- 117,605)    ->    569,131 ns/iter (+/- 37,360)
bench_filter_map_ref_sum        1,275,035 ns/iter (+/- 184,781)    ->  1,138,938 ns/iter (+/- 35,414)
bench_filter_map_chain_sum      1,456,455 ns/iter (+/- 561,567)    ->  1,222,999 ns/iter (+/- 165,497)
bench_filter_map_chain_ref_sum  5,696,314 ns/iter (+/- 698,883)    ->  4,729,670 ns/iter (+/- 1,519,231)
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Feb 27, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:18abb11c:start=1551265483687160989,finish=1551265557035009599,duration=73347848610
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---

[00:04:02] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:02] tidy error: /checkout/src/libcore/benches/iter.rs: missing trailing newline
[00:04:04] some tidy checks failed
[00:04:04] 
[00:04:04] 
[00:04:04] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:04] 
[00:04:04] 
[00:04:04] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:04] Build completed unsuccessfully in 0:00:46
[00:04:04] Build completed unsuccessfully in 0:00:46
[00:04:04] Makefile:68: recipe for target 'tidy' failed
[00:04:04] make: *** [tidy] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:015f8318
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed Feb 27 11:10:11 UTC 2019
---
travis_time:end:0fd3d0ec:start=1551265812290831832,finish=1551265812295253243,duration=4421411
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:038d9d13
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1220da68
travis_time:start:1220da68
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:11cf80f8
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@scottmcm

This comment has been minimized.

Copy link
Member

scottmcm commented Feb 27, 2019

Thanks, @timvermeulen! Great to see the other benchmarks made more representative too 🙂

@bors r+ rollup

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 27, 2019

📌 Commit 88bd624 has been approved by scottmcm

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 27, 2019

💡 This pull request was already approved, no need to approve it again.

  • There's another pull request that is currently being tested, blocking this pull request: #57760
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 27, 2019

📌 Commit 88bd624 has been approved by scottmcm

Centril added a commit to Centril/rust that referenced this pull request Feb 28, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

Centril added a commit to Centril/rust that referenced this pull request Feb 28, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

Centril added a commit to Centril/rust that referenced this pull request Feb 28, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Feb 28, 2019

Auto merge of #58801 - Centril:rollup, r=Centril
Rollup of 6 pull requests

Successful merges:

 - #58605 (Use informational target machine for metadata)
 - #58629 (rust-lldb: fix crash when printing empty string)
 - #58631 (Put Local, Static and Promoted as one Base variant of Place)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)

Failed merges:

r? @ghost

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Feb 28, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Feb 28, 2019

Auto merge of #58820 - GuillaumeGomez:rollup, r=GuillaumeGomez
Rollup of 6 pull requests

Successful merges:

 - #58689 (Add unstable option to ignore should_panic tests)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58800 (Update edition-guide)

Failed merges:

r? @ghost

pietroalbini added a commit to pietroalbini/rust that referenced this pull request Mar 1, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Mar 1, 2019

Auto merge of #58831 - pietroalbini:rollup, r=pietroalbini
Rollup of 11 pull requests

Successful merges:

 - #58605 (Use informational target machine for metadata)
 - #58629 (rust-lldb: fix crash when printing empty string)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58743 (Add tests for several E-needstest issues)
 - #58750 (Make `Unique::as_ptr`, `NonNull::dangling` and `NonNull::cast` const)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58788 (Make migrate mode work at item level granularity)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))

Failed merges:

r? @ghost

kennytm added a commit to kennytm/rust that referenced this pull request Mar 2, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Mar 2, 2019

Auto merge of #58866 - kennytm:rollup, r=kennytm
Rollup of 16 pull requests

Successful merges:

 - #58077 (Add debug-info to access variables from generator state)
 - #58216 (Set secure flags when opening a named pipe on Windows)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58743 (Add tests for several E-needstest issues)
 - #58750 (Make `Unique::as_ptr`, `NonNull::dangling` and `NonNull::cast` const)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58802 (Ensure `record_layout_for_printing()` is inlined.)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58852 (Update toolchain to build NetBSD release)

kennytm added a commit to kennytm/rust that referenced this pull request Mar 2, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Mar 2, 2019

Auto merge of #58866 - kennytm:rollup, r=kennytm
Rollup of 15 pull requests

Successful merges:

 - #58216 (Set secure flags when opening a named pipe on Windows)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58750 (Make `Unique::as_ptr`, `NonNull::dangling` and `NonNull::cast` const)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58802 (Ensure `record_layout_for_printing()` is inlined.)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58842 (Forbid duplicating Cargo as a dependency)
 - #58852 (Update toolchain to build NetBSD release)

kennytm added a commit to kennytm/rust that referenced this pull request Mar 2, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Mar 2, 2019

Auto merge of #58866 - kennytm:rollup, r=kennytm
Rollup of 14 pull requests

Successful merges:

 - #58464 (Use the correct stderr when testing libstd)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58802 (Ensure `record_layout_for_printing()` is inlined.)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58842 (Forbid duplicating Cargo as a dependency)
 - #58852 (Update toolchain to build NetBSD release)

Mark-Simulacrum added a commit to Mark-Simulacrum/rust that referenced this pull request Mar 2, 2019

Rollup merge of rust-lang#58730 - timvermeulen:internal_iteration, r=…
…scottmcm

Have all methods of Filter and FilterMap use internal iteration

This PR changes `Filter::{next, next_back, count}` and `FilterMap::{next, next_back}` to all use internal iteration. The `next` and `next_back` methods are changed to directly forward to `try_for_each` and `try_rfold` respectively, using `Result` as the `Try` type. I think that's okay? Alternatively, I could change their implementations to use `LoopState` instead if there's any benefit in doing so.

r? @scottmcm

bors added a commit that referenced this pull request Mar 2, 2019

Auto merge of #58866 - kennytm:rollup, r=kennytm
Rollup of 14 pull requests

Successful merges:

 - #58464 (Use the correct stderr when testing libstd)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58802 (Ensure `record_layout_for_printing()` is inlined.)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58842 (Forbid duplicating Cargo as a dependency)
 - #58852 (Update toolchain to build NetBSD release)

bors added a commit that referenced this pull request Mar 3, 2019

Auto merge of #58878 - Mark-Simulacrum:rollup, r=Mark-Simulacrum
Rollup of 14 pull requests

Successful merges:

 - #58425 ([self-profiler] Make the profiler faster/more efficient)
 - #58605 (Use informational target machine for metadata)
 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58842 (Forbid duplicating Cargo as a dependency)
 - #58865 (Fix C-variadic function printing)

Failed merges:

r? @ghost

bors added a commit that referenced this pull request Mar 3, 2019

Auto merge of #58866 - kennytm:rollup, r=kennytm
Rollup of 14 pull requests

Successful merges:

 - #58730 (Have all methods of Filter and FilterMap use internal iteration)
 - #58780 (ManuallyDrop != MaybeUninit)
 - #58782 (Replace `s` with `self` in docs for str methods taking self.)
 - #58785 (allow specifying attributes for tool lints)
 - #58802 (Ensure `record_layout_for_printing()` is inlined.)
 - #58821 (Fixed a syntax error in the pin docs)
 - #58830 (tidy: deny(rust_2018_idioms))
 - #58832 (Revert switching to GCP on AppVeyor)
 - #58833 (tools/rustbook: deny(rust_2018_idioms))
 - #58835 (tools/remote-test-{client,server}: deny(rust_2018_idioms))
 - #58838 (Fix typo in Vec#resize_with documentation)
 - #58842 (Forbid duplicating Cargo as a dependency)
 - #58852 (Update toolchain to build NetBSD release)
 - #58865 (Fix C-variadic function printing)

@bors bors merged commit 88bd624 into rust-lang:master Mar 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.