Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Iterator::next for GenericShunt used in iter::try_process #107674

Closed

Conversation

JustForFun88
Copy link

@JustForFun88 JustForFun88 commented Feb 4, 2023

This pull request simplifies the iterator implementation for the GenericShunt structure used in iter::try_process. Although the old implementation commands respect and admiration, but:

  1. To be honest, it is not clear not at first sight, nor from the second. And although it is delightful in its conciseness, I spent half a day trying to get through the wilds of mutual function calls.
  2. Uses too many unnecessary branches and function calls (see code below). Also, using an implicit for + return loop instead of a direct iter.next() seems odd. While the compiler will likely optimize all of this, the implementation proposed in this pull request will probably compile faster.

In any case, feel free to just close this PR if you think it's redundant.

cc @scottmcm.

P.S. More explicit old code:

impl<I, R> Iterator for GenericShunt<'_, I, R>
where
    I: Iterator<Item: Try<Residual = R>>,
{
    type Item = <I::Item as Try>::Output;

    fn next(&mut self) -> Option<Self::Item> {
        let result = self.iter.try_fold((), |acc, x| match <I::Item as Try>::branch(x) {
            ControlFlow::Continue(x) => ControlFlow::Break(ControlFlow::Break(x)),
            ControlFlow::Break(r) => {
                *self.residual = Some(r);
                ControlFlow::Break(ControlFlow::Continue(acc))
            }
        });

        let out = match result {
            ControlFlow::Continue(val) => return None,
            ControlFlow::Break(val) => match val {
                ControlFlow::Continue(_) => return None,
                ControlFlow::Break(val) => val,
            },
        };
        Some(out)
    }
}

@rustbot
Copy link
Collaborator

rustbot commented Feb 4, 2023

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @scottmcm (or someone else) soon.

Please see the contribution instructions for more information.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Feb 4, 2023
@rustbot
Copy link
Collaborator

rustbot commented Feb 4, 2023

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@scottmcm
Copy link
Member

scottmcm commented Feb 4, 2023

This is a fairly standard way to not need to manually implement next -- notice that the same pattern is used here

fn next(&mut self) -> Option<Self::Item> {
self.try_for_each(ControlFlow::Break).break_value()
}

There are lots of other ways it can be written, if any of these help you follow what it's doing:

self.try_for_each(Err).err()
self.find(|_| true)
self.find_map(Some)

but they're all using the same core idea, which is a loop that always returns the first value.

And if you look at the callers of try_process (https://github.com/rust-lang/rust/search?q=try_process), you'll see that they're doing things like sum or collect over the iterator, which tend to use the loop forms, not next:

impl Sum for $a {
fn sum<I: Iterator<Item=Self>>(iter: I) -> Self {
iter.fold(
$zero,
#[rustc_inherit_overflow_checks]
|a, b| a + b,
)
}
}

So my instinct here is to stay with the concise code for now.

That said, if you have some benchmarks showing that doing next this way makes collecting into, say, a Result<Vec<_>, _> faster, then I'd be happy to reconsider.

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 4, 2023
@JustForFun88
Copy link
Author

That said, if you have some benchmarks showing that doing next this way makes collecting into, say, a Result<Vec<_>, _> faster, then I'd be happy to reconsider.

I tried as you said to compare the compiled code (old: https://rust.godbolt.org/z/K3WaKP4z7, new: https://rust.godbolt.org/z/cWjPj781r). With optimization level 3, the code in both versions is compiled into the same asm. The difference appears only when the optimization level is less than or equal to 1 and reaches 10%. That is, there should be an improvement only in compile time. Should I close the PR?

@rust-log-analyzer
Copy link
Collaborator

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
........................................................................................ 1496/1532
....................................
failures:

---- iter::traits::iterator::iter_try_collect_uses_try_fold_not_next stdout ----
thread 'iter::traits::iterator::iter_try_collect_uses_try_fold_not_next' panicked at 'Iterator::next should not be called!', library/core/tests/iter/traits/iterator.rs:561:13
error: test failed, to rerun pass `-p core --test coretests`

failures:
    iter::traits::iterator::iter_try_collect_uses_try_fold_not_next

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants