New `Itertools::tail` #899

Philippe-Cholet · 2024-03-08T16:25:14Z

Hi, after 80 merged PRs fixing, modifying, testing, benchmarking, specializing or whatever really... I just checked this is my first PR where I actually add a "brand new" feature myself. 🤣

PS from the future: The discussion also lead me to contribute a small optimization inside VecDeque::pop_{back,front} that will be released in rust 1.79.0. 😎

codecov · 2024-03-08T16:27:15Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.50%. Comparing base (6814180) to head (b51f26a).
Report is 36 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #899      +/-   ##
==========================================
+ Coverage   94.38%   94.50%   +0.11%     
==========================================
  Files          48       48              
  Lines        6665     6807     +142     
==========================================
+ Hits         6291     6433     +142     
  Misses        374      374

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Philippe-Cholet · 2024-03-09T07:45:28Z

I wonder if libcore would want a Iterator::tail<const N: usize> (not as straightforward as this implementation in case the iterator generate less than N items but it's possible) in which case name my method Itertools::tail(n: usize) would clash.

But we could then wonder about k_smallest and its variants where [_; N] is enough storage.

Philippe-Cholet · 2024-03-14T13:26:59Z

I fused the iterator instead of checking data.len() == n (same as recent fix #900).

src/lib.rs

scottmcm · 2024-03-14T16:47:24Z

src/lib.rs

+    /// `.rev().take(n).collect_vec().into_iter().rev()`
+    /// to have the same result without consuming the entire iterator.
+    #[cfg(feature = "use_alloc")]
+    fn tail(self, n: usize) -> VecIntoIter<Self::Item>


Pondering naming: would it be worth a name that more directly emphasizes the allocating or the approach? Like a too-long name might be collect_last_n. I wonder how best to make it clear that this isn't lazy like take.

Relatedly, I'm curious how people would expect this to behave and what trait bounds they'd expect. I often expect things looking at the end of an iterator to want DoubleEndedIterator, but this is very much a "look, I need to look at the end but I don't have a double-ended iterator" kind of thing.

So I wonder if tail would make more sense for something using the *_back methods, and this version should have a longer name of some sort.

We are not doing const-generics yet but I'm sometimes thinking about it.

match iter.next_chunk() { // I want this stabilized BTW. Ok(arr) => { /*update; rotate;*/ arr.into_iter() } Err(arr_iter) => arr_iter, }

All that to say it would return core::array::IntoIter and that VecIntoIter seems the right type and I would expect a simple Vec from collect_....

I'm also wondering if tail is the best name but I did not find a better one.

src/lib.rs

phimuemue

Hi @Philippe-Cholet, looks reasonable.

I approve this so that you can merge, but I'd appreciate a second pair of eyes, maybe @scottmcm?

src/lib.rs

scottmcm

"Product" code looks good, though you probably want some test+doc updates before merging.

Philippe-Cholet · 2024-03-16T09:09:40Z

I polished the commits and added @scottmcm as co-author.
I particularly like his idea of skipping the starting part of the iterator.

Is there a better name than tail? (see this)
If not then I think this can be merged.

Note for myself: File a rust issue for a possible method more optimized than .pop_front()+.push_back(_), compared to my vector implementation (see this).

EDIT: Discussion there.

src/lib.rs

The `.tail(1)` test is for code coverage.

If the iterator is exact sized, then `.collect()` finishes the work. More generally, if the size hint knows enough and `nth` is efficient, this might skip most of the iterator efficiently. In the tests, `.filter(..)` is there to ensure that `tail` can't leverage a precise `size_hint` to entirely skip the iteration after initial `.collect()`. Co-Authored-By: scottmcm <scottmcm@users.noreply.github.com>

`rotate_left` is more efficient on `VecDeque` than on a slice. `VecDeque::from(vec)` is O(1) on recent rust. Co-Authored-By: scottmcm <scottmcm@users.noreply.github.com>

…cap, r=<try> Add invariant to VecDeque::pop_* that len < cap if pop successful Similar to rust-lang#114370 for VecDeque instead of Vec. I initially come from rust-itertools/itertools#899 where we noticed that `pop_front;push_back;` was slower than expected so `@scottmcm` suggested I file an issue which lead to https://internals.rust-lang.org/t/vecdeque-pop-front-push-back/20483 where **kornel** mentionned rust-lang#114334 (fixed by rust-lang#114370). This is my first time with codegen tests, I based the test on what was done for Vec.

…e_cap, r=Nilstrieb Add invariant to VecDeque::pop_* that len < cap if pop successful Similar to rust-lang#114370 for VecDeque instead of Vec. I initially come from rust-itertools/itertools#899 where we noticed that `pop_front;push_back;` was slower than expected so `@scottmcm` suggested I file an issue which lead to https://internals.rust-lang.org/t/vecdeque-pop-front-push-back/20483 where **kornel** mentionned rust-lang#114334 (fixed by rust-lang#114370). This is my first time with codegen tests, I based the test on what was done for Vec.

Rollup merge of rust-lang#123089 - Philippe-Cholet:vecdeque_pop_assume_cap, r=Nilstrieb Add invariant to VecDeque::pop_* that len < cap if pop successful Similar to rust-lang#114370 for VecDeque instead of Vec. I initially come from rust-itertools/itertools#899 where we noticed that `pop_front;push_back;` was slower than expected so `@scottmcm` suggested I file an issue which lead to https://internals.rust-lang.org/t/vecdeque-pop-front-push-back/20483 where **kornel** mentionned rust-lang#114334 (fixed by rust-lang#114370). This is my first time with codegen tests, I based the test on what was done for Vec.

Philippe-Cholet force-pushed the tail-but-no-head branch from ea470e7 to 42cac22 Compare March 8, 2024 22:31

Philippe-Cholet mentioned this pull request Mar 9, 2024

Fix Itertools::k_smallest on short unfused iterators #900

Merged

Philippe-Cholet force-pushed the tail-but-no-head branch from 42cac22 to a3557a7 Compare March 14, 2024 13:25

Philippe-Cholet requested a review from phimuemue March 14, 2024 14:20

scottmcm reviewed Mar 14, 2024

View reviewed changes