Skip to content

Endless Iterators #90093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Endless Iterators #90093

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions library/core/src/iter/adapters/chain.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use crate::iter::traits::EndlessIterator;
use crate::iter::{DoubleEndedIterator, FusedIterator, Iterator, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -324,3 +325,9 @@ where
B: TrustedLen<Item = A::Item>,
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<A: EndlessIterator, B> EndlessIterator for Chain<A, B> where B: Iterator<Item = A::Item> {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<A, B: EndlessIterator> EndlessIterator for Chain<A, B> where A: Iterator<Item = B::Item> {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the Zip issue: If A is non-empty but finite and B is empty then then Chain is non-empty and finite.

9 changes: 9 additions & 0 deletions library/core/src/iter/adapters/cloned.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use crate::iter::adapters::{
zip::try_get_unchecked, TrustedRandomAccess, TrustedRandomAccessNoCoerce,
};
use crate::iter::traits::EndlessIterator;
use crate::iter::{FusedIterator, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -141,3 +142,11 @@ where
T: Clone,
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<'a, I, T: 'a> EndlessIterator for Cloned<I>
where
I: EndlessIterator<Item = &'a T>,
T: Clone,
{
}
9 changes: 9 additions & 0 deletions library/core/src/iter/adapters/copied.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use crate::iter::adapters::{
zip::try_get_unchecked, TrustedRandomAccess, TrustedRandomAccessNoCoerce,
};
use crate::iter::traits::EndlessIterator;
use crate::iter::{FusedIterator, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -167,3 +168,11 @@ where
T: Copy,
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<'a, I, T: 'a> EndlessIterator for Copied<I>
where
I: EndlessIterator<Item = &'a T>,
T: Copy,
{
}
11 changes: 10 additions & 1 deletion library/core/src/iter/adapters/cycle.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
use crate::{iter::FusedIterator, ops::Try};
use crate::{
iter::{
traits::{EndlessIterator, EndlessOrExact},
FusedIterator,
},
ops::Try,
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: multiple separate use statements would be shorter than the nested ones here.


/// An iterator that repeats endlessly.
///
Expand Down Expand Up @@ -106,3 +112,6 @@ where

#[stable(feature = "fused", since = "1.26.0")]
impl<I> FusedIterator for Cycle<I> where I: Clone + Iterator {}

#[unstable(issue = "none", feature = "none")]
unsafe impl<I> EndlessIterator for Cycle<I> where I: Clone + EndlessOrExact {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EndlessIterator is unsafe, EndlessOrExact can be implemented by ExactSizeIterator which isn't.

If this is intended to be unsafe then we need to introduce TrustedUsizeIterator to describe this propery or use TrustedRandomAccessNoCoerce which already happens to have this requirement but also has other ones on top.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If trying to reflect a "not more than an allocatable amount" property, that may be better off with a type representing the logic of fn alloc_guard.

rust/library/alloc/src/raw_vec.rs

Lines 539 to 555 in 1067e2c

// We need to guarantee the following:
// * We don't ever allocate `> isize::MAX` byte-size objects.
// * We don't overflow `usize::MAX` and actually allocate too little.
//
// On 64-bit we just need to check for overflow since trying to allocate
// `> isize::MAX` bytes will surely fail. On 32-bit and 16-bit we need to add
// an extra guard for this in case we're running on a platform which can use
// all 4GB in user-space, e.g., PAE or x32.
#[inline]
fn alloc_guard(alloc_size: usize) -> Result<(), TryReserveError> {
if usize::BITS < 64 && alloc_size > isize::MAX as usize {
Err(CapacityOverflow.into())
} else {
Ok(())
}
}

4 changes: 4 additions & 0 deletions library/core/src/iter/adapters/filter.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::fmt;
use crate::iter::traits::EndlessIterator;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable};
use crate::ops::Try;

Expand Down Expand Up @@ -150,3 +151,6 @@ where

#[unstable(issue = "none", feature = "inplace_iteration")]
unsafe impl<I: InPlaceIterable, P> InPlaceIterable for Filter<I, P> where P: FnMut(&I::Item) -> bool {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator, P> EndlessIterator for Filter<I, P> where P: FnMut(&I::Item) -> bool {}
8 changes: 7 additions & 1 deletion library/core/src/iter/adapters/filter_map.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::fmt;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable};
use crate::iter::{adapters::SourceIter, traits::EndlessIterator, FusedIterator, InPlaceIterable};
use crate::ops::{ControlFlow, Try};

/// An iterator that uses `f` to both filter and map elements from `iter`.
Expand Down Expand Up @@ -147,3 +147,9 @@ unsafe impl<B, I: InPlaceIterable, F> InPlaceIterable for FilterMap<I, F> where
F: FnMut(I::Item) -> Option<B>
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<B, I: EndlessIterator, F> EndlessIterator for FilterMap<I, F> where
F: FnMut(I::Item) -> Option<B>
{
}
37 changes: 37 additions & 0 deletions library/core/src/iter/adapters/flatten.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::fmt;
use crate::iter::traits::{EndlessIterator, EndlessOrExact};
use crate::iter::{DoubleEndedIterator, Fuse, FusedIterator, Iterator, Map, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -138,6 +139,26 @@ where
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I, U, F> EndlessIterator for FlatMap<I, U, F>
where
I: EndlessIterator,
U: IntoIterator,
U::IntoIter: EndlessOrExact,
F: FnMut(I::Item) -> U,
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I, U, F> EndlessIterator for FlatMap<I, U, F>
where
I: EndlessOrExact,
U: IntoIterator,
U::IntoIter: EndlessIterator,
F: FnMut(I::Item) -> U,
{
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the bounds here seem redundant. If the inner iterators are Endless then it doesn't matter how many items the outer iterator has, the final FlatMap will either yield zero or infinite items.


/// An iterator that flattens one level of nesting in an iterator of things
/// that can be turned into iterators.
///
Expand Down Expand Up @@ -262,6 +283,22 @@ where
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I> EndlessIterator for Flatten<I>
where
I: EndlessIterator,
I::Item: EndlessOrExact,
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I> EndlessIterator for Flatten<I>
where
I: EndlessOrExact,
I::Item: EndlessIterator,
{
}

/// Real logic of both `Flatten` and `FlatMap` which simply delegate to
/// this type.
#[derive(Clone, Debug)]
Expand Down
4 changes: 4 additions & 0 deletions library/core/src/iter/adapters/fuse.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use crate::intrinsics;
use crate::iter::adapters::zip::try_get_unchecked;
use crate::iter::traits::EndlessIterator;
use crate::iter::{
DoubleEndedIterator, ExactSizeIterator, FusedIterator, TrustedLen, TrustedRandomAccess,
TrustedRandomAccessNoCoerce,
Expand Down Expand Up @@ -233,6 +234,9 @@ where
const MAY_HAVE_SIDE_EFFECT: bool = I::MAY_HAVE_SIDE_EFFECT;
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator> EndlessIterator for Fuse<I> {}

/// Fuse specialization trait
///
/// We only need to worry about `&mut self` methods, which
Expand Down
4 changes: 4 additions & 0 deletions library/core/src/iter/adapters/inspect.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::fmt;
use crate::iter::traits::EndlessIterator;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable};
use crate::ops::Try;

Expand Down Expand Up @@ -164,3 +165,6 @@ where

#[unstable(issue = "none", feature = "inplace_iteration")]
unsafe impl<I: InPlaceIterable, F> InPlaceIterable for Inspect<I, F> where F: FnMut(&I::Item) {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator, F> EndlessIterator for Inspect<I, F> where F: FnMut(&I::Item) {}
8 changes: 8 additions & 0 deletions library/core/src/iter/adapters/intersperse.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use crate::iter::traits::EndlessIterator;

use super::Peekable;

/// An iterator adapter that places a separator between all elements.
Expand Down Expand Up @@ -57,6 +59,9 @@ where
}
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator> EndlessIterator for Intersperse<I> where I::Item: Clone {}

/// An iterator adapter that places a separator between all elements.
///
/// This `struct` is created by [`Iterator::intersperse_with`]. See its
Expand Down Expand Up @@ -185,3 +190,6 @@ where
accum
})
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator, G: FnMut() -> I::Item> EndlessIterator for IntersperseWith<I, G> {}
9 changes: 9 additions & 0 deletions library/core/src/iter/adapters/map.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ use crate::fmt;
use crate::iter::adapters::{
zip::try_get_unchecked, SourceIter, TrustedRandomAccess, TrustedRandomAccessNoCoerce,
};
use crate::iter::traits::EndlessIterator;
use crate::iter::{FusedIterator, InPlaceIterable, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -216,3 +217,11 @@ where

#[unstable(issue = "none", feature = "inplace_iteration")]
unsafe impl<B, I: InPlaceIterable, F> InPlaceIterable for Map<I, F> where F: FnMut(I::Item) -> B {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<B, I, F> EndlessIterator for Map<I, F>
where
I: EndlessIterator,
F: FnMut(I::Item) -> B,
{
}
4 changes: 4 additions & 0 deletions library/core/src/iter/adapters/peekable.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use crate::iter::traits::EndlessIterator;
use crate::iter::{adapters::SourceIter, FusedIterator, TrustedLen};
use crate::ops::{ControlFlow, Try};

Expand Down Expand Up @@ -333,3 +334,6 @@ where
unsafe { SourceIter::as_inner(&mut self.iter) }
}
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator> EndlessIterator for Peekable<I> {}
4 changes: 4 additions & 0 deletions library/core/src/iter/adapters/rev.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use crate::iter::traits::EndlessIterator;
use crate::iter::{FusedIterator, TrustedLen};
use crate::ops::Try;

Expand Down Expand Up @@ -135,3 +136,6 @@ impl<I> FusedIterator for Rev<I> where I: FusedIterator + DoubleEndedIterator {}

#[unstable(feature = "trusted_len", issue = "37572")]
unsafe impl<I> TrustedLen for Rev<I> where I: TrustedLen + DoubleEndedIterator {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I> EndlessIterator for Rev<I> where I: EndlessIterator + DoubleEndedIterator {}
5 changes: 4 additions & 1 deletion library/core/src/iter/adapters/skip.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::intrinsics::unlikely;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable};
use crate::iter::{adapters::SourceIter, traits::EndlessIterator, FusedIterator, InPlaceIterable};
use crate::ops::{ControlFlow, Try};

/// An iterator that skips over `n` elements of `iter`.
Expand Down Expand Up @@ -237,3 +237,6 @@ where

#[unstable(issue = "none", feature = "inplace_iteration")]
unsafe impl<I: InPlaceIterable> InPlaceIterable for Skip<I> {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator> EndlessIterator for Skip<I> {}
7 changes: 7 additions & 0 deletions library/core/src/iter/adapters/skip_while.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::fmt;
use crate::iter::traits::EndlessIterator;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable};
use crate::ops::Try;

Expand Down Expand Up @@ -123,3 +124,9 @@ unsafe impl<I: InPlaceIterable, F> InPlaceIterable for SkipWhile<I, F> where
F: FnMut(&I::Item) -> bool
{
}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator, F> EndlessIterator for SkipWhile<I, F> where
F: FnMut(&I::Item) -> bool
{
}
9 changes: 8 additions & 1 deletion library/core/src/iter/adapters/step_by.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
use crate::{intrinsics, iter::from_fn, ops::Try};
use crate::{
intrinsics,
iter::{from_fn, traits::EndlessIterator},
ops::Try,
};

/// An iterator for stepping iterators by a custom amount.
///
Expand Down Expand Up @@ -233,3 +237,6 @@ where
// StepBy can only make the iterator shorter, so the len will still fit.
#[stable(feature = "iterator_step_by", since = "1.28.0")]
impl<I> ExactSizeIterator for StepBy<I> where I: ExactSizeIterator {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<I: EndlessIterator> EndlessIterator for StepBy<I> {}
6 changes: 4 additions & 2 deletions library/core/src/iter/adapters/take.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
use crate::cmp;
use crate::iter::{adapters::SourceIter, FusedIterator, InPlaceIterable, TrustedLen};
use crate::iter::{
adapters::SourceIter, traits::EndlessOrExact, FusedIterator, InPlaceIterable, TrustedLen,
};
use crate::ops::{ControlFlow, Try};

/// An iterator that only iterates over the first `n` iterations of `iter`.
Expand Down Expand Up @@ -234,7 +236,7 @@ where
}

#[stable(feature = "rust1", since = "1.0.0")]
impl<I> ExactSizeIterator for Take<I> where I: ExactSizeIterator {}
impl<I> ExactSizeIterator for Take<I> where I: EndlessOrExact {}

#[stable(feature = "fused", since = "1.26.0")]
impl<I> FusedIterator for Take<I> where I: FusedIterator {}
Expand Down
7 changes: 7 additions & 0 deletions library/core/src/iter/adapters/zip.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use crate::cmp;
use crate::fmt::{self, Debug};
use crate::iter::traits::EndlessIterator;
use crate::iter::{DoubleEndedIterator, ExactSizeIterator, FusedIterator, Iterator};
use crate::iter::{InPlaceIterable, SourceIter, TrustedLen};

Expand Down Expand Up @@ -435,6 +436,12 @@ where
// (without calling next()) would be needed to properly drop the remainder of the source.
unsafe impl<A: InPlaceIterable, B: Iterator> InPlaceIterable for Zip<A, B> where A::Item: Copy {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<A: EndlessIterator, B: Iterator> EndlessIterator for Zip<A, B> {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<A: Iterator, B: EndlessIterator> EndlessIterator for Zip<A, B> {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If one is endless and the other is finite then the resulting Zip is also finite, so those bounds aren't correct, both need to be endless.


#[stable(feature = "rust1", since = "1.0.0")]
impl<A: Debug, B: Debug> Debug for Zip<A, B> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
Expand Down
5 changes: 4 additions & 1 deletion library/core/src/iter/sources/repeat.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::iter::{FusedIterator, TrustedLen};
use crate::iter::{traits::EndlessIterator, FusedIterator, TrustedLen};

/// Creates a new iterator that endlessly repeats a single element.
///
Expand Down Expand Up @@ -127,3 +127,6 @@ impl<A: Clone> FusedIterator for Repeat<A> {}

#[unstable(feature = "trusted_len", issue = "37572")]
unsafe impl<A: Clone> TrustedLen for Repeat<A> {}

#[unstable(issue = "none", feature = "endless")]
unsafe impl<A: Clone> EndlessIterator for Repeat<A> {}
23 changes: 23 additions & 0 deletions library/core/src/iter/traits/marker.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,26 @@ pub unsafe trait InPlaceIterable: Iterator {}
#[unstable(feature = "trusted_step", issue = "85731")]
#[rustc_specialization_trait]
pub unsafe trait TrustedStep: Step {}

/// An iterator that is either empty or endless.
///
/// The iterator reports a size hint that is either `0, Some(0)` or `usize::MAX, None`.
Copy link
Member

@the8472 the8472 Oct 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it also require the size hint to be correct at at all times? Because Flatten is lazy and initially returns 0, Some(0) even when it's not empty.

Also note that generally (usize::MAX, None) can also mean "length exceeds usize::MAX", so the comment should note that the trait gives it a more specific meaning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You had be worried for a second here -- I think you mean 0, None even when it's not empty? https://doc.rust-lang.org/1.56.0/src/core/iter/adapters/flatten.rs.html#313-332

If Flatten can return 0, Some(0) when it's not empty, please file a bug!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant (0, None)

/// Since this is used to implement `ExactSizeIterator` for some adapters, iterators that
/// are only empty should not implement this.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this one. Implementing ExactSizeIterator for an always-empty iterator would be valid.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be used to implement ExactSizeIterator for Take<Repeat<T>>.

///
/// # Safety
///
/// The implementation must either be endless or empty.
#[unstable(issue = "none", feature = "endless")]
#[marker]
pub unsafe trait EndlessIterator: Iterator {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you intend to add specializations or implement other unsafe traits based on this one? Otherwise I can't see any unsafe use of that trait in the current PR.

Also, the name is confusing if it can be either empty or endless. I had to revise my reasoning several times after remembering that "endless" here can also mean empty.


#[unstable(issue = "none", feature = "endless")]
#[marker]
pub trait EndlessOrExact: Iterator {}

#[unstable(issue = "none", feature = "endless")]
impl<T: EndlessIterator> EndlessOrExact for T {}

#[unstable(issue = "none", feature = "endless")]
impl<T: ExactSizeIterator> EndlessOrExact for T {}
4 changes: 4 additions & 0 deletions library/core/src/iter/traits/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ pub use self::double_ended::DoubleEndedIterator;
pub use self::exact_size::ExactSizeIterator;
#[stable(feature = "rust1", since = "1.0.0")]
pub use self::iterator::Iterator;
#[unstable(issue = "none", feature = "endless")]
pub use self::marker::EndlessIterator;
#[unstable(issue = "none", feature = "endless")]
pub(super) use self::marker::EndlessOrExact;
#[unstable(issue = "none", feature = "inplace_iteration")]
pub use self::marker::InPlaceIterable;
#[unstable(feature = "trusted_step", issue = "85731")]
Expand Down
Loading