Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perhaps improve the documentation of drain members further #93769

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion library/alloc/src/collections/binary_heap.rs
Original file line number Diff line number Diff line change
Expand Up @@ -748,12 +748,18 @@ impl<T: Ord> BinaryHeap<T> {

/// Returns an iterator which retrieves elements in heap order.
/// The retrieved elements are removed from the original heap.
/// The remaining elements will be removed on drop in heap order.
/// On drop, the remaining elements are removed and dropped in heap order.
///
/// Note:
/// * `.drain_sorted()` is *O*(*n* \* log(*n*)); much slower than `.drain()`.
/// You should use the latter for most cases.
///
/// # Leaking
///
/// In case the iterator disappears without getting dropped (using
/// [`mem::forget`], for example), it is unspecified whether the
/// remaining elements are still in the heap or leaked.
///
/// # Examples
///
/// Basic usage:
Expand Down Expand Up @@ -1162,6 +1168,14 @@ impl<T> BinaryHeap<T> {
///
/// The elements are removed in arbitrary order.
///
/// # Leaking
///
/// When the iterator **is** dropped, it drops any elements that it has not
/// yet yielded (none if the iterator was fully consumed).
/// If the iterator **is not** dropped (with [`mem::forget`], for example),
/// it is unspecified whether the elements not yet yielded are still in the
/// heap or leaked.
///
/// # Examples
///
/// Basic usage:
Expand Down
8 changes: 8 additions & 0 deletions library/alloc/src/collections/linked_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -970,6 +970,14 @@ impl<T> LinkedList<T> {
/// Note that `drain_filter` lets you mutate every element in the filter closure, regardless of
/// whether you choose to keep or remove it.
///
/// # Leaking
///
/// When the iterator **is** dropped, it drops any elements that it has not
/// yet yielded and for which the closure returns true.
/// If the iterator **is not** dropped (with [`mem::forget`], for example),
/// it is unspecified whether the elements not yet yielded are still in the
/// list or leaked.
///
/// # Examples
///
/// Splitting a list into evens and odds, reusing the original list:
Expand Down
17 changes: 9 additions & 8 deletions library/alloc/src/collections/vec_deque/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1215,15 +1215,16 @@ impl<T, A: Allocator> VecDeque<T, A> {
unsafe { IterMut::new(ring, tail, head, PhantomData) }
}

/// Creates a draining iterator that removes the specified range in the
/// `VecDeque` and yields the removed items.
/// Removes the specified range from the `VecDeque`, returning all removed
/// elements as an iterator.
Comment on lines -1218 to +1219
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind-of prefer the original sentence here. What’s the motivation for the change? The call to drain itself does not do anything (yet) beyond creating an iterator. The returned iterator has a lifetime coupled to the mutable borrow; for an operation that “Removes the specified range … returning all removed elements as an iterator” one could expect an iterator that owns the elements and can be created with a shorter mutable borrow not coupled to a lifetime parameter of the type of the iterator.

The same applies to Vec and String

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in the commit for #92902 and should be discussed there or in #92765.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. I didn't realize that there's another unmerged PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I didn't realize anyone would notice this PR this month.

Copy link
Member

@steffahn steffahn Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just happened to look at the list of latest/new open PRs today, (mainly because I have an open PR myself where I'm a bit impatiently checking whether anything has happened w.r.t. review yet,) then happend to be quickly looking into this one for whatever reason and probably wouldn't even have started any review if I hadn't spotted the typo of "closure". 😅 I hope you don't mind my lengthy side-discussion about validity of collection types that I kind-of opened in this thread 🙈

///
/// Note 1: The element range is removed even if the iterator is not
/// consumed until the end.
/// # Leaking
///
/// Note 2: It is unspecified how many elements are removed from the deque,
/// if the `Drain` value is not dropped, but the borrow it holds expires
/// (e.g., due to `mem::forget`).
/// When the iterator **is** dropped, it drops any elements that it has not
/// yet yielded (none if the iterator was fully consumed).
/// If the iterator **is not** dropped (with [`mem::forget`], for example),
/// it is unspecified which elements remain in the vector; even elements
/// outside the range may have been removed and leaked.
///
/// # Panics
///
Expand All @@ -1240,7 +1241,7 @@ impl<T, A: Allocator> VecDeque<T, A> {
/// assert_eq!(drained, [3]);
/// assert_eq!(v, [1, 2]);
///
/// // A full range clears all contents
/// // A full range clears all contents, like `clear()` does
/// v.drain(..);
/// assert!(v.is_empty());
/// ```
Expand Down
12 changes: 7 additions & 5 deletions library/alloc/src/string.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1628,11 +1628,13 @@ impl String {
self.vec.clear()
}

/// Creates a draining iterator that removes the specified range in the `String`
/// and yields the removed `chars`.
/// Removes the specified range from the string, returning all removed
/// characters as an iterator.
///
/// Note: The element range is removed even if the iterator is not
/// consumed until the end.
/// # Leaking
///
/// In case the iterator disappears without getting dropped (using
/// [`core::mem::forget`], for example), the range remains in the string.
Comment on lines +1636 to +1637
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t understand why this would need to be specified. It encourages usage of mem::forget which usually shouldn’t be used on foreign types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my description.

Copy link
Contributor Author

@ssomers ssomers Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or clarify what you mean with "this": the entire paragraph, or the constraint about what can happen if you leak an instance of this iterator?

Copy link
Member

@steffahn steffahn Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this paragraph kind-of says to me "If you want to, during iteration, change your mind and keep the range inside of the original string, feel free to go ahead and just mem::forget the iterator. This is a supported use-case and we guarantee the precise behavior in this case". I.e. it promises a certain behavior on mem::forgetting the iterator, and thus (implicitly) encourages people to deliberately leak the iterator to achieve certain behavior. If we wanted to offer an actual documented and guaranteed way of avoiding the draining, then it should be via a (self-consuming) method on the iterator, not by giving any promises on "what happens on mem::forget IMO. (E.g. one could create a fn cancel(self) on the string::Drain iterator that explicitly doesn't remove anything from the String, or vec::Drain could provide a method that puts back the remaining elements into the original Vec.)

All we should do in these kinds of documentation is say, very explicitly, something along the lines of "please don't ever skip dropping these iterators properly, it is considered a logic error and can, among other things, leave the original structure in an arbitrary unspecified (yet safe-to-use in terms of memory-safety) state with potentially surprising and unexpected behavior when you continue using it afterwards, it can result in arbitrary amounts of additional leaks of the to-be-drained items and/or other items in the same original data structure". I.e. strongly discourage actually leaking the iterators, and perhaps provide a non-exhaustive list of things that can happen upon leaking it anyways, without giving any promises that constrain the set of things that can happen any more than necessary by Rust's memory safety guarantees.

Copy link
Member

@steffahn steffahn Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my description.

Do you mean this following sentence?

Apparently people do use mem::forget a lot and are surprised by its effect, see the history of the notice on vec::drain.

If "the history of the notice on vec::drain" is supposed to explain this, can you perhaps provide some links for me to read up on this?

Copy link
Contributor Author

@ssomers ssomers Feb 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my description.

Do you mean this following sentence?

I mean primarily that the description already lists reasons why it would need to be specified as well as reasons not to.

Apparently people do use mem::forget a lot and are surprised by its effect, see the history of the notice on vec::drain.

If "the history of the notice on vec::drain" is supposed to explain this, can you perhaps provide some links for me to read up on this?

I went over #43244 again and don't see it. It's in the existence of #74652 and its predecessors introducing the paragraph. But I probably misinterperted #73844: it's the description that inspired the clarification, not someone actually using mem::forget and expecting to still look at the vector later. Some other posts too I'm sure, but probably less than I thought.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vec::Drain could provide a method that puts back the remaining elements into the original Vec.)

Intriguing idea, I think no one mentioned that possibility yet.

All we should do in these kinds of documentation is say, very explicitly, something along the lines of

I agree, but apparently no one wanted to discourage users in the Vec::drain description.

///
/// # Panics
///
Expand All @@ -1652,7 +1654,7 @@ impl String {
/// assert_eq!(t, "α is alpha, ");
/// assert_eq!(s, "β is beta");
///
/// // A full range clears the string
/// // A full range clears the string, like `clear()` does
/// s.drain(..);
/// assert_eq!(s, "");
/// ```
Expand Down
25 changes: 18 additions & 7 deletions library/alloc/src/vec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1799,13 +1799,16 @@ impl<T, A: Allocator> Vec<T, A> {
self.len += count;
}

/// Creates a draining iterator that removes the specified range in the vector
/// and yields the removed items.
/// Removes the specified range from the vector, returning all removed
/// elements as an iterator.
///
/// When the iterator **is** dropped, all elements in the range are removed
/// from the vector, even if the iterator was not fully consumed. If the
/// iterator **is not** dropped (with [`mem::forget`] for example), it is
/// unspecified how many elements are removed.
/// # Leaking
///
/// When the iterator **is** dropped, it drops any elements that it has not
/// yet yielded (none if the iterator was fully consumed).
/// If the iterator **is not** dropped (with [`mem::forget`], for example),
/// it is unspecified which elements remain in the vector; even elements
/// outside the range may have been removed and leaked.
///
/// # Panics
///
Expand All @@ -1820,7 +1823,7 @@ impl<T, A: Allocator> Vec<T, A> {
/// assert_eq!(v, &[1]);
/// assert_eq!(u, &[2, 3]);
///
/// // A full range clears the vector
/// // A full range clears the vector, like `clear()` does
/// v.drain(..);
/// assert_eq!(v, &[]);
/// ```
Expand Down Expand Up @@ -2731,6 +2734,14 @@ impl<T, A: Allocator> Vec<T, A> {
/// Note that `drain_filter` also lets you mutate every element in the filter closure,
/// regardless of whether you choose to keep or remove it.
///
/// # Leaking
///
/// When the iterator **is** dropped, it drops any elements that it has not
/// yet yielded and for which the closure returns true.
/// If the iterator **is not** dropped (with [`mem::forget`], for example),
/// it is unspecified which elements remain in the vector; even elements
/// outside the range may have been removed and leaked.
///
/// # Examples
///
/// Splitting an array into evens and odds, reusing the original allocation:
Expand Down
2 changes: 1 addition & 1 deletion library/std/src/collections/hash/set.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ impl<T, S> HashSet<T, S> {
self.base.is_empty()
}

/// Clears the set, returning all elements in an iterator.
/// Clears the set, returning all elements as an iterator.
///
/// # Examples
///
Expand Down