Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for `try_reserve`: RFC 2116 fallible collection allocation #48043

Open
aturon opened this issue Feb 7, 2018 · 57 comments
Open

Tracking issue for `try_reserve`: RFC 2116 fallible collection allocation #48043

aturon opened this issue Feb 7, 2018 · 57 comments

Comments

@aturon
Copy link
Member

@aturon aturon commented Feb 7, 2018

This is a tracking issue for the try_reserve part of the RFC "fallible collection allocation" (rust-lang/rfcs#2116).

Steps:

API:

impl /* each of String, Vec<T>, VecDeque<T> */ {
    pub fn try_reserve(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
    pub fn try_reserve_exact(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
}

impl /* each of HashMap<K, V> and HashSet<T> */ {
    pub fn try_reserve(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
}

/// The error type for `try_reserve` methods.
#[derive(Clone, PartialEq, Eq, Debug)]
pub enum TryReserveError { // in std::collections
    /// Error due to the computed capacity exceeding the collection's maximum
    /// (usually `isize::MAX` bytes).
    CapacityOverflow,

    /// The memory allocator returned an error
    AllocError {
        /// The layout of allocation request that failed
        layout: Layout,

        #[doc(hidden)]
        #[unstable(feature = "container_error_extra", issue = "0", reason = "\
            Enable exposing the allocator’s custom error value \
            if an associated type is added in the future: \
            https://github.com/rust-lang/wg-allocators/issues/23")]
        non_exhaustive: (),
    },
}

impl From<LayoutErr> for TryReserveError {
    fn from(_: LayoutErr) -> Self {
        TryReserveError::CapacityOverflow
    }
}
@snf
Copy link
Contributor

@snf snf commented Feb 22, 2018

Hi, this feature will require oom and try_reserve to be implemented. I'm needing try_reserve and I'm willing to work (slowly) on it if noone else does.
I see that the previous commit by @gankro is here and seems to match the RFC description.
Is there anything else I should consider before rebasing that PR?

@snf
Copy link
Contributor

@snf snf commented Mar 3, 2018

I can use some mentorship on what's the best way to implement oom=panic/abort . Should I emit a global variable through rustc depending on this case?

I was taking inspiration from #32900 but there might be an easier way to emit a boolean global variable.

@aturon
Copy link
Member Author

@aturon aturon commented Mar 5, 2018

cc @gankro, can you help mentor?

bors added a commit that referenced this issue Mar 9, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
bors added a commit that referenced this issue Mar 12, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
bors added a commit that referenced this issue Mar 12, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
bors added a commit that referenced this issue Mar 13, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
bors added a commit that referenced this issue Mar 14, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
bors added a commit that referenced this issue Mar 15, 2018
Fallible allocation

Implementing RFC#2116 [Fallible Allocation](#48043) .
Work in progress. Initially adding @gankro's try_reserve for Vec.
@bluss bluss mentioned this issue Mar 25, 2018
19 of 21 tasks complete
@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Apr 5, 2018

try_reserve methods are implemented in Nightly, but the accepted RFC also includes the ability to make allocation failure cause a panic rather than abort and that part is not implemented. Unfortunately the RFC does not discuss how this would be implemented, other than being controlled by a flag in Cargo.toml. It’s not clear to me how this can work.

@glandium
Copy link
Contributor

@glandium glandium commented May 4, 2018

Unfortunately the RFC does not discuss how this would be implemented, other than being controlled by a flag in Cargo.toml. It’s not clear to me how this can work.

We've been talking about panic::set_hook-like hooking for the new oom lang item in #49668, this could be how this works.

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented May 29, 2018

@alexcrichton, how does #51041 fit with the goal of implementing some way to opt into oom=panic?

@alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented May 29, 2018

@SimonSapin we'd decide that the regression is acceptable, and we'd revert the change.

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented May 29, 2018

You use a conditional tense, but oom=panic is already in an accepted RFC (though the implementation strategy is not quite settled).

@alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented May 29, 2018

Ok sure, we can figure it out in the implementation details? AFAIK there's no need to revert the change as it has basically no benefit today. It's a trivial change to revert as well, it's one line in the standard library

Centril added a commit to Centril/rust that referenced this issue Mar 12, 2020
…r, r=sfackler

Implement Error for TryReserveError

I noticed that the Error trait wasn't implemented for TryReserveError. (rust-lang#48043)

Not sure if the error messages and code style are 100% correct, it's my first time contributing to the Rust std.
@LenaWil LenaWil mentioned this issue Mar 14, 2020
0 of 3 tasks complete
@malbarbo
Copy link
Contributor

@malbarbo malbarbo commented May 19, 2020

@SimonSapin any update?

@Ericson2314
Copy link
Contributor

@Ericson2314 Ericson2314 commented May 20, 2020

We now have a renewed effort at #72314 so we can experiment with try_* methods across the board. I wish (albeit from the sidelines) that such a PR had landed before so we could already be experimenting with those things, but even though that's gone slower than I hope I still think it's prudent to wait and not stabilize this.

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented May 20, 2020

@malbarbo I’m not sure what you’re expecting, this is the thread where further discussion about try_reserve should happen. As you can read above, the current status is that stabilization is blocked on deciding what extra data if any TryReserveError::AllocErr should carry, which ties into whether generic allocators should have an associated error types. And more generally there is not good consensus to stabilize before generic allocators overall are more settled.

@Ericson2314 #72314 is "Stability annotations on generic parameters", which would help making containers generic on the allocator type, but how is it related to try_* methods? For example Vec can have try_reserve without an extra type parameter.

@TimDiekmann
Copy link
Member

@TimDiekmann TimDiekmann commented May 20, 2020

ties into whether generic allocators should have an associated error types.

I don't think this is going to be fully resolved until we get some allocators on collections upstream and this is currently blocked by #72314.

@kornelski
Copy link
Contributor

@kornelski kornelski commented Nov 16, 2020

#77118 has been merged. Can stabilization of this proceed?

@TimDiekmann
Copy link
Member

@TimDiekmann TimDiekmann commented Nov 16, 2020

I don't really see, how #77118 is related to this? Where to you want to add an unstable generic annotation?

@kornelski
Copy link
Contributor

@kornelski kornelski commented Nov 17, 2020

Previous comment mentioned it was blocked on #72314, and #72314 has been replaced by #77118.

@Ericson2314
Copy link
Contributor

@Ericson2314 Ericson2314 commented Nov 17, 2020

The rust-lang/wg-allocators#48 meta issue's "Allocator aware types" track lays out the progression from #77118 to rust-lang/wg-allocators#39, "try_* methods for containers".

The idea is not that try_* methods themselves inherently need custom allocators, but custom allocators could be leveraged to prevent the use of non-try_* methods, fostering a oom-robust ecosystem just as we have a burgeoning no_std ecosystem.

Also, do note that the only reason we started with try_reserve is that it allows fragile hacking around not having try_* methods more broadly.

@Manishearth
Copy link
Member

@Manishearth Manishearth commented Dec 17, 2020

So, I'm probably missing something, but is there a similar RFC/tracking issue/etc for Box::try_new? Is there some place I can track progress for this?

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Dec 18, 2020

#77118 has been merged. Can stabilization of this proceed?

In the June 2019 to October 2019 part of this thread, the part of custom allocators that was deemed affecting the design of try_resreve is error types.

That is, if error values for allocation failures (potentially) carry meaningful information then the error type of try_reserve should forward that information to its own callers. In particular if the allocator trait ends up with an associated type for its own errors then the error type of try_reserve should be generic.

By this logic (which my 2019 proposal did not adhere to), try_reserve should only be stabilized after the allocation trait is stabilized. #77118 is only one of the steps required for that.


is there a similar RFC/tracking issue/etc for Box::try_new?

By the letter of RFC 2116 I believe there is not. That RFC only proposed try_reserve for collection types in order to keep its scope small and not get into the design of try_push, try_extend, etc.

In the spirit of that RFC however, try_new is the minimal viable API for fallible allocation in non-collection containers in the same way that try_reserve is for containers. So I’d personally be fine with including it here.

@Manishearth
Copy link
Member

@Manishearth Manishearth commented Dec 18, 2020

In the spirit of that RFC however, try_new is the minimal viable API for fallible allocation in non-collection containers in the same way that try_reserve is for containers. So I’d personally be fine with including it here.

I'm talking to some folks doing Linux kernel work in Rust who would like to see this. Should I just open a PR for it, feature gate it with try_new (or try_reserve?) for the same tracking issue?

@Ericson2314
Copy link
Contributor

@Ericson2314 Ericson2314 commented Dec 18, 2020

@Manishearth maybe rust-lang/wg-allocators#7 might help. You could at least see what unstable annotations those PRs used.

@baumanj
Copy link

@baumanj baumanj commented Dec 18, 2020

I'm talking to some folks doing Linux kernel work in Rust who would like to see this. Should I just open a PR for it, feature gate it with try_new (or try_reserve?) for the same tracking issue?

In the meantime, you may be interested in (or already know about) https://crates.io/crates/fallible_collections. We've been using (and contributing to) that crate for awhile now to support the rust MP4 parser that Gecko uses. I've found it quite nice to work with.

Manishearth added a commit to Manishearth/rust that referenced this issue Dec 22, 2020
cc rust-lang#48043

It was suggested in rust-lang#48043 (comment) that `Box::try_*` follows the spirit of RFC 2116. This PR is an attempt to add the relevant APIs, tied to the same feature gate. Happy to make any changes or turn this into an RFC if necessary.

cc @rust-lang/wg-allocators
@Manishearth
Copy link
Member

@Manishearth Manishearth commented Dec 22, 2020

Opened #80310

bors added a commit to rust-lang-ci/rust that referenced this issue Dec 31, 2020
Add fallible Box, Arc, and Rc allocator APIs

cc rust-lang#48043

It was suggested in rust-lang#48043 (comment) that `Box::try_*` follows the spirit of RFC 2116. This PR is an attempt to add the relevant APIs, tied to the same feature gate. Happy to make any changes or turn this into an RFC if necessary.

cc `@rust-lang/wg-allocators`
bors added a commit to rust-lang-ci/rust that referenced this issue Jan 1, 2021
Add fallible Box, Arc, and Rc allocator APIs

cc rust-lang#48043

It was suggested in rust-lang#48043 (comment) that `Box::try_*` follows the spirit of RFC 2116. This PR is an attempt to add the relevant APIs, tied to the same feature gate. Happy to make any changes or turn this into an RFC if necessary.

cc `@rust-lang/wg-allocators`
bors added a commit to rust-lang-ci/rust that referenced this issue Jan 1, 2021
Add fallible Box, Arc, and Rc allocator APIs

cc rust-lang#48043

It was suggested in rust-lang#48043 (comment) that `Box::try_*` follows the spirit of RFC 2116. This PR is an attempt to add the relevant APIs, tied to the same feature gate. Happy to make any changes or turn this into an RFC if necessary.

cc `@rust-lang/wg-allocators`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet