Tracking issue for custom allocators in standard collections

[alexcrichton]

The new Alloc trait defined in RFC 1398 was added in #42313 but the libs teamdecided to hold off on Vec integration just yet to be conservative.

This issue is intended to track the integration of the Alloc trait into standard collections like Vec, BTreeMap, etc. I'm not personally aware of any current blockers, but what we'll probably want to do as part of this issue includes:

• Develop a set of conventions for collections which support custom allocators
• Ensure no regressions with integration of a custom allocator:
  • error messages should look as they do today
  • no performance regressions at runtime
  • ideally minimal compile-time impact

[Ericson2314]

There are some very interesting issues here:

• "Flat collections" (Vec, VecDec) can easily store a &mut to the allocator they were crated with, indirection-heavy ones would be significantly bloated if every Box also had an allocator reference.

• Ideally all collections would be resilient in the face of allocation failure, but that will take some major algorithms design work. It's better to convert quickly blowing up on OOM as they do today.

• I had the idea of doing trait InfallableAllocator = Allocator<Error=!> to signify an allocator that wouldn't fail (and thus must panick on OOM). If, when we en-mass convert collections, we have them take an InfallableAllocator parameter, it will be clearer to the end-user that the collections are not oom-resilient, and there would be less panics strune about.

  • I'm not up-to-date with all the discussion of invalid Layouts, so I'll need to double-check that ! wouldn't instead need to be some strict Unsupported;. Then again, it isn't like the collections will be able to handle an unsupported layout either...

• Not sure what the ETA on default parameters is, so we're probably best off reexporting collections in std with the allocator parameter specialized to the global allcoator. This will work today, right?

[joshlf]

@Ericson2314

I had the idea of doing trait InfallableAllocator = Allocator<Error=!> to signify an allocator that wouldn't fail (and thus must panick on OOM). If, when we en-mass convert collections, we have them take an InfallableAllocator parameter, it will be clearer to the end-user that the collections are not oom-resilient, and there would be less panics strune about.

I believe that the current design doesn't allow you to parametrize the error, so it must return the concrete error type with two arms - Unsupported and Exhausted - and if you want to panic on OOM, it's your (the client's) responsibility to do that by panicking yourself or by calling the oom method.

[Ericson2314]

@joshlf err yes, this is predicated on us adding an associated error type.

[ghost]

To support the allocator trait in the collection types, the constructors will need to be duplicated.
@pnkfelix has used the _in suffix and appended the allocator to the list of arguments:

impl RawVec<T, A>
    fn with_capacity(n: usize) -> RawVec<T, HeapAlloc>;
    fn with_capacity_in(n: usize, a: A) -> RawVec<T, A>;
}

I personally quite like this approach.

[ghost]

@Ericson2314 mentioned this:

"Flat collections" (Vec, VecDec) can easily store a &mut to the allocator they were crated with, indirection-heavy ones would be significantly bloated if every Box also had an allocator reference.

I am currently implementing a non-trivial allocator myself (for a relocatable heap) and noticed that it don't need to keep anything around for deallocation (the type information is still needed).

This still works when multible allocators are used.

I wonder if this could be integrated somehow? Associated types to indicate what needs to be stored?

[joshlf]

@s3bk

I am currently implementing a non-trivial allocator myself (for a relocatable heap) and noticed that it don't need to keep anything around for deallocation (the type information is still needed).

This still works when multible allocators are used.

I wonder if this could be integrated somehow? Associated types to indicate what needs to be stored?

So the idea is that, for allocators where all instances of the type refer to the same global singleton, you wouldn't actually need to store a reference? Presumably that wouldn't work for allocators that could actually have multiple distinct instances, right?

This motivates a follow-up: if you were implementing a global singleton, couldn't you just make the actual Alloc type that you expose to users be a ZST that delegates to some private instance under the hood?

[ghost]

@joshlf

So the idea is that, for allocators where all instances of the type refer to the same global singleton, you wouldn't actually need to store a reference? Presumably that wouldn't work for allocators that could actually have multiple distinct instances, right?

Yes, for allocators with multiple instances the allocation function would benefit from non-zero knowledge.
(Mine needs to know the base address for instance. And working with more than one allocator is perfectly reasonable.)
What I was trying to point at, is that the pointer passed to dealloc is enough information to recover the actual instance used to allocate the memory.

[rkruppe]

@s3bk How common is the case where you will only ever need to deallocate? All the standard collections and smart pointers have methods that allocate new memory from the same allocator (usually clone; Rc and Arc have make_mut).

Hm, now that I think about it, those methods already need to clone the allocator, so that would potentially offer a way out. Although it's unclear to me if we can switch between storing A and A::DeallocInfo in the collection depending on whether A: Clone, and it's also not clear to me if that would be a good design (it seems to work out mostly by chance, assuming it works out at all).

[ghost]

@rkruppe there could be

trait Alloc {
    type DeallocInfo;
    fn dealloc(&self, ptr: *mut T, layout: Layout);
    fn dealloc_with(info: Self::DeallocInfo, ptr: *mut T, layout: Layout);
    // snip
}
struct Rc<T, A: Alloc> {
  dealloc: A::DeallocInfo;
  // snip
}
impl<T> Rc<T, HeapAlloc> {
    fn make_mut(&mut self) -> &mut T { 
         // use HeapAlloc to potentially clone T
    }
}
impl<T, A: Alloc> Rc<T, A> {
    fn make_mut_in(&mut self, a: A) -> &mut T { 
         // use A to potentially clone T
    }
}

This would keep Rc as thin as possible, not break existing code and allow to use make_mut_in for code that is generic over Alloc

edit: The case for only dealloc: Box, and many non-std types that only provide a minimal api.
edit 2: completely wrong.

[rkruppe]

@s3bk make_mut_in makes HeapAlloc a special case for no good reason, granting it better ergonomics than other allocators that can support make_mut just as well.

edit: The case for only dealloc: Box, and many non-std types that only provide a minimal api.

Box::clone allocates. Do you have examples of such non-std types?

[ghost]

@rkruppe It looks like I was completely wrong here. Most cases are covered by Box and the others reallocate.
Not sure how the the ergonomics could be solved, apart from adding another trait.

[joshlf]

@s3bk

Most cases are covered by Box and the others reallocate.

Is that a problem? The key property of DeallocInfo is the ability to, given a pointer allocated from the allocator, locate the allocator itself. Thus, you should be able to have any method that takes an already-allocated pointer have a _with variant - realloc, shrink, etc.

[joshlf]

@s3bk

There's another angle to consider here: how do we obtain a DeallocInfo? Presumably we need to guarantee that an object of that type has the same lifetime as the Alloc it was obtained from, so how do we ensure that? I'm imagining something like fn dealloc_info(&mut self) -> &mut Self::DeallocInfo, but then the lifetimes get pretty hairy. In particular, once somebody has a &mut DeallocInfo, the allocator can't be used again until the reference goes away because it's a mutable reference. This means you can allocate one Box at a time from an allocator...

[ghost]

@joshlf The lifetime problem is indeed tricky…

What if a lifetime would be added to the Alloc trait?

trait Alloc<'a> {
    type DeallocInfo;
    fn dealloc_info(&self) -> Self::DeallocInfo;
    ...
}
impl<'a> Alloc<'a> for &'a MyAllocator {
    type DeallocInfo = MyDeallocInfo<'a>;
    …
}

[joshlf]

@s3bk that's an interesting approach, but doesn't doing impl<'a> Alloc<'a> for &'a MyAllocator require all allocators to be threadsafe? I'm writing an allocator right now that very much benefits from not needing to be threadsafe.

[ghost]

@joshlf this crude example should not be Send or Sync. So it could safely be non-theadsafe .
Or am I missing something?

[joshlf]

@s3bk Ah I think you're right about that. Good call.

[Ericson2314]

Not quite sure where the conversation is now, but we should probably discuss

type SomeCollection<T> = collections::SomeCollection<T, GlobalAlloc>;

As a temporary way to get the ball rolling here without impacting breaking std, needing newtypes or language design work for default params with aliases.

(Picking up the discussion from #43112 (comment).)

[Ericson2314]

Cross post #32838 (comment)

I've started generalizing the collections myself; the easiest way to demonstrate how the associated return type helps is to just do the code change, I suppose.

[alexreg]

So, as far as I can tell, the real issue here is the one with knowing which allocator to use for further allocations or deallocation using a type. In the case of a single instance of each Alloc type, this can be solved all statically (no memory or runtime overhead), as mentioned. For multiple instances of an allocator type, there needs to be a (mutable) reference in each value to the allocator used. I would thus propose an associated type on Alloc with an AllocRef trait bound. The AllocRef trait would have a constructor method that takes the Alloc instance – for a singleton Alloc type (including the global allocator), the type implementing AllocRef would be zero-sized, and the constructor method wouldn't care about the instance it was passed, whereas it would simple store the reference for multiple-instance allocators.

This would seem to achieve the best of both worlds – no additional cost whatsoever over the current situation when you use the global allocator, and the necessary (but minimum) cost when you use custom allocators.

[SimonSapin]

@alexreg I don’t think there is a need for an AllocRef trait.

What happens at the moment for RawVec is that it has a type parameter A: Alloc and contains a value of type A directly. When the allocator type is zero-size like struct Global; in std::heap, this field has no overhead. For allocators that need to keep per-instance state, the idea is that the trait would be implemented not for the allocator struct, but for references to it: impl<'a> Alloc for &'a MyAllocator, and what you pass to generic collections is such a reference. (In this case with &_ rather than &mut _ so that you can use the same allocator in multiple collections at the same time, the allocator needs to use somthing like Cell or Mutex internally.)

[Ericson2314]

Here's a few PRs on this. Thanks @glandium!

Compiler

1. #50097 preparatory work in rustc to allow extra type params on Box. Take 2; it's actually merged, but only supports ZST.

2. Get non-0 sized types working for Box too. Some of #47043 (upon which the previous PR was based) might be useful. Thankfully no library effort is blocked on this.

Library

1. #50882 convert box to use Alloc trait.
2. #52420 convert other collections to use Alloc trait, and allow fallibility polymorphism

[gnzlbg]

The current API of shrink_in_place is broken.

It returns a Result which might indicate an error, but it cannot actually ever error: the memory block that ptr might point to on success must fit the new size, but this is always the case. First, the pointer does not change (the API is "in place"), so the alignment does not change. Second, the original size is larger than the new desired smaller size, so the original memory block, and every possible block in between, all fit the desired smaller size because they are all larger than it, and have the same alignment.

Note that this means that an implementation of shrink_in_place that actually does nothing beyond just simply returning success is ok. Such an implementation would be very fast, but not very useful. The problem is that not doing anything is actually the right call in some cases (e.g. shrinking a 1Gb allocation by 1 byte), this means that if an implementation were to error to indicate that an allocation cannot be shrunk at all, the user would then need to differentiate between whether this is because shrink_in_place is not supported by the allocator, or because for the size requested not shrinking the allocation is the right call.

So how do we fix this. Well, first I thought about passing a size range, that is, a desired size, and a larger size, so that if the allocation cannot be shrunk in place to be in that range, we can error.

But this is too complicated: shrink_in_place should always succeed and return the Excess. This allows the user to check, whether the new size of the allocation is small enough, or not, and do something else.

Also, while the API of grow_in_place is fine, if we fix this by going the "return Excess" route, I find the arguments for splitting realloc_in_place into grow/shrink_in_place might be worth revisiting. Note that for grow_in_place we can return a meaningful error if we cannot grow the allocation enough, but we can also grow the allocation beyond the desired size, and currently the user would then need to use usable_size to obtain that size, and potentially, we would need to add a grow_in_place_excess method.

At this point, I think I would just prefer a realloc_in_place method that always returns Excess, so that the user can shrink or grow their allocation, and just check the Excess to reuse the extra space, or to see if the allocation is small or big enough. This basically trims 4 methods (shrink/grow_in_place/_excess), two of which their API is currently broken, into 1 method with a functioning API.

EDIT: we could also make this method return a Result, but if the user is shrinking the allocation that will be meaningless. The alternative would be just two methods:

• shrink_in_place(...) -> Excess
• grow_in_place(...) -> Result<Excess, ...>

Most ABIs support returning at least two scalar values in registers, so depending on how many registers it takes to return Result<Excess,...>, this might be another pro for realloc_in_place(...) -> Excess.

[Ericson2314]

I just rebased my old branch making a WIP PR #52420 for this.

[fitzgen]

Hello!

I've been working on a bump allocator and my use cases have required me to fork a couple (and eventually I'll probably end up forking more) std collection types to get them to work with my bump allocator. My hope is that once this issue is resolved and stabilized, I can delete my forked versions and switch back to the std ones. I have a few notes on my experience of porting these types to use my bump allocator that I think also generalize pretty well to the problem of adding support for generic allocators to the std collections.

Collections I have ported thus far:

• Vec: https://github.com/fitzgen/bumpalo/blob/master/src/collections/vec.rs
• String: https://github.com/fitzgen/bumpalo/blob/master/src/collections/string.rs

General notes:

• A bunch of trait implementations aren't feasible for custom allocators, like Default and FromIterator, because the trait's signature doesn't provide access to the allocator. In these cases, it seems fine to just have the appropriate bounds on the impl block to exclude custom allocators. It does raise the question of whether we want to consider introducing additional traits in the future that also provide an allocator.

• Box::leak will be interesting when the Box's allocator is constrained by lifetimes. Eg, if my custom allocator is &'bump Bump, I had better not be able to leak the box to a &'static reference, only &'bump at most. I haven't ported Box yet, and instead turned String::into_boxed_str into String::into_bump_str where it returns &'bump str. (This was just an oversight on my part, where I thought that this was the "equivalent" for bump allocators, but as long as Box::leak does the right thing, then we don't need into_bump_str. Basically I've just been trying to port collections quickly and need to go back and clean this up...)

• My bump allocator doesn't run any Drop implementations, so I don't really want to run any collection's Drops either (deallocation is a no-op anyways, but I have not verified whether LLVM sees through everything). I realize this isn't applicable to the general custom-allocators case though, so I'm not proposing anything in particular, just musing out loud here.

• Mostly, this porting has actually been really easy, and I expect adding custom allocators to std types themselves should be pretty easy too! The hardest parts have been trying to remove unstable usage, and trying to reuse as much of std as possible -- neither of which is an issue for adding custom allocator support to std itself!

• Finally, I just want to express my desire and excitement for a future where std collections are parameterized by custom allocators! It will be awesome!

I hope these notes are helpful in some small way!

[SimonSapin]

Regarding Default and FromIterator: those can exist iff the allocator type implements Default.

Regarding Box::leak, this is an interesting case. I think the today’s type system might not be able to express this in its full generality as that would require something like "associated lifetimes" in traits (if that even makes sense). The simplest for now would be to only implement it for Box<T, std::alloc::Global>.

Even if "deallocation" of the memory directly owned by Vec is a no-op, you should still run drop_in_place on the (slice of) elements. You could for example have Vec<Box<Foo>, &'bump Bump> and those inner boxes do need dropping to avoid memory leaks.

[Ericson2314]

Yes that is what I did for Default. We can add additional methods like new_in and from_iterator_in; new classes can be made later.

[glandium]

@fitzgen See the allocator_api crate, which has the Alloc trait, Box and RawVec.

[fitzgen]

@SimonSapin

Regarding Box::leak, this is an interesting case. I think the today’s type system might not be able to express this in its full generality as that would require something like "associated lifetimes" in traits (if that even makes sense). The simplest for now would be to only implement it for Box<T, std::alloc::Global>.

I think this signature would work with Rust today (although I have not tried it yet):

impl<T, A: Alloc> Box<T, A> {
    pub fn leak<'a>(self) -> &'a T
    where
        A: 'a,
    { ... }
}

[fitzgen]

@fitzgen See the allocator_api crate, which has the Alloc trait, Box and RawVec.

Filed fitzgen/bumpalo#2

[lachlansneff]

Has there been any progress with stabilizing this?

[Ericson2314]

@glandium just got their PR for Box less blocked. Once that lands I'll rebase my PR for the other collections.

[joshlf]

@Ericson2314 @glandium Link to that PR?

[passchaos]

@joshlf maybe #58457

[joshlf]

@passchaos Looks like it; thanks!

[lachlansneff]

It'd be useful if we could use the traditional collection apis (new, with_capacity, etc) if our allocator type parameter implemented Default.

[SimonSapin]

As long as #27336 is not implemented, that would make some currently-valid programs ambiguous because type inference doesn’t know what allocator to pick.

[lachlansneff]

That's been feature-gated for 3 years. Can it not just be stabilized?

[SimonSapin]

Assuming yea mean #27336, it’s better discussed in the relevant tracking issue. I haven’t been following this one, but from a quick glance at the thread its implementation is not complete yet and it’s not "just" a matter of flipping the stabilization switch.

[Manishearth]

Is there an RFC or design document talking about what the current plans for allocators in stdlib collections are? The allocators RFC leaves this stuff undecided, and this issue seems to be an implementation tracking issue.

[SimonSapin]

At this point I think we need a champion who would summarize what’s still unresolved in the (numerous) discussions so far and see, try to build consensus, and make set of proposals (possibly as a new RFC).

Note that I am not volunteering at this time :]

[gnzlbg]

Such a champion might probably want to start a working group, similar to the unsafe-code-guidelines, with its own github repo, where multiple parts of the "Allocators issue" can be explored in parallel, with active meetings to make progress, and where the parts of the issue that achieve consensus get merged slowly into a document with rationale, which is then sliced into suitable RFCs.

Summarizing the discussions effectively and putting them into a single comment would be a lot of work, and it won't IMO help much, because 10 people are going to answer to the summary, asking questions and discussing 10 different aspects of it.

The champion of such a working group would probably need to be a core team or lib team member with the time and will to see this through, and that will probably require somehow fitting allocators in the 2019 roadmap.

[SimonSapin]

Regarding the 2019 roadmap, allocators are already the one item specifically mention for the library team in rust-lang/rfcs#2657. Though to some extent this might be wishful thinking as long as no-one steps up to lead this work.

I feel that this is a much smaller topic than unsafe code guidelines, but on further thought you’re probably right that this is a better model to keep sub-discussions organized.

[Ericson2314]

Once Box<T, A> has landed, I'll finish off https://github.com/QuiltOS/rust/commits/allocator-error / #52420 . I'd be happy to make an RFC at that point describing what I did.