Allocators, take III

[pnkfelix]

Update: RFC has been accepted:
text on master: https://github.com/rust-lang/rfcs/blob/master/text/1398-kinds-of-allocators.md
tracking issue: rust-lang/rust#32838

Tasks before FCP

• add discussion of Allocator trait objects, interaction with associated Error type, and whether it matters
• API updates (remove Kind: Copy, rename Kind, fn dealloc return type, ...)
• add discussion of lifetime-tied alternative API(s)
• add discussion of why methods take &mut self (vs self or &self)
• revisit fn oom API design and the associated protocol
• remove associated Error type
• add fn extend_in_place fn realloc_in_place method (returning a Result<(), SeparateUnitError>)
• revisit (again) fn oom API design given that associated Error is now gone
• remove the use of NonZero
• make Layout support zero-sized inputs (delaying all checks, if any, to the allocator itself).

---

• (optional) finish prototyping against collections-prime, e.g. HashMap, Vec (and associated iterators)
  • BinaryHeap
  • BTreeSet
  • BtreeMap
  • Vec (signatures)
  • LinkedList
  • VecDeque
  • HashSet
  • HashMap
  • hash::RawTable (signatures, implementation)
  • alloc::RawVec (signatures, implementation)
  • String (or not...)
  • Box
  • Rc
  • Arc

Summary

Add a standard allocator interface and support for user-defined allocators, with the following goals:

1. Allow libraries (in libstd and elsewhere) to be generic with respect to the particular allocator, to support distinct, stateful, per-container allocators.
2. Require clients to supply metadata (such as block size and alignment) at the allocation and deallocation sites, to ensure hot-paths are as efficient as possible.
3. Provide high-level abstraction over the layout of an object in memory.

Regarding GC: We plan to allow future allocators to integrate themselves with a standardized reflective GC interface, but leave specification of such integration for a later RFC. (The design describes a way to add such a feature in the future while ensuring that clients do not accidentally opt-in and risk unsound behavior.)

rendered

[pnkfelix]

cc @Gankro

[arielb1]

An &'s mut MegaEmbedded would be very dangerous with the aliasing rules. You probably want a &'s UnsafeCell<MegaEmbedded>.

[arielb1]

(By "sane" we mean for example that the input arguments do not cause an arithmetic overflow during computation of the size of the memory block -- if they do, then it is reasonable for an allocator with this error type to respond that insufficent memory was available, rather than e.g. panicking.)

I am quite sure that arithmetic overflow during computation of the input size is an OOM basically by definition.

[arielb1]

This condition
strongly implies that some series of deallocations would
allow a subsequent reissuing of the original allocation
request to succeed.

Not really. If you are running 4GiB of RAM with overcommit/swap disabled and try to malloc all of it, your malloc is going to fail and will not succeed until the system's configuration changes. Of course, allocators SHOULD NOT leak memory on dealloc.

[pnkfelix]

@arielb1 wrote:

I am quite sure that arithmetic overflow during computation of the input size is an OOM basically by definition.

True.

I spent a little while trying to find weasel wording here that would cover zero sized allocations (which are also an Error in this API). I don't remember offhand how each part of the text addressed it, but the the phrasing here is not great.

[pnkfelix]

An &'s mut MegaEmbedded would be very dangerous with the aliasing rules.

Hmm, okay yes I see, the returned blocks alias the embedded array, but LLVM is allowed to assume that only the &mut MegaEmbedded itself accesses the contents of the array.

[pnkfelix]

You probably want a &'s UnsafeCell<MegaEmbedded>

... But this does not seem quite right to me ... this would allow multiple clients to reference the pool, but the point of using &mut was to ensure that there was only one client of the allocator.

Hmm. I am not sure how to resolve this for the example.

[rphmeier]

Really glad that this topic is getting some love. Ironically, I had just started rehashing my allocators crate for the first time in a month, including adding a re-implementation of a few key data structures.

I am slightly doubtful of the necessity for an associated Error type. It seems to me that there are only a few discrete ways an allocator can have an error, and consumers of allocators will intentionally be generic to the point of completely ignoring the associated type completely. It additionally increases the complexity of having any allocators as trait objects.

In the case you describe with DumbBumpPool, I don't completely believe that thread interference is in fact a valid reason to fail an allocation request. It seems more sane to just retry the allocation in a loop until it succeeds or hits a hard error like OOM, since that's what users will basically be doing.

Consider this extremely contrived example.

fn use_alloc<A>(alloc: A) where A: Allocator {
    let my_block = alloc::Kind { size: 1024, align: 8 };
    let my_addr;
    // try the allocation until it works or hits a non-transient error
    loop {
        match alloc.alloc(&my_block) => {
            Ok(addr) => { my_addr = addr; break; }
            Err(e) => {
                if !e.is_transient() {
                    // panic or something
                }
            }
        }
    }
    // use my_addr here
}

I know we're trying to move above and beyond the old-school mechanisms of malloc() and free(), but it's a lot more useful to only receive an error when it's really meaningful. Transient errors seem to just signal to the user to retry an allocation until it works.

[jnicholls]

Huge fan of this concept, I'm actually currently struggling with the fact that I can't use a specific allocator for any of the libstd data structures, which would make my life a lot easier working with shared memory pages...

[bstrie]

Bikeshedding, but is the name "Kind" going to get confusing if we ever get higher-kinded anything?

[gnzlbg]

The typical reasons given for use of custom allocators in C++ are among the following: [...]

The points you mention are important, but the raison d'être of the Allocator concept was dealing with Intel's near and far pointers, although [0] sells this as "supporting different and incompatible memory models".

This RFC explores this tangentially for GC, but I would like to also see some examples for computing devices (like GPGPUs or XeonPhis), for example:

• Given two Vec<T>s, one with memory allocated/deallocated with malloc, and the other one with cudaMalloc (or Intel's TBB scalable_aligned_malloc):
  • How can it be specified that the pointers in the Vec implementation point to different incompatible memory regions and that subtracting these pointers doesn't make sense even in unsafe code?
  • How could Vec implement clone/move (with copying of memory between memory regions), or how could a good error message be emitted at compile-time/run-time if an user tries to do so and is not supported? [1]

[0] From the Alexander Stepanov and Meng Lee, The Standard Template Library, HP Technical Report HPL-95-11(R.1), 1995 (emphasis is mine):

One of the common problems in portability is to be able to encapsulate the information about the memory
model. This information includes the knowledge of **pointer types**, the type of their difference, the type of
the size of objects in this memory model, as well as the **memory allocation and deallocation primitives** for it.
STL addresses this problem by providing a standard set of requirements for allocators, which are objects that
encapsulate this information.

[1] The example in Using an A:Allocator from the client side doesn't attempt this so I guess that since the types of the Vec are different this will just be a compiler error. It would be nice to have a discussion of the pros and cons of trying to make move/clone work within the Allocator framework (which would complicate the whole thing even more) vs going for "the type system forbids it and the users need to deal with this explicitly" route (e.g. having a free function clone_vec_from_alloc_a1_to_a2 that deals with it).

[oli-obk]

It would be nice to have a discussion of the pros and cons of trying to make move/clone work within the Allocator framework (which would complicate the whole thing even more) vs going for "the type system forbids it and the users need to deal with this explicitly" route (e.g. having a free function clone_vec_from_alloc_a1_to_a2 that deals with it).

This function already exists in the form let a2vec: Vec<_, A2> = Vec::from_iter(&a1vec);. But it would be nice to have trait for cloning between allocators.

[TyOverby]

What happens when you drop an allocator that still has memory that is being used?

[Gankro]

@TyOverby You will use-after-free

[oli-obk]

I thought that is prevented by implementing Allocator for references to the actual allocator instead of directly for the allocator type. (which is why Allocator is an unsafe trait)

[Gankro]

You can either have the user own the allocator (so Vec<T, Pool>), or give it a reference to the allocator (Vec<T, &mut Pool>). Either will prevent the user of the Vec from producing a use-after-free (The Vec and Allocator still need to be implemented correctly, of course).

[pnkfelix]

[an associated Error item] additionally increases the complexity of having any allocators as trait objects.

Hmm I will admit that I had not considered this drawback. I'll have to think on it.

[sfackler]

The associated error type seems somewhat similar to when we were considering the same thing for Read and Write. It ended up being way too hard to work with the traits in a generic context so we gave up and stuck with the concrete io::Error.

[Gankro]

Note: discussion on IRC found that RefCell<Allocator> is unsound, because someone can overwrite your allocator with a different one.

let alloc = RefCell::new(Pool::new());
let vec = Vec::with_cap_and_alloc(10, &alloc);
*alloc.get_mut() = Pool::new();
// vec is now using-after-free

Several solutions can be taken to this. Off the top of my head the easiest would be a new-type wrapper over RefCell that doesn't expose &mut A explicitly, only exposing the allocator interface for &AllocatorRefCell.

[petrochenkov]

@pnkfelix

Hmm I will admit that I had not considered this drawback. I'll have to think on it.

Ability to have type erased allocators (i.e. Allocator trait objects in Rust terminology) seems to be a pretty important requirement*, at least they are part of (extended) C++ now.
Motivational paper: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3525.pdf
Final specification: (part of) http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4480.html

^{*I'm not talking from my own experience}

[pnkfelix]

@sfackler

The associated error type seems somewhat similar to when we were considering the same thing for Read and Write

I had originally thought that there would not be much demand for Allocator trait objects, and so I didn't think the analogy with Read/Write was valid.

But @petrochenkov 's recent comment clearly indicates that there may well be demand for Allocator trait objects.

I'm still not entirely convinced... I would be a little disappointed if the only type of allocator error available was the zero-sized MemoryExhausted (or something effectively equivalent to it).

[gnzlbg]

It would be nice to know what exactly happened/is going to happen with
type-erased allocators and the STL2 in C++. IIRC the consensus was that the
default allocator of the std containers should be a type-erased allocator,
so that you can pass containers around through binary APIs independently of
the allocator they use, which is nice.

It seemed that after 20 years of having the allocator in the container
type, the pain is just too big and not worth it since most containers don't
allocate that often, and when they do, they do so in bulk, so that the cost
of a virtual function dispatch becomes negligible when compared to the cost
of malloc.

When you don't want to pay for virtual dispatch, you can always specify a
specific allocator in the container type.

On Mon, Dec 7, 2015 at 11:17 PM, Felix S Klock II notifications@github.com
wrote:

@sfackler https://github.com/sfackler

The associated error type seems somewhat similar to when we were
considering the same thing for Read and Write

I had originally thought that there would not be much demand for Allocator
trait objects, and so I didn't think the analogy with Read/Write was
valid.

But @petrochenkov https://github.com/petrochenkov 's recent comment
#1398 (comment)
clearly indicates that there may well be demand for Allocator trait
objects.

I'm still not entirely convinced... I would be a little disappointed if
the only type of allocator error available was the zero-sized
MemoryExhausted (or something effectively equivalent to it).

—
Reply to this email directly or view it on GitHub
#1398 (comment).

[joshlf]

What's the story here on internal locking? I'm working on an allocator implementation, and I've run across a problem: There's no way for me to express to types that use the Allocator trait that a particular implementation is safe for concurrent access. I want to do some fine-grained locking in the internals of my allocator that should speed things up significantly, but it doesn't seem that there's a way for me to allow consumers (who aren't aware of my implementation in particular, but instead use the Allocator trait) to benefit from it.

[glaebhoerl]

@joshlf An idea from #1974 was to do impl Allocator for &MyAllocator to express that a particular allocator permits shared access. Is this an answer to your question?

[joshlf]

@glaebhoerl

An idea from #1974 was to do impl Allocator for &MyAllocator to express that a particular allocator permits shared access. Is this an answer to your question?

That's a cool way of doing it for a particular concrete type, but is there any way that code that was generic on any Allocator - e.g., DataStructure<T, A: Allocator> - could take advantage of that? I suppose what I'm asking for is some kind of specialization on the basis of whether or not A, in addition to being an Allocator, is also, for example, a SynchronizableAllocator.

Backpressure

Another unrelated idea: It'd be good if there were some way to have backpressure between allocators. For example, if I'm implementing an allocator that provides extra functionality on top of another existing allocator, and my allocator performs caching, it would be useful if the allocator I'm wrapping could inform me if memory was getting tight so I'd know to free some of the caches I was using. One option off the top of my head would be to allow registering "low memory" callbacks that an allocator can invoke to poke downstream allocators to try freeing any memory if they can.

A good example of this is in Section 3.4 of this paper.

[Ericson2314]

@joshlf Allocator + Copy would effectively require implementstions of that sort, because the only way the allocator can be Copy if the allocated things need not be is &-indirection.

[joshlf]

@Ericson2314

But there's no way to specialize, right? No way to make it so that Allocator gets one implementation and Allocator + Copy gets a different implementation?

[Ericson2314]

@joshlf I'm not too familiar with the trait specialization stuff, it might be possible. My hunch is taking advantage of it would entail a vastly different algorithm, but try it out!

[joshlf]

@Ericson2314 Unfortunately I think it's going to be impossible soon thanks to issue 36889. Here's a short example: https://is.gd/xgT6cG

[Ericson2314]

Make a trait that just you implement?

[joshlf]

I don't follow - how does that solve this?

[Ericson2314]

rust-lang/#36889 only applies to inherent impls, not trait impls.

[joshlf]

Hmmm interesting. Seeing as the inherent impl variant is going away, maybe the trait impl variant will soon too? Or is there a good reason to keep the trait impl variant around that doesn't apply to inherent impls?

[joshlf]

Maybe I'm missing something, but it looks like that doesn't work either: https://is.gd/YdiPhl

[SimonSapin]

From the appendix:

   // == ALLOCATOR-SPECIFIC QUANTITIES AND LIMITS ==
   // usable_size

   /// Returns bounds on the guaranteed usable size of a successful
   /// allocation created with the specified `layout`.
   ///
   /// In particular, for a given layout `k`, if `usable_size(k)` returns
   /// `(l, m)`, then one can use a block of layout `k` as if it has any
   /// size in the range `[l, m]` (inclusive).
   ///
   /// (All implementors of `fn usable_size` must ensure that
   /// `l <= k.size() <= m`)

@pnkfelix, is that last equation right? An allocator can return less memory than requested? Or should the equation be k.size() <= l <= m?

[pnkfelix]

@SimonSapin no, an allocator cannot return less memory than requested.

The significance of l here is that a user who has allocated a block via layout k where k.usable_size() == (l, m) is not allowed to use a layout of size < l when they eventually deallocate the block.

[SimonSapin]

@pnkfelix I see, thanks. I think it would be worth expanding the doc-comment of Alloc::usable_size to mention the case of using a different layout for alloc and dealloc, perhaps with an example.