Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upTracking issue for custom allocators in standard collections #42774
Comments
alexcrichton
added
B-unstable
T-libs
labels
Jun 20, 2017
This comment has been minimized.
This comment has been minimized.
|
There are some very interesting issues here:
|
This comment has been minimized.
This comment has been minimized.
I believe that the current design doesn't allow you to parametrize the error, so it must return the concrete error type with two arms - |
This comment has been minimized.
This comment has been minimized.
|
@joshlf err yes, this is predicated on us adding an associated error type. |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 11, 2017
|
To support the allocator trait in the collection types, the constructors will need to be duplicated. impl RawVec<T, A>
fn with_capacity(n: usize) -> RawVec<T, HeapAlloc>;
fn with_capacity_in(n: usize, a: A) -> RawVec<T, A>;
}I personally quite like this approach. |
alexcrichton
referenced this issue
Jul 12, 2017
Closed
Consider reverting the merge of collections into alloc #43112
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 14, 2017
|
@Ericson2314 mentioned this:
I am currently implementing a non-trivial allocator myself (for a relocatable heap) and noticed that it don't need to keep anything around for deallocation (the type information is still needed). This still works when multible allocators are used. I wonder if this could be integrated somehow? Associated types to indicate what needs to be stored? |
This comment has been minimized.
This comment has been minimized.
So the idea is that, for allocators where all instances of the type refer to the same global singleton, you wouldn't actually need to store a reference? Presumably that wouldn't work for allocators that could actually have multiple distinct instances, right? This motivates a follow-up: if you were implementing a global singleton, couldn't you just make the actual |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 15, 2017
Yes, for allocators with multiple instances the allocation function would benefit from non-zero knowledge. |
This comment has been minimized.
This comment has been minimized.
|
@s3bk How common is the case where you will only ever need to deallocate? All the standard collections and smart pointers have methods that allocate new memory from the same allocator (usually Hm, now that I think about it, those methods already need to clone the allocator, so that would potentially offer a way out. Although it's unclear to me if we can switch between storing |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 15, 2017
•
|
@rkruppe there could be trait Alloc {
type DeallocInfo;
fn dealloc(&self, ptr: *mut T, layout: Layout);
fn dealloc_with(info: Self::DeallocInfo, ptr: *mut T, layout: Layout);
// snip
}
struct Rc<T, A: Alloc> {
dealloc: A::DeallocInfo;
// snip
}
impl<T> Rc<T, HeapAlloc> {
fn make_mut(&mut self) -> &mut T {
// use HeapAlloc to potentially clone T
}
}
impl<T, A: Alloc> Rc<T, A> {
fn make_mut_in(&mut self, a: A) -> &mut T {
// use A to potentially clone T
}
}This would keep Rc as thin as possible, not break existing code and allow to use
|
This comment has been minimized.
This comment has been minimized.
|
@s3bk
|
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 15, 2017
|
@rkruppe It looks like I was completely wrong here. Most cases are covered by Box and the others reallocate. |
This comment has been minimized.
This comment has been minimized.
Is that a problem? The key property of |
This comment has been minimized.
This comment has been minimized.
|
There's another angle to consider here: how do we obtain a |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 22, 2017
|
@joshlf The lifetime problem is indeed tricky… What if a lifetime would be added to the Alloc trait? trait Alloc<'a> {
type DeallocInfo;
fn dealloc_info(&self) -> Self::DeallocInfo;
...
}
impl<'a> Alloc<'a> for &'a MyAllocator {
type DeallocInfo = MyDeallocInfo<'a>;
…
} |
Mark-Simulacrum
added
the
C-tracking-issue
label
Jul 22, 2017
This comment has been minimized.
This comment has been minimized.
|
@s3bk that's an interesting approach, but doesn't doing |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Jul 22, 2017
|
@joshlf this crude example should not be Send or Sync. So it could safely be non-theadsafe . |
This comment has been minimized.
This comment has been minimized.
|
@s3bk Ah I think you're right about that. Good call. |
This comment has been minimized.
This comment has been minimized.
|
Not quite sure where the conversation is now, but we should probably discuss type SomeCollection<T> = collections::SomeCollection<T, GlobalAlloc>;As a temporary way to get the ball rolling here without impacting breaking std, needing newtypes or language design work for default params with aliases. (Picking up the discussion from #43112 (comment).) |
This comment has been minimized.
This comment has been minimized.
|
Cross post #32838 (comment) I've started generalizing the collections myself; the easiest way to demonstrate how the associated return type helps is to just do the code change, I suppose. |
matthewjasper
referenced this issue
Apr 3, 2018
Open
Custom global allocators (RFC 1398, RFC 1974) #287
This comment has been minimized.
This comment has been minimized.
|
So, as far as I can tell, the real issue here is the one with knowing which allocator to use for further allocations or deallocation using a type. In the case of a single instance of each This would seem to achieve the best of both worlds – no additional cost whatsoever over the current situation when you use the global allocator, and the necessary (but minimum) cost when you use custom allocators. |
This comment has been minimized.
This comment has been minimized.
|
@alexreg I don’t think there is a need for an What happens at the moment for |
Ericson2314
referenced this issue
Jun 29, 2018
Closed
A pure `alloc` library, with today's `alloc` becoming `global_alloc` #20
This comment has been minimized.
This comment has been minimized.
|
Here's a few PRs on this. Thanks @glandium! Compiler
Library |
This comment has been minimized.
This comment has been minimized.
|
The current API of It returns a Note that this means that an implementation of So how do we fix this. Well, first I thought about passing a size range, that is, a desired size, and a larger size, so that if the allocation cannot be shrunk in place to be in that range, we can error. But this is too complicated: Also, while the API of At this point, I think I would just prefer a EDIT: we could also make this method return a
Most ABIs support returning at least two scalar values in registers, so depending on how many registers it takes to return |
Ericson2314
referenced this issue
Jul 16, 2018
Closed
WIP: Allocator- and fallibility-polymorphic collections #52420
This comment has been minimized.
This comment has been minimized.
|
I just rebased my old branch making a WIP PR #52420 for this. |
Ericson2314
referenced this issue
Aug 2, 2018
Open
Tracking issue for RFC #2116: fallible collection allocation #48043
This comment has been minimized.
This comment has been minimized.
|
Hello! I've been working on a bump allocator and my use cases have required me to fork a couple (and eventually I'll probably end up forking more) Collections I have ported thus far:
General notes:
I hope these notes are helpful in some small way! |
This comment has been minimized.
This comment has been minimized.
|
Regarding Regarding Even if "deallocation" of the memory directly owned by |
This comment has been minimized.
This comment has been minimized.
|
Yes that is what I did for |
This comment has been minimized.
This comment has been minimized.
|
@fitzgen See the |
This comment has been minimized.
This comment has been minimized.
I think this signature would work with Rust today (although I have not tried it yet): impl<T, A: Alloc> Box<T, A> {
pub fn leak<'a>(self) -> &'a T
where
A: 'a,
{ ... }
} |
This comment has been minimized.
This comment has been minimized.
Filed fitzgen/bumpalo#2 |
This comment has been minimized.
This comment has been minimized.
lachlansneff
commented
Feb 16, 2019
|
Has there been any progress with stabilizing this? |
This comment has been minimized.
This comment has been minimized.
|
@glandium just got their PR for |
This comment has been minimized.
This comment has been minimized.
|
@Ericson2314 @glandium Link to that PR? |
This comment has been minimized.
This comment has been minimized.
passchaos
commented
Feb 16, 2019
This comment has been minimized.
This comment has been minimized.
|
@passchaos Looks like it; thanks! |
This comment has been minimized.
This comment has been minimized.
lachlansneff
commented
Feb 16, 2019
|
It'd be useful if we could use the traditional collection apis ( |
This comment has been minimized.
This comment has been minimized.
|
As long as #27336 is not implemented, that would make some currently-valid programs ambiguous because type inference doesn’t know what allocator to pick. |
This comment has been minimized.
This comment has been minimized.
lachlansneff
commented
Feb 17, 2019
•
|
That's been feature-gated for 3 years. Can it not just be stabilized? |
This comment has been minimized.
This comment has been minimized.
|
Assuming yea mean #27336, it’s better discussed in the relevant tracking issue. I haven’t been following this one, but from a quick glance at the thread its implementation is not complete yet and it’s not "just" a matter of flipping the stabilization switch. |
lachlansneff
referenced this issue
Feb 17, 2019
Open
Tracking Issue for RFC 213: Default Type Parameter Fallback #27336
This comment has been minimized.
This comment has been minimized.
|
Is there an RFC or design document talking about what the current plans for allocators in stdlib collections are? The allocators RFC leaves this stuff undecided, and this issue seems to be an implementation tracking issue. |
This comment has been minimized.
This comment has been minimized.
|
At this point I think we need a champion who would summarize what’s still unresolved in the (numerous) discussions so far and see, try to build consensus, and make set of proposals (possibly as a new RFC). Note that I am not volunteering at this time :] |
This comment has been minimized.
This comment has been minimized.
|
Such a champion might probably want to start a working group, similar to the unsafe-code-guidelines, with its own github repo, where multiple parts of the "Allocators issue" can be explored in parallel, with active meetings to make progress, and where the parts of the issue that achieve consensus get merged slowly into a document with rationale, which is then sliced into suitable RFCs. Summarizing the discussions effectively and putting them into a single comment would be a lot of work, and it won't IMO help much, because 10 people are going to answer to the summary, asking questions and discussing 10 different aspects of it. The champion of such a working group would probably need to be a core team or lib team member with the time and will to see this through, and that will probably require somehow fitting allocators in the 2019 roadmap. |
This comment has been minimized.
This comment has been minimized.
|
Regarding the 2019 roadmap, allocators are already the one item specifically mention for the library team in rust-lang/rfcs#2657. Though to some extent this might be wishful thinking as long as no-one steps up to lead this work. I feel that this is a much smaller topic than unsafe code guidelines, but on further thought you’re probably right that this is a better model to keep sub-discussions organized. |
This comment has been minimized.
This comment has been minimized.
|
Once |
alexcrichton commentedJun 20, 2017
The new
Alloctrait defined in RFC 1398 was added in #42313 but the libs teamdecided to hold off onVecintegration just yet to be conservative.This issue is intended to track the integration of the
Alloctrait into standard collections likeVec,BTreeMap, etc. I'm not personally aware of any current blockers, but what we'll probably want to do as part of this issue includes: