DList extremely wasteful in node allocation

[dlangBugzillaToGithub]

alex (@alexrp) reported this on 2012-08-02T14:18:38Z

Transfered from https://issues.dlang.org/show_bug.cgi?id=8500

CC List

• andrei (@andralex)
• issues.dlang (@jmdavis)

Description

The DList container currently allocates a Node struct with the GC instead of managing their memory manually. I don't know if there's anything preventing manual memory management here, but the current approach is extremely wasteful.

[dlangBugzillaToGithub]

issues.dlang (@jmdavis) commented on 2012-08-02T15:38:29Z

I wouldn't expect it to do anything else until custom allocators are implemented. Then the allocator used will determine the allocation scheme used.

[dlangBugzillaToGithub]

andrei (@andralex) commented on 2012-08-02T15:45:27Z

This is following the traditional approach of Java and other languages. Allocators will take care of this.

[dlangBugzillaToGithub]

alex (@alexrp) commented on 2012-08-02T17:57:31Z

> I wouldn't expect it to do anything else until custom allocators are implemented. Then the allocator used will determine the allocation scheme used.

I'd honestly expected it to use malloc/free like Array(T).

> This is following the traditional approach of Java and other languages.
Allocators will take care of this.

Right, I'm just saying that the inefficiency of insertions (which is one of the most common operations next to removal) in DList almost negates any performance gained by using it instead of a plain Array(T) for some use cases (for me, instruction streams in a JIT compiler). For large workloads, it'll induce a lot of GC cycles, scanning, and freeing, which is way worse for throughput than the slightly less efficient insertion and removal algorithms on an Array(T) which at least use malloc/free.

I know allocators will solve this, but I think that a malloc/free approach in the meantime would be reasonable enough (if doable).