FEAT: flexible run size index #377

Closed
pbalcer opened this Issue Dec 22, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@pbalcer

pbalcer commented Dec 22, 2016

Flexible run size index

Rationale

Currently a single run of small memory blocks has a fixed size, equal to the size of a single chunk. This choice was made for several reasons, but most importantly is significantly simplified implementation. The downside of this is that the allocation thread scaling gets worse as the requested memory blocks get bigger. For example, a run with unit size of 32 kilobytes only contains a few memory blocks and gets quickly exhausted. This causes the bucket to request blocks from the global heap and that naturally increases contention. The second issue with the fixed size is that blocks as small as 128 kilobytes need to be rounded up the size of a chunk, and that increases the amount of space that is being wasted.

Description

To combat the above issues, a run allocation class will now be able to specify what's the desired number of allocations per run and that number will be used to calculate the number of required chunks. Initially, the number of allocations per run won't be able to exceed the number of bits in the bitmap, but this limitation might be relaxed with introduction of a customizable bitmap size (which will be done as a separate patch).
Allocation classes will be adjusted to make use of this new feature, and the number of sizes that are handled by runs will be increased - this will improve scaling of large blocks.

API Changes

A new CTL option will be implemented for allocation classes that reflect the number of memory blocks per run. This number will be tunable in runtime.

Implementation details

There are many instances in the code where the assumption of a fixed run size was made.
This change does not break compatibility with the old version of the layout, but the old library won't work with the runs that have size index different than 1.

@pbalcer pbalcer added this to the 1.3 milestone Dec 22, 2016

@pbalcer pbalcer self-assigned this Dec 22, 2016

@pbalcer pbalcer modified the milestones: 1.4, 1.3 Dec 22, 2016

@pbalcer pbalcer referenced this issue in pmem/pmdk Jan 31, 2017

Merged

obj: run dynamic size index #1629

@krzycz

This comment has been minimized.

Show comment
Hide comment
@krzycz

krzycz Feb 1, 2017

Regarding the last sentence - shouldn't we bump up the on-media format version number? Or at least use compat flags to indicate this feature is enabled?

krzycz commented Feb 1, 2017

Regarding the last sentence - shouldn't we bump up the on-media format version number? Or at least use compat flags to indicate this feature is enabled?

@pbalcer

This comment has been minimized.

Show comment
Hide comment
@pbalcer

pbalcer Feb 1, 2017

We should set a compat flag as a step before the next release - yes.

pbalcer commented Feb 1, 2017

We should set a compat flag as a step before the next release - yes.

@pbalcer pbalcer closed this Apr 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment