Walk interface is not suitable for common applications

[gareth-rees]

The MPS offers an object-walking interface via mps_arena_formatted_objects_walk(). But this is unsuitable for certain applications:

1. You can't update references in the object being walked (because there might be a summary on the segment and the summary will need to be updated if the references change). This affects applications like hot reloading of code, where the run-time might need to write all objects belonging to a class (say) so that they point to the new class definitions.

2. The callback is called for every object, so that there is per-object function call overhead.

3. It requires a Walk method for every segment class, that needs to be very similar to the existing Scan and Reclaim methods, but which can't share any code with them.

Here are some sketch requirements and design for a new function solving all these problems:

Requirements

req.walk.all: It must be possible for the client program to visit all automatically managed formatted objects using a callback.

req.walk.assume-format: The callback should not need to switch on the format, as this may be awkward in a program which has modules using different pools with different formats.

req.walk.examine: It must be possible for the callback to examine other automatically managed memory while walking the objects.

req.walk.modify: It must be possible for the callback to modify the references in the objects.

req.walk.overhead: The overhead of calling the callback should be minimized.

req.walk.perf: The performance of subsequent collections should not be affected.

req.walk.closure: The callback must have access to arbitrary data from the caller.

req.walk.maint: The interface should be easy to implement and maintain.

Design

A new public function mps_pool_walk() visits the live formatted objects in an automatically managed pool.

sol.walk.all: The client program must know which pools it has created so it can call mps_pool_walk for each pool.

sol.walk.assume-format: All objects in a pool share the same format, so the callback does not need to switch on the format.

sol.walk.examine: mps_pool_walk() must only be called when the arena is parked, and so there is no read barrier on any object.

sol.walk.modify: mps_pool_walk() arranges for write-protection to be removed from each segment while it is being walked and restored afterwards if necessary.

sol.walk.overhead: The callback is called for contiguous regions of formatted objects (not just for each object) where possible so that the per-object function call overhead is minimized.

sol.walk.perf: The callback uses the scanning protocol so that every reference is fixed and the summary is maintained.

sol.walk.closure: mps_pool_walk() takes a closure pointer which is passed to the callback.

sol.walk.maint: This design reuses the scanning protocol so we can provide a generic implementation that iterates over the ring of segments in the pool. We set up an empty white set in the ScanState so that the MPS_FIX1 test always fails and _mps_fix2 is never called. We can refactor the way that pools call the scan method so that we can pass in a scan method via the scan state. (This would be an equivalent of FormatScan() in the usual scanning case.) There is a small difficulty in the case of LO in that it has formatted objects but its segments don’t have a scan method but we can afford to write one.

Interface

mps_res_t mps_pool_walk(mps_pool_t pool, mps_area_scan_t scan_area, void *closure);

Visit all regions of formatted objects in a pool. The pool must be automatically managed. The pool’s arena must be parked.

pool is the pool whose formatted objects you want to visit.

scan_area is an area scanning function. See Area scanners.

closure is an arbitrary pointer that will be passed to scan_area.