collections reclaim: emitter ownership for lists, maps, and sets - #239
Merged
Conversation
the string ownership machinery becomes kind-aware: tracked locals carry a kind (string, list, map, set), retains dispatch to the matching runtime operation, and the pre-pass poisons names bound with conflicting kinds in sibling scopes. releases stay string-gated until containers are tagged, so collections currently gain balanced retains and nothing else — behavior parity by construction.
the string ownership discipline generalizes to collection handles: kind-tagged tracking with the same pre-pass, poison rules, and borrowed-by-default classification — except collection literals and collection-typed call results, which arrive owned (a fresh container or a transferred count). containers created with a known element type are tagged so runtime inserts retain, and the last release cascades into elements. two hard-won rules are baked in. tagged containers never release an element before the container itself dies: overwritten and removed elements leak, because an untracked borrow of one may still be live — releasing on overwrite corrupted the compiler's own method registry through exactly that path. and stores of collections into other containers keep an emitter-side compensating retain, because an empty-annotated List[List] is indistinguishable from List[Int] in the type strings and is created untagged. churn-shaped work (a list and map built per loop iteration, 200k iterations) goes from 218 mb peak and unbounded growth to 2.6 mb constant, twice as fast. std_pipeline pays ~10% in rc traffic. compiler fixed point holds; seed refreshed.
kacy
added a commit
that referenced
this pull request
Jul 28, 2026
* release a container's count when an element leaves it a tagged list or map took a count on every heap element it stored and then held it until the container itself died. removing a key, overwriting one, clearing, and the list equivalents dropped the slot and orphaned the value, so anything that evicts in a loop grew without limit: 800k insert-then-remove rounds on a Map[Int, String] peaked at 38 mb against 8 mb for the Map[Int, Int] control. lists behaved the same way on remove, clear, and assignment over an index. the omission was deliberate when it went in (#239), and for a real reason: releasing on overwrite corrupted the emitter's own method registry through an untracked borrow of the element. what has changed since is the emitter, which now retains a borrowed element into every place that outlives the read, so the container can drop the one count it owns and nothing else. overwrites retain the incoming element before releasing what it displaces, so `m[k] = m[k]` cannot free the value mid-assignment. primitive containers are skipped rather than released with a no-op tag, since their storage can be narrower than a handle. map.values() had to be fixed first: it built an untagged list of the map's stored pointers, taking no count at all, so the first eviction after that call freed values the list still pointed at. valgrind reported the invalid read. it now builds a string-tagged list for a heap-valued map, matching what the sort and slice copies already do. the same shape already dangled when the map itself died, so this closes a latent hole rather than one the eviction release opened. sets need no change: they copy element bytes into their own storage instead of holding the caller's handle. * add a regression case for container eviction churns every eviction shape and asserts the high-water mark does not move across a hundred thousand rounds; on the unfixed runtime that half reports "grew 4688kb". the more important half reads a value through a live binding after each kind of eviction, and reads back a values() list after the map is cleared, so a release that fires too early shows up as a use-after-free rather than as a smaller number. registered in the curated memcheck set for that reason.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
arc phase b complete — collections now free. the string ownership discipline generalizes to collection handles:
List[String],List[List],List[Map], string-valued maps — including empty annotated literals and globals), so runtime inserts retain and the last release cascades into elements.ir_reset's= {}in assign position also had to learn the tagged constructors). they leak until escape analysis can prove otherwise; only the container's death releases.List[List]is indistinguishable fromList[Int]in the type strings and is created untagged, so collection-into-container stores keep an emitter-side count. double-counting a tagged container is a bounded leak; a missing count is a dangling element.the numbers
churn-shaped work — a 50-element list and a map built per iteration, 200k iterations (the long-running-server shape):
constant memory where growth was unbounded. std_pipeline pays ~10% in rc traffic (826→904ms) and drops 1.65→1.45 gb peak — its memory is bytes objects and buffers, the next reclamation target. compiler self-compile stays ~2s.
what was tested