ARC: shard the weak-reference table#401
Draft
DTW-Thalion wants to merge 5 commits into
Draft
Conversation
Per issue gnustep#398. The reference-count fast paths in arc.mm seeded their CAS loops with __sync_fetch_and_add(refCount, 0), a full read-modify-write used only to read the count word, and updated it with sequentially consistent __sync_val_compare_and_swap. The seed only needs a plain load, and the exchanges do not need seq_cst. View the count word through std::atomic<uintptr_t> and operate on it directly. A relaxed load seeds each loop; compare_exchange_weak is acquire-release on the retain and weak-flag paths and release on the decrement; the final release takes an acquire fence before running -dealloc. No functional or ABI change. Isolating just the atomic pattern with no Objective-C involved, on a 32-core x86-64 host with clang 18.1.3: __sync seed + CAS, seq_cst 27.6 ns relaxed load seed + acq/rel CAS 14.4 ns and a retain/release pair on a real object drops from about 31 ns to about 16 ns. All 194 tests pass, along with an 8-thread retain/release balance stress where the count returns to its exact starting value and an 8-thread weak load/store/dealloc race stress.
Every weak operation (objc_storeWeak, objc_loadWeakRetained, objc_delete_weak_refs and friends) keyed off a single global weak-table lock, so weak traffic on unrelated objects serialised on one mutex and scaled negatively with core count. Split the table into address-keyed stripes, each with its own lock, so weak operations on different objects proceed in parallel. Control blocks are type-stable per stripe, so the owning stripe can be read without holding a lock, and the shards are cache-line aligned to avoid false sharing between them.
The number of weak-table stripes now defaults to OBJC_WEAK_SHARD_COUNT (64) and can be overridden at build time with -DOBJC_WEAK_SHARD_COUNT, or per process with the OBJC_WEAK_SHARD_COUNT environment variable, rounded down to a power of two no greater than OBJC_WEAK_SHARD_MAX (256). The stripe array is reserved at OBJC_WEAK_SHARD_MAX and only the active count is used, so run-time tuning needs no allocation. This lets a deployment trade a little memory for more weak-reference throughput without recompiling.
The slot-based weak operations chose which shard to lock from a lock-free peek of the slot, then re-read the slot under the lock. A concurrent objc_storeWeak could repoint the slot to a block in a different shard in that window, leaving the operation holding the wrong shard's lock while it dereferenced the block, whose object another thread's objc_delete_weak_refs (holding the right shard) was then free to deallocate. The result was a use-after-free, caught by AddressSanitizer under the weak load/store/ dealloc stress and reproducible about 90% of the time with more than one shard (a single shard never hits it). Re-check under the lock that the slot still resolves to the peeked block, and retry if it moved, in objc_loadWeakRetained, objc_storeWeak, objc_copyWeak, objc_moveWeak and objc_destroyWeak. objc_initWeak is unaffected: it locks the shard of the caller-owned object, not a slot.
The sharded weak table protects each control block with its shard's lock, but a weak slot itself is read and written under different shard locks (whichever shard owns the block it points at), so no single lock serialises the slot. The lock-free peeks that pick a shard therefore raced the slot writes: benign in practice on strongly-ordered hardware (the peek is re-validated under the lock), but a formal data race, and unsound on weakly-ordered targets where a peek could observe a freshly published block pointer before the block's construction and mistake the control block for the referent. Access weak slots atomically: an acquire load pairs with a release store, so observing a block pointer implies observing its construction, and read the block's isa atomically too (a recycle re-publishes it on another thread). This is the discipline Apple's objc4 uses for weak slots. Acquire/release/relaxed atomics on a pointer word compile to plain loads and stores on x86-64 and AArch64, so there is no measurable cost. Confirmed with ThreadSanitizer: the weak operations now introduce no data races beyond those already present on the unsharded runtime.
Member
|
This looks more invasive than it should be. The original refactoring to make the type used for a weak ref table distinct was intended to make it easy to provide a replacement that did the sharding internally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #399 and depends on it, so the diff currently includes #399's atomic-refcount commit; that drops out once #399 merges.
The weak-reference table used a single global lock, so weak operations on unrelated objects serialised on one mutex and scaled negatively with core count. This splits the table into address-keyed stripes, each with its own lock, so weak traffic on different objects proceeds in parallel.
Follow-up commits make the stripe count configurable (a build flag or the OBJC_WEAK_SHARD_COUNT environment variable), close a lock-selection race the sharding introduced (a use-after-free found under AddressSanitizer, reproducible with more than one shard), and make weak-slot access atomic so the sharding is also sound on weakly-ordered targets (checked under ThreadSanitizer).
Opening as a draft while #399 is in review.