Commit ff64aee
committed
[JSC] Introduce StrongBlock
https://bugs.webkit.org/show_bug.cgi?id=320824
rdar://183834334
Reviewed by Keith Miller.
This patch replaces existing HandleBlock / HandleSet with new StrongBlock and
StrongSet. The mechanism is significantly more efficient in terms of memory
and faster performance.
The purpose of StrongBlock and StrongSet are offering Strong<> root slots.
Unlike MarkedBlock and WeakBlock, StrongBlock's Strong<>'s Slot can be
deallocated synchronously at any point, while MarkedBlock and WeakBlock's slots
become dead only when GC runs. This means that StrongBlock and StrongSet
should follow to the design like high-performance malloc/free. Based on
that, our new design is based on segregated page implementation in libpas.
Each StrongBlock has control header and slots. Initially it starts with
bump allocation mode, so StrongSet maintains bump cursor and allocate slots.
It is possible that some slots get deallocated while using this
StrongBlock, so deallocated slots are constructing intrusive freelist:
each slot is pointing at the previous freelist's head. And StrongSet
first allocate a slot from this freelist, and then continue bump allocation.
When all slots become deallocated while StrongSet is using this
StrongBlock, then we reset the bump cursor to use this block from the beginning.
StrongSet retires the currently used block when freelist and bump cursor
both exhausted. This means that StrongBlock is full and all slots are active.
Then, StrongSet retires this block and use the next block.
Once retired block will eventually get deallocated slots. Each
deallocation constructs a freelist inside a StrongBlock, and StrongBlock
is monitoring the ratio of deallocated / active. And when it crossed the
threshold, we move this StrongBlock state "available" for the reuse.
StrongSet will eventually look for a new block for allocation and find
it from available linked-list.
Once StrongBlock becomes available, we reduce the threshold further and
still attempt to detect when StrongBlock becomes fully empty. Then we
destroy the freelist as now we can use bump allocation for this block.
And we remove it from available linked-list and move it to one spare block slot.
If we already have a spare block (which guarantees it is empty), then we
deallocate the whole StrongBlock to reduce memory footprint. We keep one
empty block as a spare block, and StrongSet will reuse it when available
(freelist mode) blocks are exhausted.
Tests: JSTests/stress/strong-handle-gc.js
Tools/TestWebKitAPI/Tests/JavaScriptCore/StrongBlock.cpp
* JSTests/stress/strong-handle-gc.js: Added.
(shouldBe):
(churn):
* Source/JavaScriptCore/CMakeLists.txt:
* Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj:
* Source/JavaScriptCore/Sources.txt:
* Source/JavaScriptCore/heap/Handle.h:
* Source/JavaScriptCore/heap/HandleBlock.h: Removed.
* Source/JavaScriptCore/heap/HandleBlockInlines.h: Removed.
* Source/JavaScriptCore/heap/HandleSet.cpp: Removed.
* Source/JavaScriptCore/heap/HandleSet.h: Removed.
* Source/JavaScriptCore/heap/Heap.cpp:
(JSC::Heap::Heap):
(JSC::Heap::clearConcurrentRetainedDataIfPossible):
* Source/JavaScriptCore/heap/Heap.h:
* Source/JavaScriptCore/heap/HeapInlines.h:
(JSC::Heap::forEachProtectedCell):
* Source/JavaScriptCore/heap/Strong.h:
* Source/JavaScriptCore/heap/StrongBlock.h: Added.
(JSC::StrongBlock::BlockListNode::block):
(JSC::StrongBlock::strongSet const):
(JSC::StrongBlock::usedCount const):
(JSC::StrongBlock::isEmpty const):
(JSC::StrongBlock::isFull const):
(JSC::StrongBlock::isAvailable const):
(JSC::StrongBlock::isCurrent const):
(JSC::StrongBlock::freeListHead const):
(JSC::StrongBlock::freeNotifyThreshold const):
(JSC::StrongBlock::blocksNode):
(JSC::StrongBlock::availableNode):
(JSC::StrongBlock::setCurrent):
(JSC::StrongBlock::pushFreeSlot):
(JSC::StrongBlock::setFreeNotifyThreshold):
(JSC::StrongBlock::~StrongBlock):
(JSC::StrongBlock::blockContaining):
(JSC::StrongBlock::blockFor):
(JSC::StrongBlock::payload const):
(JSC::StrongBlock::payloadEnd const):
(JSC::StrongBlock::indexOf const):
(JSC::StrongBlock::slotAtIndex const):
(JSC::StrongBlock::encodeFreeListEntry):
(JSC::StrongBlock::decodeFreeListEntry):
(JSC::StrongBlock::setFreeListHead):
(JSC::StrongBlock::incrementUsedCount):
(JSC::StrongBlock::decrementUsedCount):
(JSC::StrongBlock::resetToBumpMode):
* Source/JavaScriptCore/heap/StrongInlines.h:
(JSC::shouldStrongDestructorGrabLock>::Strong):
(JSC::shouldStrongDestructorGrabLock>::set):
(JSC::=):
* Source/JavaScriptCore/heap/StrongSet.cpp: Added.
(JSC::StrongBlock::StrongBlock):
(JSC::StrongBlock::create):
(JSC::StrongSet::StrongSet):
(JSC::StrongSet::~StrongSet):
(JSC::StrongSet::destroyBlock):
(JSC::StrongSet::appendAvailable):
(JSC::StrongSet::takeAvailable):
(JSC::StrongSet::retireCurrentBlock):
(JSC::StrongSet::installCurrentBlock):
(JSC::StrongSet::allocateSlow):
(JSC::StrongSet::didFreeSlot):
(JSC::StrongSet::didBecomeEmpty):
(JSC::StrongSet::visitAggregateImpl):
* Source/JavaScriptCore/heap/StrongSet.h: Added.
(JSC::StrongSet::vm):
(JSC::StrongSet::blockCount const):
(JSC::StrongSet::availabilityCount const):
(JSC::StrongSet::availableBlockCount const):
(JSC::StrongSet::setFor):
(JSC::StrongSet::tryAllocateFromCurrent):
(JSC::StrongSet::allocate):
(JSC::StrongSet::deallocate):
(JSC::StrongSet::deallocateFromCurrentBlock):
(JSC::StrongSet::forEachSlot):
(JSC::StrongSet::forEachLiveCell):
(JSC::StrongSet::forEachStrongHandle):
* Source/WebCore/WebCorePrefix.h:
* Tools/TestWebKitAPI/CMakeLists.txt:
* Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* Tools/TestWebKitAPI/Tests/JavaScriptCore/StrongBlock.cpp: Added.
(TestWebKitAPI::TEST(JavaScriptCore_StrongBlock, Geometry)):
(TestWebKitAPI::TEST(JavaScriptCore_StrongBlock, FreeListEntriesAreTaggedNonCells)):
(TestWebKitAPI::StrongSetTest::vm):
(TestWebKitAPI::StrongSetTest::set):
(TestWebKitAPI::StrongSetTest::fill):
(TestWebKitAPI::StrongSetTest::drain):
(TestWebKitAPI::StrongSetTest::drainScattered):
(TestWebKitAPI::TEST_F(StrongSetTest, Addressing)):
(TestWebKitAPI::TEST_F(StrongSetTest, FirstBlockIsLazy)):
(TestWebKitAPI::TEST_F(StrongSetTest, GrowsAcrossBlocks)):
(TestWebKitAPI::TEST_F(StrongSetTest, FreeWhileStillBumpIsReusedImmediately)):
(TestWebKitAPI::TEST_F(StrongSetTest, FreeListHandsBackEverySlotExactlyOnce)):
(TestWebKitAPI::TEST_F(StrongSetTest, DrainToEmptyRestoresBumpMode)):
(TestWebKitAPI::TEST_F(StrongSetTest, ReclamationHysteresis)):
(TestWebKitAPI::TEST_F(StrongSetTest, NonCurrentEmptyBlockBecomesSpare)):
(TestWebKitAPI::TEST_F(StrongSetTest, CurrentEmptyBeforeSpareLeavesOnlyOneIdleBlock)):
(TestWebKitAPI::TEST_F(StrongSetTest, EmptyBlocksBeyondSpareAreFreed)):
(TestWebKitAPI::TEST_F(StrongSetTest, FreeIntoNonCurrentBlockLeavesCurrentBlockAlone)):
(TestWebKitAPI::TEST_F(StrongSetTest, RetiredFullBlockRejoinsOnlyAtWatermark)):
(TestWebKitAPI::TEST_F(StrongSetTest, ReAdmittedBlockAllocatesOnlyFreedSlots)):
(TestWebKitAPI::TEST_F(StrongSetTest, ReinstalledSpareBumpsFromTheStart)):
(TestWebKitAPI::TEST_F(StrongSetTest, AvailableChainDrainsInAppendOrder)):
(TestWebKitAPI::TEST_F(StrongSetTest, AvailableChainUnlinksBlockThatEmptiesWhileAvailable)):
(TestWebKitAPI::TEST_F(StrongSetTest, ChainIsEmptyAfterLinkedBlocksAreDestroyed)):
(TestWebKitAPI::TEST_F(StrongSetTest, ThrashPatternCausesNoRepeatedEligibilityTransitions)):
(TestWebKitAPI::TEST_F(StrongSetTest, CursorHandoffAcrossBlockSwitchLosesNoSlot)):
(TestWebKitAPI::TEST_F(StrongSetTest, ForEachStrongHandleAcrossBlocks)):
(TestWebKitAPI::TEST_F(StrongSetTest, ForEachStrongHandleSkipsEmptySlots)):
(TestWebKitAPI::TEST_F(StrongSetTest, StrongHandlesSurviveGCAcrossBlocks)):
(TestWebKitAPI::TEST_F(StrongSetTest, SetTransitionsAreMarkedCorrectly)):
(TestWebKitAPI::TEST_F(StrongSetTest, ChurnDoesNotGrowBlockCount)):
Canonical link: https://commits.webkit.org/318435@main1 parent 516a3d0 commit ff64aee
21 files changed
Lines changed: 1640 additions & 476 deletions
File tree
- JSTests/stress
- Source
- JavaScriptCore
- JavaScriptCore.xcodeproj
- heap
- WebCore
- Tools/TestWebKitAPI
- TestWebKitAPI.xcodeproj
- Tests/JavaScriptCore
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
830 | 830 | | |
831 | 831 | | |
832 | 832 | | |
833 | | - | |
834 | 833 | | |
835 | | - | |
836 | 834 | | |
837 | 835 | | |
838 | 836 | | |
| |||
871 | 869 | | |
872 | 870 | | |
873 | 871 | | |
| 872 | + | |
874 | 873 | | |
875 | 874 | | |
| 875 | + | |
876 | 876 | | |
877 | 877 | | |
878 | 878 | | |
| |||
Lines changed: 10 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
715 | 715 | | |
716 | 716 | | |
717 | 717 | | |
718 | | - | |
719 | 718 | | |
| 719 | + | |
| 720 | + | |
720 | 721 | | |
721 | 722 | | |
722 | 723 | | |
| |||
1915 | 1916 | | |
1916 | 1917 | | |
1917 | 1918 | | |
1918 | | - | |
1919 | | - | |
1920 | 1919 | | |
1921 | 1920 | | |
1922 | 1921 | | |
| |||
3883 | 3882 | | |
3884 | 3883 | | |
3885 | 3884 | | |
3886 | | - | |
3887 | | - | |
3888 | 3885 | | |
| 3886 | + | |
| 3887 | + | |
| 3888 | + | |
3889 | 3889 | | |
3890 | 3890 | | |
3891 | 3891 | | |
| |||
5795 | 5795 | | |
5796 | 5796 | | |
5797 | 5797 | | |
5798 | | - | |
5799 | | - | |
5800 | 5798 | | |
5801 | 5799 | | |
5802 | 5800 | | |
| |||
7660 | 7658 | | |
7661 | 7659 | | |
7662 | 7660 | | |
7663 | | - | |
7664 | | - | |
7665 | 7661 | | |
7666 | | - | |
7667 | | - | |
7668 | 7662 | | |
7669 | 7663 | | |
7670 | 7664 | | |
| |||
7755 | 7749 | | |
7756 | 7750 | | |
7757 | 7751 | | |
| 7752 | + | |
7758 | 7753 | | |
7759 | 7754 | | |
| 7755 | + | |
| 7756 | + | |
7760 | 7757 | | |
7761 | 7758 | | |
7762 | 7759 | | |
| |||
11858 | 11855 | | |
11859 | 11856 | | |
11860 | 11857 | | |
11861 | | - | |
11862 | | - | |
11863 | 11858 | | |
11864 | 11859 | | |
11865 | | - | |
11866 | 11860 | | |
11867 | 11861 | | |
11868 | 11862 | | |
| |||
12680 | 12674 | | |
12681 | 12675 | | |
12682 | 12676 | | |
| 12677 | + | |
12683 | 12678 | | |
12684 | 12679 | | |
| 12680 | + | |
12685 | 12681 | | |
12686 | 12682 | | |
12687 | 12683 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
534 | 534 | | |
535 | 535 | | |
536 | 536 | | |
537 | | - | |
538 | 537 | | |
539 | 538 | | |
540 | 539 | | |
| |||
567 | 566 | | |
568 | 567 | | |
569 | 568 | | |
| 569 | + | |
570 | 570 | | |
571 | 571 | | |
572 | 572 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
48 | 47 | | |
49 | 48 | | |
50 | 49 | | |
| |||
126 | 125 | | |
127 | 126 | | |
128 | 127 | | |
129 | | - | |
130 | 128 | | |
131 | 129 | | |
132 | 130 | | |
| |||
This file was deleted.
This file was deleted.
0 commit comments