Opportunistic GC should give up if the Heap is paged out

https://bugs.webkit.org/show_bug.cgi?id=85411

Reviewed by Filip Pizlo.

Opportunistic GC is punishing us severely in limited memory situations because its
assumptions about how much time a collection will take are way out of whack when the Heap
has been paged out by the OS. We should add a simple detection function to the Heap that
detects if its is paged out. It will do this by iterating each block of both the MarkedSpace
and CopiedSpace. If that operation takes longer than a fixed amount of time (e.g. 100ms),
the function returns true. This function will only be run prior to an opportunistic
collection (i.e. it will not run during our normal allocation-triggered collections).

In my tests, steady state was drastically improved in high memory pressure situations (i.e.
the browser was still usable, significant reduction in SPODs). Occasionally, a normal GC
would be triggered due to pages doing things in the background, which would cause a
significant pause. As we close pages we now cause normal collections rather than full
collections, which prevents us from collecting all of the dead memory immediately. One
nice way to deal with this issue might be to do incremental sweeping.


* heap/CopiedSpace.cpp:
(JSC::isBlockListPagedOut): Helper function to reduce code duplication when iterating over
to-space, from-space, and the oversize blocks.
(JSC):
(JSC::CopiedSpace::isPagedOut): Tries to determine whether or not CopiedSpace is paged out
by iterating all of the blocks.
* heap/CopiedSpace.h:
(CopiedSpace):
* heap/Heap.cpp:
(JSC::Heap::isPagedOut): Tries to determine whether the Heap is paged out by asking the
MarkedSpace and CopiedSpace if they are paged out.
(JSC):
* heap/Heap.h:
(Heap):
(JSC::Heap::increaseLastGCLength): Added this so that the GC timer can linearly back off
each time it determines that the Heap is paged out.
* heap/MarkedAllocator.cpp:
(JSC::MarkedAllocator::isPagedOut): Tries to determine if this particular MarkedAllocator's
list of blocks are paged out.
(JSC):
* heap/MarkedAllocator.h:
(MarkedAllocator):
* heap/MarkedSpace.cpp:
(JSC::MarkedSpace::isPagedOut): For each MarkedAllocator, check to see if they're paged out.
* heap/MarkedSpace.h:
(MarkedSpace):
* runtime/GCActivityCallback.cpp:
(JSC::DefaultGCActivityCallback::cancel):
(JSC):
* runtime/GCActivityCallback.h:
(JSC::GCActivityCallback::cancel):
(DefaultGCActivityCallback):
* runtime/GCActivityCallbackCF.cpp: Added a constant of 100ms for the timeout in determining
whether the Heap is paged out or not.
(JSC):
(JSC::DefaultGCActivityCallbackPlatformData::timerDidFire): Added the check to see if we
should attempt a collection based on whether or not we can iterate the blocks of the Heap in
100ms. If we can't, we cancel the timer and tell the Heap we just wasted 100ms more trying to
do a collection. This gives us a nice linear backoff so we're not constantly re-trying in
steady state paged-out-ness.
(JSC::DefaultGCActivityCallback::cancel): Added this function which, while currently doing
exactly the same thing as willCollect, is more obvious as to what it's doing when we call it
in timerDidFire.


Canonical link: https://commits.webkit.org/102992@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@115915 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/JavaScriptCore/ChangeLog

@@ -1,3 +1,70 @@
+2012-05-02  Mark Hahnenberg  <mhahnenberg@apple.com>
+
+        Opportunistic GC should give up if the Heap is paged out
+        https://bugs.webkit.org/show_bug.cgi?id=85411
+
+        Reviewed by Filip Pizlo.
+
+        Opportunistic GC is punishing us severely in limited memory situations because its 
+        assumptions about how much time a collection will take are way out of whack when the Heap 
+        has been paged out by the OS. We should add a simple detection function to the Heap that 
+        detects if its is paged out. It will do this by iterating each block of both the MarkedSpace 
+        and CopiedSpace. If that operation takes longer than a fixed amount of time (e.g. 100ms), 
+        the function returns true. This function will only be run prior to an opportunistic 
+        collection (i.e. it will not run during our normal allocation-triggered collections).
+
+        In my tests, steady state was drastically improved in high memory pressure situations (i.e. 
+        the browser was still usable, significant reduction in SPODs). Occasionally, a normal GC
+        would be triggered due to pages doing things in the background, which would cause a 
+        significant pause. As we close pages we now cause normal collections rather than full 
+        collections, which prevents us from collecting all of the dead memory immediately. One 
+        nice way to deal with this issue might be to do incremental sweeping.
+
+
+        * heap/CopiedSpace.cpp:
+        (JSC::isBlockListPagedOut): Helper function to reduce code duplication when iterating over 
+        to-space, from-space, and the oversize blocks.
+        (JSC):
+        (JSC::CopiedSpace::isPagedOut): Tries to determine whether or not CopiedSpace is paged out
+        by iterating all of the blocks.
+        * heap/CopiedSpace.h:
+        (CopiedSpace):
+        * heap/Heap.cpp:
+        (JSC::Heap::isPagedOut): Tries to determine whether the Heap is paged out by asking the 
+        MarkedSpace and CopiedSpace if they are paged out.
+        (JSC):
+        * heap/Heap.h:
+        (Heap):
+        (JSC::Heap::increaseLastGCLength): Added this so that the GC timer can linearly back off 
+        each time it determines that the Heap is paged out.
+        * heap/MarkedAllocator.cpp:
+        (JSC::MarkedAllocator::isPagedOut): Tries to determine if this particular MarkedAllocator's
+        list of blocks are paged out.
+        (JSC):
+        * heap/MarkedAllocator.h:
+        (MarkedAllocator):
+        * heap/MarkedSpace.cpp:
+        (JSC::MarkedSpace::isPagedOut): For each MarkedAllocator, check to see if they're paged out.
+        * heap/MarkedSpace.h:
+        (MarkedSpace):
+        * runtime/GCActivityCallback.cpp:
+        (JSC::DefaultGCActivityCallback::cancel):
+        (JSC):
+        * runtime/GCActivityCallback.h:
+        (JSC::GCActivityCallback::cancel):
+        (DefaultGCActivityCallback):
+        * runtime/GCActivityCallbackCF.cpp: Added a constant of 100ms for the timeout in determining
+        whether the Heap is paged out or not.
+        (JSC):
+        (JSC::DefaultGCActivityCallbackPlatformData::timerDidFire): Added the check to see if we 
+        should attempt a collection based on whether or not we can iterate the blocks of the Heap in 
+        100ms. If we can't, we cancel the timer and tell the Heap we just wasted 100ms more trying to 
+        do a collection. This gives us a nice linear backoff so we're not constantly re-trying in
+        steady state paged-out-ness.
+        (JSC::DefaultGCActivityCallback::cancel): Added this function which, while currently doing 
+        exactly the same thing as willCollect, is more obvious as to what it's doing when we call it 
+        in timerDidFire.
+
 2012-05-02  Yong Li  <yoli@rim.com>
 
         Fix GCC X86 build error

Source/JavaScriptCore/heap/CopiedSpace.cpp

@@ -281,4 +281,29 @@ size_t CopiedSpace::capacity()
     return calculatedCapacity;
 }
 
+static bool isBlockListPagedOut(double deadline, DoublyLinkedList<HeapBlock>* list)
+{
+    unsigned itersSinceLastTimeCheck = 0;
+    HeapBlock* current = list->head();
+    while (current) {
+        current = current->next();
+        ++itersSinceLastTimeCheck;
+        if (itersSinceLastTimeCheck >= Heap::s_timeCheckResolution) {
+            double currentTime = WTF::monotonicallyIncreasingTime();
+            if (currentTime > deadline)
+                return true;
+            itersSinceLastTimeCheck = 0;
+        }
+    }
+
+    return false;
+}
+
+bool CopiedSpace::isPagedOut(double deadline)
+{
+    return isBlockListPagedOut(deadline, m_toSpace) 
+        || isBlockListPagedOut(deadline, m_fromSpace) 
+        || isBlockListPagedOut(deadline, &m_oversizeBlocks);
+}
+
 } // namespace JSC

Source/JavaScriptCore/heap/CopiedSpace.h

@@ -70,6 +70,7 @@ class CopiedSpace {
     size_t capacity();
 
     void freeAllBlocks();
+    bool isPagedOut(double deadline);
 
     static CopiedBlock* blockFor(void*);
 

Source/JavaScriptCore/heap/Heap.cpp

@@ -345,6 +345,11 @@ Heap::~Heap()
     ASSERT(!capacity());
 }
 
+bool Heap::isPagedOut(double deadline)
+{
+    return m_objectSpace.isPagedOut(deadline) || m_storageSpace.isPagedOut(deadline);
+}
+
 // The JSGlobalData is being destroyed and the collector will never run again.
 // Run all pending finalizers now because we won't get another chance.
 void Heap::lastChanceToFinalize()

Source/JavaScriptCore/heap/Heap.h

@@ -76,6 +76,12 @@ namespace JSC {
         static Heap* heap(const JSValue); // 0 for immediate values
         static Heap* heap(const JSCell*);
 
+        // This constant determines how many blocks we iterate between checks of our 
+        // deadline when calling Heap::isPagedOut. Decreasing it will cause us to detect 
+        // overstepping our deadline more quickly, while increasing it will cause 
+        // our scan to run faster. 
+        static const unsigned s_timeCheckResolution = 16;
+
         static bool isMarked(const void*);
         static bool testAndSetMarked(const void*);
         static void setMarked(const void*);
@@ -149,11 +155,14 @@ namespace JSC {
         void getConservativeRegisterRoots(HashSet<JSCell*>& roots);
 
         double lastGCLength() { return m_lastGCLength; }
+        void increaseLastGCLength(double amount) { m_lastGCLength += amount; }
 
         JS_EXPORT_PRIVATE void discardAllCompiledCode();
 
         void didAllocate(size_t);
 
+        bool isPagedOut(double deadline);
+
     private:
         friend class CodeBlock;
         friend class LLIntOffsetsExtractor;

Source/JavaScriptCore/heap/MarkedAllocator.cpp

@@ -3,9 +3,28 @@
 
 #include "GCActivityCallback.h"
 #include "Heap.h"
+#include <wtf/CurrentTime.h>
 
 namespace JSC {
 
+bool MarkedAllocator::isPagedOut(double deadline)
+{
+    unsigned itersSinceLastTimeCheck = 0;
+    HeapBlock* block = m_blockList.head();
+    while (block) {
+        block = block->next();
+        ++itersSinceLastTimeCheck;
+        if (itersSinceLastTimeCheck >= Heap::s_timeCheckResolution) {
+            double currentTime = WTF::monotonicallyIncreasingTime();
+            if (currentTime > deadline)
+                return true;
+            itersSinceLastTimeCheck = 0;
+        }
+    }
+
+    return false;
+}
+
 inline void* MarkedAllocator::tryAllocateHelper()
 {
     if (!m_freeList.head) {

Source/JavaScriptCore/heap/MarkedAllocator.h

@@ -32,7 +32,9 @@ class MarkedAllocator {
     void addBlock(MarkedBlock*);
     void removeBlock(MarkedBlock*);
     void init(Heap*, MarkedSpace*, size_t cellSize, bool cellsNeedDestruction);
-
+
+    bool isPagedOut(double deadline);
+
 private:
     friend class LLIntOffsetsExtractor;
 

Source/JavaScriptCore/heap/MarkedSpace.cpp

@@ -70,6 +70,20 @@ void MarkedSpace::canonicalizeCellLivenessData()
     }
 }
 
+bool MarkedSpace::isPagedOut(double deadline)
+{
+    for (size_t cellSize = preciseStep; cellSize <= preciseCutoff; cellSize += preciseStep) {
+        if (allocatorFor(cellSize).isPagedOut(deadline) || destructorAllocatorFor(cellSize).isPagedOut(deadline))
+            return true;
+    }
+
+    for (size_t cellSize = impreciseStep; cellSize <= impreciseCutoff; cellSize += impreciseStep) {
+        if (allocatorFor(cellSize).isPagedOut(deadline) || destructorAllocatorFor(cellSize).isPagedOut(deadline))
+            return true;
+    }
+
+    return false;
+}
 
 void MarkedSpace::freeBlocks(MarkedBlock* head)
 {

Source/JavaScriptCore/heap/MarkedSpace.h

@@ -78,6 +78,8 @@ class MarkedSpace {
     void didAddBlock(MarkedBlock*);
     void didConsumeFreeList(MarkedBlock*);
 
+    bool isPagedOut(double deadline);
+
 private:
     friend class LLIntOffsetsExtractor;
 

Source/JavaScriptCore/runtime/GCActivityCallback.cpp

@@ -54,5 +54,9 @@ void DefaultGCActivityCallback::synchronize()
 {
 }
 
+void DefaultGCActivityCallback::cancel()
+{
+}
+
 }
 

Source/JavaScriptCore/runtime/GCActivityCallback.h

@@ -46,6 +46,7 @@ class GCActivityCallback {
     virtual void didAllocate(size_t) { }
     virtual void willCollect() { }
     virtual void synchronize() { }
+    virtual void cancel() { }
 
 protected:
     GCActivityCallback() {}
@@ -63,6 +64,7 @@ class DefaultGCActivityCallback : public GCActivityCallback {
     virtual void didAllocate(size_t);
     virtual void willCollect();
     virtual void synchronize();
+    virtual void cancel();
 
 #if USE(CF)
 protected:

Source/JavaScriptCore/runtime/GCActivityCallbackCF.cpp

@@ -56,13 +56,22 @@ struct DefaultGCActivityCallbackPlatformData {
 const double gcTimeSlicePerMB = 0.01; // Percentage of CPU time we will spend to reclaim 1 MB
 const double maxGCTimeSlice = 0.05; // The maximum amount of CPU time we want to use for opportunistic timer-triggered collections.
 const double timerSlop = 2.0; // Fudge factor to avoid performance cost of resetting timer.
+const double pagingTimeOut = 0.1; // Time in seconds to allow opportunistic timer to iterate over all blocks to see if the Heap is paged out.
 const CFTimeInterval decade = 60 * 60 * 24 * 365 * 10;
 const CFTimeInterval hour = 60 * 60;
 
 void DefaultGCActivityCallbackPlatformData::timerDidFire(CFRunLoopTimerRef, void *info)
 {
     Heap* heap = static_cast<Heap*>(info);
     APIEntryShim shim(heap->globalData());
+#if !PLATFORM(IOS)
+    double startTime = WTF::monotonicallyIncreasingTime();
+    if (heap->isPagedOut(startTime + pagingTimeOut)) {
+        heap->activityCallback()->cancel();
+        heap->increaseLastGCLength(pagingTimeOut);
+        return;
+    }
+#endif
     heap->collectAllGarbage();
 }
 
@@ -135,4 +144,9 @@ void DefaultGCActivityCallback::synchronize()
     CFRunLoopAddTimer(d->runLoop.get(), d->timer.get(), kCFRunLoopCommonModes);
 }
 
+void DefaultGCActivityCallback::cancel()
+{
+    cancelTimer(d.get());
+}
+
 }