From 0048b8c6dc758e21705b0f16bb0d0d4abeda8d3d Mon Sep 17 00:00:00 2001 From: Emil Soman Date: Sat, 17 Oct 2015 02:40:37 +0700 Subject: [PATCH] Add GC stats for shady objects in memory When number of shady objects cross shady objects limit, a major GC is triggered. Knowing the trend of shady objects in memory is useful to understand why major GCs happen. Shady objects ============= When an object survives a GC, it's considered an old object and Ruby will stop marking it in future minor GCs. Now there is a chance that an old object may refer to a new object, for example, there is a global array into which we push a local object. Since the global array is old, the marking algorithm won't check this array in the next minor GC. This means the local object which was newly created, but refereneced in the array will be GCd since the marking algorithm didn't find any reference to it. To solve this, Ruby adds a write barrier around each object so Ruby knows when something in the old generation was touched, and remember to include it in future marking runs. But objects created directly from C extensions do not have write barriers, so these are kind of "shady" in the sense that Ruby has no guarantee that these objects hold references to other objects. So Ruby remembers to include shady objects when running the marking algorithm. --- lib/rbkit/rbkit_gc.rb | 9 +++++++-- spec/gc_stat_spec.rb | 2 ++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/rbkit/rbkit_gc.rb b/lib/rbkit/rbkit_gc.rb index 98be137..19c3103 100644 --- a/lib/rbkit/rbkit_gc.rb +++ b/lib/rbkit/rbkit_gc.rb @@ -18,13 +18,15 @@ class RbkitGC # [heap_final_slots] Count of zombie objects # [heap_swept_slots] Count of slots swept after last GC # [old_objects] Count of old generation objects - # [old_objects_limit] Old generation object count after which GC is triggered + # [old_objects_limit] Old generation object count after which major GC is triggered # [total_allocated_objects] Number of created objects in the lifetime of the process # [total_freed_objects] Number of freed objects in the lifetime of the process # [malloc_increase_bytes] Malloc'ed bytes since last GC # [malloc_increase_bytes_limit] Minor GC is triggered when malloc_increase_bytes exceeds this value # [oldmalloc_increase_bytes] Malloc'ed bytes for old objects since last major GC # [oldmalloc_increase_bytes_limit] Major GC is triggered with oldmalloc_increase_bytes exceeds this value + # [remembered_wb_unprotected_objects] Count of objects on ruby heap created by C extensions which don't have write protection aka Shady objects + # [remembered_wb_unprotected_objects_limit] Max no of shady objects after which major GC will be triggered # [total_heap_size] heap_allocated_pages * max slots per page * size of one slot # [total_memsize] ObjectSpace.memsize_of_all def self.stat @@ -43,7 +45,8 @@ def self.stat :old_objects, :old_objects_limit, :total_allocated_objects, :total_freed_objects, :malloc_increase_bytes, :malloc_increase_bytes_limit, :oldmalloc_increase_bytes, - :oldmalloc_increase_bytes_limit + :oldmalloc_increase_bytes_limit, :remembered_wb_unprotected_objects, + :remembered_wb_unprotected_objects_limit ].each do |key| stats[key] = data[key] end @@ -65,6 +68,8 @@ def self.stat stats[:malloc_increase_bytes_limit] = data[:malloc_limit] stats[:oldmalloc_increase_bytes] = data[:oldmalloc_increase] stats[:oldmalloc_increase_bytes_limit] = data[:oldmalloc_limit] + stats[:remembered_wb_unprotected_objects] = data[:remembered_shady_object] + stats[:remembered_wb_unprotected_objects_limit] = data[:remembered_shady_object_limit] end no_of_allocated_pages = stats[:heap_allocated_pages] rescue 0 diff --git a/spec/gc_stat_spec.rb b/spec/gc_stat_spec.rb index c7c1bdc..9433672 100644 --- a/spec/gc_stat_spec.rb +++ b/spec/gc_stat_spec.rb @@ -24,6 +24,8 @@ :malloc_increase_bytes_limit, :oldmalloc_increase_bytes, :oldmalloc_increase_bytes_limit, + :remembered_wb_unprotected_objects, + :remembered_wb_unprotected_objects_limit, :total_heap_size, :total_memsize ] end