Skip to content

Commit

Permalink
Add GC stats for shady objects in memory
Browse files Browse the repository at this point in the history
When number of shady objects cross shady objects limit, a major GC is
triggered. Knowing the trend of shady objects in memory is useful to
understand why major GCs happen.

Shady objects
=============

When an object survives a GC, it's considered an old object and Ruby
will stop marking it in future minor GCs. Now there is a chance that an
old object may refer to a new object, for example, there is a global
array into which we push a local object. Since the global array is old,
the marking algorithm won't check this array in the next minor GC. This
means the local object which was newly created, but refereneced in the
array will be GCd since the marking algorithm didn't find any reference
to it.

To solve this, Ruby adds a write barrier around each object so Ruby
knows when something in the old generation was touched, and remember to
include it in future marking runs.

But objects created directly from C extensions do not have write
barriers, so these are kind of "shady" in the sense that Ruby has no
guarantee that these objects hold references to other objects.
So Ruby remembers to include shady objects when running the marking
algorithm.
  • Loading branch information
emilsoman committed Oct 16, 2015
1 parent ceb14ed commit 0048b8c
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 2 deletions.
9 changes: 7 additions & 2 deletions lib/rbkit/rbkit_gc.rb
Expand Up @@ -18,13 +18,15 @@ class RbkitGC
# [heap_final_slots] Count of zombie objects
# [heap_swept_slots] Count of slots swept after last GC
# [old_objects] Count of old generation objects
# [old_objects_limit] Old generation object count after which GC is triggered
# [old_objects_limit] Old generation object count after which major GC is triggered
# [total_allocated_objects] Number of created objects in the lifetime of the process
# [total_freed_objects] Number of freed objects in the lifetime of the process
# [malloc_increase_bytes] Malloc'ed bytes since last GC
# [malloc_increase_bytes_limit] Minor GC is triggered when malloc_increase_bytes exceeds this value
# [oldmalloc_increase_bytes] Malloc'ed bytes for old objects since last major GC
# [oldmalloc_increase_bytes_limit] Major GC is triggered with oldmalloc_increase_bytes exceeds this value
# [remembered_wb_unprotected_objects] Count of objects on ruby heap created by C extensions which don't have write protection aka Shady objects
# [remembered_wb_unprotected_objects_limit] Max no of shady objects after which major GC will be triggered
# [total_heap_size] heap_allocated_pages * max slots per page * size of one slot
# [total_memsize] ObjectSpace.memsize_of_all
def self.stat
Expand All @@ -43,7 +45,8 @@ def self.stat
:old_objects, :old_objects_limit, :total_allocated_objects,
:total_freed_objects, :malloc_increase_bytes,
:malloc_increase_bytes_limit, :oldmalloc_increase_bytes,
:oldmalloc_increase_bytes_limit
:oldmalloc_increase_bytes_limit, :remembered_wb_unprotected_objects,
:remembered_wb_unprotected_objects_limit
].each do |key|
stats[key] = data[key]
end
Expand All @@ -65,6 +68,8 @@ def self.stat
stats[:malloc_increase_bytes_limit] = data[:malloc_limit]
stats[:oldmalloc_increase_bytes] = data[:oldmalloc_increase]
stats[:oldmalloc_increase_bytes_limit] = data[:oldmalloc_limit]
stats[:remembered_wb_unprotected_objects] = data[:remembered_shady_object]
stats[:remembered_wb_unprotected_objects_limit] = data[:remembered_shady_object_limit]
end

no_of_allocated_pages = stats[:heap_allocated_pages] rescue 0
Expand Down
2 changes: 2 additions & 0 deletions spec/gc_stat_spec.rb
Expand Up @@ -24,6 +24,8 @@
:malloc_increase_bytes_limit,
:oldmalloc_increase_bytes,
:oldmalloc_increase_bytes_limit,
:remembered_wb_unprotected_objects,
:remembered_wb_unprotected_objects_limit,
:total_heap_size,
:total_memsize ]
end
Expand Down

0 comments on commit 0048b8c

Please sign in to comment.