Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Trigger full GC from CodeManager to fix memory use #2225

Merged
merged 3 commits into from

2 participants

@ryoqun
Collaborator

Just read the commit message. It should be enough to understand the issue.

Also, this pull request contains two other clean-up commits.

Closes #2224.

ryoqun added some commits
@ryoqun ryoqun Sort CompiledCode init code and use LLVM guard fd7693b
@ryoqun ryoqun Fix mis-aligned text 3897b86
@ryoqun ryoqun Trigger full GC from CodeManager to fix memory use
There is a possible situation of excessive memory usage.

It's when many CompiledCode objects are repeatedly created and internalized. In
that case, many MachineCode objects are internally created but never collected.
Thus, memory usage increases forever.

The following test code exhibits the problem:

    prc = proc do
    end

    loop do
      prc.block.instance_variable_set(:@compiled_code,
                                      prc.block.compiled_code.dup)
      prc.call
    end

In such a situation, CompiledCode objects are collected immediately as young
objects by GC. It means that the full (mature) GC is never triggered.

When CompiledCode is internalized, it creates a MachineCode object, a kind of
CodeResource, which is managed and garbage-collected by CodeManager.

Currently, CodeManager only collects unneeded code resources in full (mature)
GC. So, if there is no trigger for the full GC, code resources are never
collected. This is usually fine because code resources are created very
spontaneously under normal conditions.

To fix this, when centain number of code resources (currently, 64 MiB in data
size wise) are newly added, trigger the full GC from CodeManager.

Closes #2224.
dff83c1
@dbussink dbussink merged commit cbe57a7 into rubinius:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 21, 2013
  1. @ryoqun
  2. @ryoqun

    Fix mis-aligned text

    ryoqun authored
Commits on Mar 25, 2013
  1. @ryoqun

    Trigger full GC from CodeManager to fix memory use

    ryoqun authored
    There is a possible situation of excessive memory usage.
    
    It's when many CompiledCode objects are repeatedly created and internalized. In
    that case, many MachineCode objects are internally created but never collected.
    Thus, memory usage increases forever.
    
    The following test code exhibits the problem:
    
        prc = proc do
        end
    
        loop do
          prc.block.instance_variable_set(:@compiled_code,
                                          prc.block.compiled_code.dup)
          prc.call
        end
    
    In such a situation, CompiledCode objects are collected immediately as young
    objects by GC. It means that the full (mature) GC is never triggered.
    
    When CompiledCode is internalized, it creates a MachineCode object, a kind of
    CodeResource, which is managed and garbage-collected by CodeManager.
    
    Currently, CodeManager only collects unneeded code resources in full (mature)
    GC. So, if there is no trigger for the full GC, code resources are never
    collected. This is usually fine because code resources are created very
    spontaneously under normal conditions.
    
    To fix this, when centain number of code resources (currently, 64 MiB in data
    size wise) are newly added, trigger the full GC from CodeManager.
    
    Closes #2224.
This page is out of date. Refresh to see the latest.
View
10 vm/builtin/compiledcode.cpp
@@ -48,11 +48,11 @@ namespace rubinius {
CompiledCode* CompiledCode::create(STATE) {
CompiledCode* code = state->new_object<CompiledCode>(G(compiled_code));
code->local_count(state, Fixnum::from(0));
- code->set_executor(CompiledCode::default_executor);
- code->machine_code_ = NULL;
code->inliners_ = 0;
code->prim_index_ = -1;
+ code->set_executor(CompiledCode::default_executor);
+ code->machine_code_ = NULL;
#ifdef ENABLE_LLVM
code->jit_data_ = NULL;
#endif
@@ -65,8 +65,10 @@ namespace rubinius {
code->copy_object(state, this);
code->set_executor(CompiledCode::default_executor);
- code->jit_data_ = NULL;
code->machine_code_ = NULL;
+#ifdef ENABLE_LLVM
+ code->jit_data_ = NULL;
+#endif
return code;
}
@@ -251,7 +253,9 @@ namespace rubinius {
machine_code_->set_execute_status(MachineCode::eJIT);
+#ifdef ENABLE_LLVM
jit_data_ = rd;
+#endif
machine_code_->unspecialized = exec;
// See if we can also just make this the normal execute
View
4 vm/builtin/compiledcode.hpp
@@ -33,12 +33,14 @@ namespace rubinius {
Tuple* lines_; // slot
Tuple* local_names_; // slot
Symbol* file_; // slot
- ConstantScope* scope_; // slot
+ ConstantScope* scope_; // slot
LookupTable* breakpoints_; // slot
MachineCode* machine_code_;
+#ifdef ENABLE_LLVM
jit::RuntimeDataHolder* jit_data_;
+#endif
public:
// Access directly from assembly, so has to be public.
View
8 vm/gc/code_manager.cpp
@@ -26,6 +26,7 @@ namespace rubinius {
, freed_resources_(0)
, total_allocated_(0)
, total_freed_(0)
+ , gc_triggered_(0)
, bytes_used_(0)
{
first_chunk_ = new Chunk(chunk_size_);
@@ -58,12 +59,17 @@ namespace rubinius {
current_chunk_ = c;
}
- void CodeManager::add_resource(CodeResource* cr) {
+ void CodeManager::add_resource(CodeResource* cr, bool* collect_now) {
utilities::thread::Mutex::LockGuard guard(mutex_);
total_allocated_ += cr->size();
bytes_used_ += cr->size();
+ if(total_allocated_ - gc_triggered_ > cGCTriggerThreshold) {
+ gc_triggered_ = total_allocated_;
+ *collect_now = true;
+ }
+
for(;;) {
while(current_index_ < chunk_size_) {
if(current_chunk_->resources[current_index_] == 0) {
View
4 vm/gc/code_manager.hpp
@@ -25,6 +25,7 @@ namespace rubinius {
class CodeManager {
const static int cDefaultChunkSize = 64;
+ const static int cGCTriggerThreshold = 64 * 1024 * 1024;
/**
* A chunk of memory used to store an array of references to CodeResource
@@ -55,6 +56,7 @@ namespace rubinius {
int freed_resources_;
int total_allocated_;
int total_freed_;
+ int gc_triggered_;
size_t bytes_used_;
@@ -83,7 +85,7 @@ namespace rubinius {
CodeManager(SharedState* shared, int chunk_size=cDefaultChunkSize);
~CodeManager();
- void add_resource(CodeResource* cr);
+ void add_resource(CodeResource* cr, bool* collect_now);
void clear_marks();
void sweep();
int calculate_size();
View
2  vm/objectmemory.cpp
@@ -902,7 +902,7 @@ namespace rubinius {
void ObjectMemory::add_code_resource(CodeResource* cr) {
SYNC_TL;
- code_manager_.add_resource(cr);
+ code_manager_.add_resource(cr, &collect_mature_now);
}
void* ObjectMemory::young_start() {
Something went wrong with that request. Please try again.