Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

XXX increase gem memory for now.

  • Loading branch information...
commit f1619b72d99e1a2d2a41e6fe2eaf32353462348c 1 parent 68f4d0c
@krono krono authored
Showing with 2 additions and 2 deletions.
  1. +2 −2 etc/system.conf
View
4 etc/system.conf
@@ -134,6 +134,6 @@ STN_TRAN_LOG_SIZES = 100, 100;
GEM_MAX_SMALLTALK_STACK_DEPTH = 15000;
GEM_TEMPOBJ_SCOPES_SIZE = 20000;
-GEM_TEMPOBJ_CACHE_SIZE = 200000;
-SHR_PAGE_CACHE_SIZE_KB = 200000;
+GEM_TEMPOBJ_CACHE_SIZE = 500000;
@jc00ke Collaborator
jc00ke added a note

I had to bump GEM_TEMPOBJ_CACHE_SIZE to 600000 in order to get this little stress test to run without hitting OutOfMemory

I had to bump GEM_TEMPOBJ_CACHE_SIZE to 800000 to get this benchmark to run without hitting OutOfMemory

@Monty Collaborator
Monty added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+SHR_PAGE_CACHE_SIZE_KB = 1000000;
KEYFILE = $MAGLEV_HOME/etc/maglev.demo.key;

4 comments on commit f1619b7

@jc00ke
Collaborator

I had to bump GEM_TEMPOBJ_CACHE_SIZE to 600000 in order to get this little stress test to run without hitting OutOfMemory

I had to bump GEM_TEMPOBJ_CACHE_SIZE to 800000 to get this benchmark to run without hitting OutOfMemory

@krono
Collaborator

I don't know the sensible default, I just increased the size for the (later introduced) hpi branch.

@jc00ke
Collaborator

I'm glad you did! I happened to see this commit when looking through the history, so I knew what to change. Thank you!

@krono
Collaborator

You’re welcome.

@Monty
Collaborator
@Monty
Collaborator

Hi Jesse,

Here are some thoughts on GEM_TEMPOBJ_CACHE_SIZE and benchmarks.

I see that Tobias set the default TOC to 100MB. I don't have a problem with that.

Dale uses 50MB temp obj cache on his production apps, but YMMV depending on the app. It's a footprint tradeoff with the number of simultaneous gems you want to run, as unlike the shared page cache memory, each gem will add the default temp obj cache size to its private memory. (The two have different purposes, though, so your problem would not have been resolved by increasing the SPC size.)

If you are running hundreds of gems, too large a temp obj cache might not be the greatest idea. In many cases, the solution is simply to commit more often, say after 1 million rather than 10 million objects. Or in your case you could use the --tocsize to increase it just for the gem running the benchmark. I was talking to a customer recently who bought a machine with 2TB RAM so they could run a very large number of users and still have a 1TB shared page cache. (Hope they don't ever have to do a core dump ;-)

Speaking of benchmarks, I played around with your benchmark to see what times various rubies took, and to make sure --tocsize was a workable solution for MagLev. It would be more interesting if objects were committed to a database, especially complex objects -- even though I'm pretty much down on synthetic benchmarks these days.

These are on my iMac at home.

Note: filter out "native" so we're not annoyed by these warnings in MagLev - Mac OS X ONLY
native code DISABLED, offset from code memory to libgcilnk.so exceeds 32bits
native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:27766017, ipOffset 56, native code disabled for remainder of session

monty: $ maglev-ruby --tocsz 800000 Jesse_create.rb | grep -v native
      user     system      total        real
just create objects  1.740000   0.050000   1.790000 (  1.788838)
append objects  3.730000   0.260000   3.990000 (  3.986524)
append fixnum  1.590000   0.070000   1.660000 (  1.663824)

JRuby provided the most interesting experiment. Maybe you can duplicate. When I used -J-Xmx400m it took minutes and I finally had to kill it. However, -J-Xmx600m worked speedily.

monty: $ rvm use jruby
Using /Users/monty/MagLev/rvm/gems/jruby-1.6.7.2
monty: $ ruby -v -J-Xmx600m Jesse_create.rb 
jruby 1.6.7.2 (ruby-1.8.7-p357) (2012-05-01 26e08ba) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_37) [darwin-x86_64-java]
      user     system      total        real
just create objects  0.931000   0.000000   0.931000 (  0.905000)
append objects  5.214000   0.000000   5.214000 (  5.214000)
append fixnum  0.753000   0.000000   0.753000 (  0.753000)

monty: $ rvm use 1.9.3
Using /Users/monty/MagLev/rvm/gems/ruby-1.9.3-p194
monty: $ ruby -v Jesse_create.rb 
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin12.0.0]
       user     system      total        real
just create objects  3.010000   0.000000   3.010000 (  3.002260)
append objects  3.560000   0.170000   3.730000 (  4.052806)
append fixnum  0.660000   0.040000   0.700000 (  0.835421)

# This is a pretty old Rubinius build
monty: $ rvm use rbx-2.0.testing
Using /Users/monty/MagLev/rvm/gems/rbx-2.0.testing
monty: $ ruby -v Jesse_create.rb 
rubinius 2.0.0dev (1.8.7 163857dd yyyy-mm-dd JI) [x86_64-apple-darwin12.0.0]
      user     system      total        real
just create objects  0.740755   0.002370   0.743125 (  0.759318)
append objects 12.210643   0.241345  12.451988 ( 12.437300)
append fixnum  0.668290   0.063092   0.731382 (  0.725233)

These are on my Linux desktop at work.

monty@congo:Tests $ maglev-ruby --tocsz 800000 Jesse_create.rb 
      user     system      total        real
just create objects  2.110000   0.000000   2.110000 (  2.111728)
append objects  6.500000   0.260000   6.760000 (  6.779151)
append fixnum  2.200000   0.030000   2.230000 (  2.233112)

monty@congo:Tests $ rvm use jruby
Using /congo1/users/monty/MagLev/rvm/gems/jruby-1.7.0
monty@congo:Tests $ ruby -v -J-Xmx800m Jesse_create.rb 
jruby 1.7.0 (1.9.3p203) 2012-10-22 ff1ebbe on Java HotSpot(TM) 64-Bit Server VM 1.6.0_24-b07 [linux-amd64]
       user     system      total        real
just create objects  1.960000   0.150000   2.110000 (  1.605000)
append objects 26.560000   0.320000  26.880000 ( 14.191000)
append fixnum  2.970000   0.020000   2.990000 (  1.645000)

monty@congo:Tests $ rvm use 1.8.7
Using /congo1/users/monty/MagLev/rvm/gems/ruby-1.8.7-p371
monty@congo:Tests $ ruby -v Jesse_create.rb
ruby 1.8.7 (2012-10-12 patchlevel 371) [x86_64-linux]
      user     system      total        real
just create objects  6.540000   0.000000   6.540000 (  6.549201)
append objects 21.110000   1.430000  22.540000 ( 22.565353)
append fixnum  3.500000   0.070000   3.570000 (  3.576146)

monty@congo:Tests $ rvm use 1.9.3
Using /congo1/users/monty/MagLev/rvm/gems/ruby-1.9.3-p286
monty@congo:Tests $ ruby -v Jesse_create.rb
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-linux]
       user     system      total        real
just create objects 10.060000   0.000000  10.060000 ( 10.067001)
append objects 11.680000   0.260000  11.940000 ( 12.877087)
append fixnum  2.700000   0.050000   2.750000 (  3.131213)

monty@congo:Tests $ rvm use rbx-2.0.testing 
Using /congo1/users/monty/MagLev/rvm/gems/rbx-2.0.testing
monty@congo:Tests $ ruby -v Jesse_create.rb
rubinius 2.0.0dev (1.8.7 163857dd yyyy-mm-dd) [x86_64-unknown-linux-gnu]
      user     system      total        real
just create objects  2.930000   0.000000   2.930000 (  2.937823)
append objects 20.610000   0.470000  21.080000 ( 21.095711)
append fixnum  2.430000   0.190000   2.620000 (  2.622894)

Cheers,
Monty

Please sign in to comment.
Something went wrong with that request. Please try again.