Improve performance of fiber creation by using pool allocation strategy. #2224

ioquatix · 2019-06-02T11:52:58Z

It would be good to get some feedback on this PR.

$ make benchmark ITEM=fiber COMPARE_RUBY="../build-master/ruby"
Calculating -------------------------------------
                     compare-ruby  built-ruby 
          make_chain       77.840      73.809 i/s -     500.000 times in 6.423414s 6.774286s
        resume_chain      161.783     168.189 i/s -     500.000 times in 3.090556s 2.972852s
  vm2_fiber_allocate      68.114k    112.117k i/s -    200.000k times in 2.936240s 1.783851s
    vm2_fiber_switch       7.812M      8.295M i/s -     10.000M times in 1.280088s 1.205587s

Comparison:
                       make_chain
        compare-ruby:        77.8 i/s 
          built-ruby:        73.8 i/s - 1.05x  slower ** n/c within margin of error

                     resume_chain
          built-ruby:       168.2 i/s 
        compare-ruby:       161.8 i/s - 1.04x  slower ** n/c within margin of error

               vm2_fiber_allocate
          built-ruby:    112117.0 i/s 
        compare-ruby:     68114.3 i/s - 1.65x  slower ** significant improvement

                 vm2_fiber_switch
          built-ruby:   8294714.5 i/s 
        compare-ruby:   7811962.9 i/s - 1.06x  slower ** n/c within margin of error

ioquatix · 2019-06-02T11:53:25Z

@ko1 what do you think?

ioquatix · 2019-06-02T11:54:11Z

I want to test this in real world code too, but I am on holiday I don't have my desktop computer so I can't really test it at the extreme limits.

ioquatix · 2019-06-02T11:56:15Z

Also, I don't expose Fiber::Pool although conceptually it could be exposed pretty easily and I think that would be a good idea.

ioquatix · 2019-06-04T05:58:27Z

@ko1 I get strange error related to refinements on my desktop but not on travis:

% make test-all
../revision.h unchanged
config.status: creating ruby-runner.h
making mjit_build_dir.so
Run options: "--ruby=./miniruby -I../lib -I. -I.ext/common  ../tool/runruby.rb --extout=.ext  -- --disable-gems" --excludes-dir=../test/excludes --name=!/memory_leak/

# Running tests:

Leaked thread: Rinda::TestRingServer#test_ring_server_ipv6_multicast: #<Thread:0x000056040e910740@/home/samuel/Documents/ioquatix/ruby/lib/rinda/tuplespace.rb:617 sleep>
[ 7833/20993] TestFileExhaustive#test_birthtime = 0.07 s                                                                                   
  1) Error:
TestFileExhaustive#test_birthtime:
NotImplementedError: birthtime is unimplemented on this filesystem
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_file_exhaustive.rb:628:in `birthtime'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_file_exhaustive.rb:628:in `block in test_birthtime'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_file_exhaustive.rb:627:in `each'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_file_exhaustive.rb:627:in `test_birthtime'

[ 8996/20993] TestGc#test_latest_gc_info = 0.11 s                                                                             
  2) Failure:
TestGc#test_latest_gc_info [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_gc.rb:135]:
<:force> expected but was
<:oldmalloc>.

Leaked file descriptor: TestGem#test_use_gemdeps_automatic: 8 : #<File:/dev/null>
Leaked file descriptor: TestGem#test_use_gemdeps_automatic: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_pre_uninstall: 8
Closed file descriptor: TestGem#test_self_pre_uninstall: 9
Leaked file descriptor: TestGem#test_use_gemdeps_missing_gem: 8 : #<File:/dev/null>      
Leaked file descriptor: TestGem#test_use_gemdeps_missing_gem: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_try_activate_missing_dep: 8
Closed file descriptor: TestGem#test_self_try_activate_missing_dep: 9
Leaked file descriptor: TestGem#test_auto_activation_of_specific_gemdeps_file: 8 : #<File:/dev/null>
Leaked file descriptor: TestGem#test_auto_activation_of_specific_gemdeps_file: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_ensure_gem_directories: 8
Closed file descriptor: TestGem#test_self_ensure_gem_directories: 9
Leaked file descriptor: TestGem#test_self_use_gemdeps: 8 : #<File:/dev/null>                                          
Leaked file descriptor: TestGem#test_self_use_gemdeps: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_finish_resolve_wtf: 8
Closed file descriptor: TestGem#test_self_finish_resolve_wtf: 9
Leaked file descriptor: TestGem#test_auto_activation_of_used_gemdeps_file: 8 : #<File:/dev/null>
Leaked file descriptor: TestGem#test_auto_activation_of_used_gemdeps_file: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_use_gemdeps_argument_missing: 8
Closed file descriptor: TestGem#test_use_gemdeps_argument_missing: 9
Leaked file descriptor: TestGem#test_use_gemdeps_specific: 8 : #<File:/dev/null>   
Leaked file descriptor: TestGem#test_use_gemdeps_specific: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_sources: 8
Closed file descriptor: TestGem#test_self_sources: 9
Leaked file descriptor: TestGem#test_use_gemdeps: 8 : #<File:/dev/null>          
Leaked file descriptor: TestGem#test_use_gemdeps: 9 : #<File:/dev/null>
Closed file descriptor: TestGem#test_self_path: 8
Closed file descriptor: TestGem#test_self_path: 9
[14216/20993] TestPatternMatching::TestPatternMatchingRefinements#test_refinements = 0.00 s                             
  3) Error:
TestPatternMatching::TestPatternMatchingRefinements#test_refinements:
NoMatchingPatternError: #<TestPatternMatching::TestPatternMatchingRefinements::C1:0x00005604100fefc8>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_pattern_matching.rb:1126:in `block in test_refinements'
    /home/samuel/Documents/ioquatix/ruby/test/lib/test/unit/assertions.rb:51:in `assert_block'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_pattern_matching.rb:1125:in `test_refinements'

[16887/20993] TestRefinement#test_include_into_refinement = 0.00 s                                            
  4) Error:
TestRefinement#test_include_into_refinement:
NoMethodError: undefined method `foo' for #<TestRefinement::IncludeIntoRefinement::C:0x000056040e8fa9e0>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:10:in `invoke_foo_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:802:in `test_include_into_refinement'

[16891/20993] TestRefinement#test_instance_method_should_use_refinements = 0.00 s
  5) Error:
TestRefinement#test_instance_method_should_use_refinements:
NameError: undefined method `z' for class `TestRefinement::Foo'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:122:in `instance_method'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:122:in `instance_method_z'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:251:in `test_instance_method_should_use_refinements'

[16899/20993] TestRefinement#test_method_should_use_refinements = 0.00 s                 
  6) Error:
TestRefinement#test_method_should_use_refinements:
NameError: undefined method `z' for class `TestRefinement::Foo'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:118:in `method'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:118:in `method_z'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:230:in `test_method_should_use_refinements'

[16900/20993] TestRefinement#test_mixed_using = 0.00 s                  
  7) Failure:
TestRefinement#test_mixed_using [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:1742]:
<[:bar, [:R1, :orig_foo]]> expected but was
<[:bar, :orig_foo]>.

[16902/20993] TestRefinement#test_module_inclusion = 0.00 s
  8) Failure:
TestRefinement#test_module_inclusion [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:1045]:
<"FooExt#x"> expected but was
<"Foo#x">.

[16903/20993] TestRefinement#test_module_inclusion2 = 0.00 s
  9) Failure:
TestRefinement#test_module_inclusion2 [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:1069]:
<"FooExt2#x"> expected but was
<"Foo#x">.

[16904/20993] TestRefinement#test_module_using = 0.00 s     
 10) Failure:
TestRefinement#test_module_using [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:995]:
<"FooExt#x"> expected but was
<"Foo#x">.

[16908/20993] TestRefinement#test_new_method = 0.00 s               
 11) Error:
TestRefinement#test_new_method:
NoMethodError: undefined method `z' for #<TestRefinement::Foo:0x000056040e5cdbc8>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:98:in `invoke_z_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:181:in `test_new_method'

[16909/20993] TestRefinement#test_new_method_on_subclass = 0.00 s
 12) Error:
TestRefinement#test_new_method_on_subclass:
NoMethodError: undefined method `z' for #<TestRefinement::FooSub:0x000056040e5c54a0>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:98:in `invoke_z_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:286:in `test_new_method_on_subclass'

[16912/20993] TestRefinement#test_override = 0.00 s              
 13) Failure:
TestRefinement#test_override [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:153]:
<"FooExt#x"> expected but was
<"Foo#x">.

[16917/20993] TestRefinement#test_prepend_into_refinement = 0.00 s                  
 14) Error:
TestRefinement#test_prepend_into_refinement:
NoMethodError: undefined method `foo' for #<TestRefinement::PrependIntoRefinement::C:0x000056040df737c8>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:10:in `invoke_foo_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:865:in `test_prepend_into_refinement'

[16919/20993] TestRefinement#test_public_send_should_use_refinements = 0.00 s
 15) Error:
TestRefinement#test_public_send_should_use_refinements:
NoMethodError: undefined method `z' for #<TestRefinement::Foo:0x000056040df66e88>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:110:in `public_send'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:110:in `public_send_z_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:215:in `test_public_send_should_use_refinements'

[16926/20993] TestRefinement#test_refine_module = 0.00 s                                  
 16) Failure:
TestRefinement#test_refine_module [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:478]:
<"M@M2#foo"> expected but was
<"M#foo">.

[16946/20993] TestRefinement#test_send_should_use_refinements = 0.00 s      
 17) Error:
TestRefinement#test_send_should_use_refinements:
NoMethodError: undefined method `z' for #<TestRefinement::Foo:0x00005603d61e8860>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:102:in `send_z_on'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:205:in `test_send_should_use_refinements'

[16949/20993] TestRefinement#test_super = 0.00 s                                      
 18) Failure:
TestRefinement#test_super [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:160]:
<"FooExt#y Foo#y"> expected but was
<"Foo#y">.

[16953/20993] TestRefinement#test_super_not_chained = 0.00 s        
 19) Failure:
TestRefinement#test_super_not_chained [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:167]:
<"FooExt2#y Foo#y"> expected but was
<"Foo#y">.

[16954/20993] TestRefinement#test_super_to_module = 0.00 s  
 20) Failure:
TestRefinement#test_super_to_module [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:1957]:
[ruby-core:79588] [Bug #13227].
<"Parent -> Child"> expected but was
<"#<TestRefinement::SuperToModule::Child:0x00005603d621cb38>">.

[16955/20993] TestRefinement#test_symbol_proc = 0.00 s    
 21) Failure:
TestRefinement#test_symbol_proc [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:1839]:
<"FooExt#x"> expected but was
<"Foo#x">.

[16957/20993] TestRefinement#test_to_proc = 0.00 s               
 22) Error:
TestRefinement#test_to_proc:
TypeError: wrong argument type TestRefinement::ToProc::ReturnProc (expected Proc)
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:2099:in `call'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:2191:in `test_to_proc'

[16958/20993] TestRefinement#test_tostring = 0.00 s
 23) Failure:
TestRefinement#test_tostring [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:2065]:
<"ok"> expected but was
<"#<TestRefinement::ToString:0x00005603d62bc6b0>">.

[16968/20993] TestRefinement#test_using_same_class_refinements = 0.00 s    
 24) Failure:
TestRefinement#test_using_same_class_refinements [/home/samuel/Documents/ioquatix/ruby/test/ruby/test_refinement.rb:174]:
<"FooExt#a"> expected but was
<"Foo#a">.

[19135/20993] TestSymbol#test_to_proc_arg_with_refinements = 0.00 s                                      
 25) Error:
TestSymbol#test_to_proc_arg_with_refinements:
NoMethodError: undefined method `hoge' for #<TestSymbol::TestToPRocArgWithRefinements:0x00005603d8f01e28>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_symbol.rb:166:in `_test_to_proc_arg_with_refinements_call'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_symbol.rb:176:in `test_to_proc_arg_with_refinements'

[19136/20993] TestSymbol#test_to_proc_arg_with_refinements_override = 0.00 s
 26) Error:
TestSymbol#test_to_proc_arg_with_refinements_override:
NoMethodError: undefined method `hoge' for #<TestSymbol::TestToPRocArgWithRefinements:0x00005603d8f113c8>
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_symbol.rb:166:in `_test_to_proc_arg_with_refinements_call'
    /home/samuel/Documents/ioquatix/ruby/test/ruby/test_symbol.rb:191:in `test_to_proc_arg_with_refinements_override'

Finished tests in 616.142118s, 34.0587 tests/s, 4392.5807 assertions/s.                                        
20985 tests, 2706454 assertions, 13 failures, 13 errors, 36 skips

ruby -v: ruby 2.7.0dev (2019-06-04 fiber-pool cbce6d4f11) [x86_64-linux]
make: *** [uncommon.mk:767: yes-test-all] Error 26

ioquatix · 2019-06-04T06:01:50Z

On 32-bit travis test:

test_thread.rb           ...Fstderr output is not empty
   
   fiber_pool_expand: 0x414660a0, 32 x [131072]
   bootstraptest.tmp.rb: failed to allocate memory (NoMemoryError)
Fstderr output is not empty
   
   fiber_pool_expand: 0x414a80a0, 32 x [131072]
   bootstraptest.tmp.rb: failed to allocate memory (NoMemoryError)
.............................................
#1351 test_thread.rb:28:in `<top (required)>': 
   begin
     :ok if 5000 == 5000.times{|e|
       (1..2).map{
         Thread.new{
         }
       }.each{|e|
         e.join()
       }
     }
   rescue ThreadError => e
     :ok if /can't create Thread/ =~ e.message
   end
  #=> "" (expected "ok")  
#1352 test_thread.rb:42:in `<top (required)>': 
   begin
     :ok if 5000 == 5000.times{|e|
       (1..2).map{
         Thread.new{
         }
       }.each{|e|
         e.join(1000000000)
       }
     }
   rescue ThreadError => e
     :ok if /can't create Thread/ =~ e.message
   end
  #=> "" (expected "ok")  
FAIL 2/1397 tests failed
uncommon.mk:741: recipe for target 'yes-btest-ruby' failed
make: *** [yes-btest-ruby] Error 1
The command "$SETARCH make -s test TESTOPTS="${TESTOPTS=$JOBS -q --tty=no}"" exited with 2.

ioquatix · 2019-06-04T22:03:28Z

@ko1 I managed to catch rb_memerror and it seems something is wrong with rb_thread_recycle_stack. I'm no longer using this function in fiber code, but now thread has got issues.

#0  0x00007fabb3530908 in nanosleep () from /usr/lib/libc.so.6
#1  0x00007fabb353080e in sleep () from /usr/lib/libc.so.6
#2  0x00005571a6d113a0 in rb_memerror () at ../gc.c:9388
#3  0x00005571a6d115a1 in ruby_memerror () at ../gc.c:9366
#4  objspace_xmalloc0 (objspace=0x5571a82e7240, size=<optimized out>) at ../gc.c:9641
#5  0x00005571a6ea9bdd in rb_thread_recycle_stack (size=131072) at ../vm.c:2461
#6  rb_thread_recycle_stack (size=131072) at ../vm.c:2453
#7  th_init (th=th@entry=0x5571a846b6c0, self=self@entry=93946641300080) at ../vm.c:2655
#8  0x00005571a6ec5848 in ruby_thread_init (self=93946641300080) at ../vm.c:2689
#9  rb_thread_alloc (klass=klass@entry=93946641601040) at ../vm.c:2702
#10 0x00005571a6e7360b in thread_s_new (argc=0, argv=0x7fabb30320d0, klass=93946641601040) at ../thread.c:872
#11 0x00005571a6eabe3d in vm_call_cfunc_with_frame (ci=0x5571a8454f60, cc=<optimized out>, calling=<optimized out>, reg_cfp=0x7fabb3131ec0, 
    ec=0x5571a82e76b8) at ../vm_insnhelper.c:2212
#12 vm_call_cfunc (ec=<optimized out>, reg_cfp=<optimized out>, calling=<optimized out>, ci=<optimized out>, cc=<optimized out>)
    at ../vm_insnhelper.c:2230
#13 0x00005571a6ebdada in vm_sendish (method_explorer=<optimized out>, block_handler=<optimized out>, cc=<optimized out>, ci=<optimized out>, 
    reg_cfp=<optimized out>, ec=<optimized out>) at ../vm_insnhelper.c:3628
#14 vm_exec_core (ec=0x7ffd59d78210, initial=140726110749200) at ../insns.def:771
#15 0x00005571a6eb4bac in rb_vm_exec (ec=0x5571a82e76b8, mjit_enable_p=1) at ../vm.c:1890
#16 0x00005571a6eb56c8 in invoke_iseq_block_from_c (me=0x0, is_lambda=-1290592376, cref=<optimized out>, passed_block_handler=<optimized out>, 
    argv=<optimized out>, argc=1, self=93946620106395, captured=0x7fabb3131fb8, ec=0x5571a82e76b8) at ../vm.c:1102
#17 invoke_block_from_c_bh (argc=<optimized out>, passed_block_handler=<optimized out>, cref=<optimized out>, is_lambda=<optimized out>, 
    force_blockarg=<optimized out>, argv=<optimized out>, block_handler=<optimized out>, ec=<optimized out>) at ../vm.c:1120
#18 vm_yield (argc=1, argv=<optimized out>, ec=<optimized out>) at ../vm.c:1165
#19 rb_yield_0 (argv=<optimized out>, argc=1) at ../vm_eval.c:994
#20 catch_i (tag=<optimized out>, data=<optimized out>) at ../vm_eval.c:1915
#21 0x00005571a82fe968 in ?? ()
#22 0x00007fabb3131f88 in ?? ()
#23 0x00005571a6eb5e19 in vm_yield (argv=<optimized out>, argc=<optimized out>, ec=<optimized out>) at ../vm.c:1151
#24 rb_yield_0 (argv=<optimized out>, argc=<optimized out>) at ../vm_eval.c:994
#25 rb_yield_values2 (argc=<optimized out>, argv=<optimized out>) at ../vm_eval.c:1040
#26 0x00005571a6f4e27e in collect_i (i=<optimized out>, ary=8409590266739931136, argc=<optimized out>, argv=<optimized out>, 
    blockarg=<optimized out>) at ../enum.c:539
#27 0x00005571a6eb0f2d in vm_yield_with_cfunc (ec=ec@entry=0x5571a82e76b8, self=93946641377040, argc=77, argc@entry=1, 
    argv=argv@entry=0x7ffd59d787b8, block_handler=0, me=0x0, captured=<optimized out>, captured=<optimized out>) at ../vm_insnhelper.c:2865
#28 0x00005571a6ec430e in invoke_block_from_c_bh (argc=<optimized out>, passed_block_handler=<optimized out>, cref=<optimized out>, 
    is_lambda=<optimized out>, force_blockarg=<optimized out>, argv=<optimized out>, block_handler=<optimized out>, ec=<optimized out>)
    at ../vm_core.h:1433
#29 vm_yield (argc=1, argv=0x7ffd59d787b8, ec=0x5571a82e76b8) at ../vm.c:1165

ioquatix · 2019-06-05T23:35:32Z

@ko1 this is now only blocking on one test failure which I don't know how should be fixed:

  1) Failure:
3729TestGc#test_latest_gc_info [C:/projects/ruby/test/ruby/test_gc.rb:135]:
3730<:force> expected but was
3731<:oldmalloc>.

ioquatix · 2019-07-16T08:30:33Z

Replace previous stack cache with fiber pool cache. The fiber pool allocates many stacks in a single memory region. Stack allocation becomes O(log N) and fiber creation is amortized O(1). Around 10x performance improvement was measured in micro-benchmarks.

On 32-bit platforms, expanding the fiber pool by a large amount may fail, even if a smaller amount may succeed. We limit the maximum size of a single allocation to maximise the number of fibers that can be allocated. Additionally, we implement the book-keeping required to free allocations when their usage falls to zero.

`madvise(free)` and similar operations are good because they avoid swap usage by clearing the dirty bit on memory pages which are mapped but no longer needed. However, there is some performance penalty if there is no memory pressure. Therefore, we do it by default, but it can be avoided.

…pace. We use COROUTINE_LIMITED_ADDRESS_SPACE to select platforms where address space is 32-bits or less. Fiber pool implementation enables more book keeping, and reduces upper limits, in order to minimise address space utilisation.

…ise.

If `mmap` fails to allocate memory, try half the size, and so on. Limit FIBER_POOL_ALLOCATION_MAXIMUM_SIZE to 1024 stacks. In typical configurations this limits the memory mapped region to ~128MB per allocation.

ioquatix · 2019-07-18T08:55:18Z

Okay, it was merged.

ioquatix · 2019-07-18T10:12:32Z

ioquatix force-pushed the fiber-pool branch 5 times, most recently from c340778 to bc8b220 Compare June 4, 2019 04:51

ioquatix force-pushed the fiber-pool branch 4 times, most recently from ea78187 to 3a0928b Compare June 4, 2019 12:00

ioquatix force-pushed the fiber-pool branch from fa8ac15 to aae6524 Compare June 4, 2019 23:42

ioquatix changed the title ~~Fiber pool~~ Improve performance of fiber creation by using pool allocation strategy. Jun 5, 2019

ioquatix force-pushed the fiber-pool branch 6 times, most recently from e9ea7ea to 6d49a46 Compare June 20, 2019 11:37

ioquatix force-pushed the fiber-pool branch 6 times, most recently from 4aea3c8 to eac8077 Compare July 1, 2019 05:11

ioquatix force-pushed the fiber-pool branch 7 times, most recently from 2ecab3e to 88f05f5 Compare July 16, 2019 01:56

ioquatix added 14 commits July 18, 2019 14:50

Make FIBER_USE_NATIVE the default and reformat code.

917037c

Remove unused vm_stack recycling.

962f2ad

Stack copying implementation of coroutines.

4bd2302

Improve build process and coroutine implementation selection.

34234c3

Add ucontext and copy coroutine implementations to test matrix.

ff3bd49

Add details of fiber pool and coroutine selection to NEWS.

61dc2d9

Enable madvise to release stack space back to OS.

159cd2d

Add FIBER_POOL_ALLOCATION_FREE to control allocation/free strategy.

3802708

Add note about setting vm.max_map_count for Linux.

b46d2a3

Add experimental RUBY_SHARED_FIBER_POOL_FREE_STACKS to control madv…

3eb1204

…ise.

ioquatix force-pushed the fiber-pool branch from 3330760 to 3eb1204 Compare July 18, 2019 02:50

Improve fiber_pool_expand allocation strategy.

7ed0e4c

If `mmap` fails to allocate memory, try half the size, and so on. Limit FIBER_POOL_ALLOCATION_MAXIMUM_SIZE to 1024 stacks. In typical configurations this limits the memory mapped region to ~128MB per allocation.

ioquatix force-pushed the fiber-pool branch from 89e9452 to 7ed0e4c Compare July 18, 2019 07:16

ioquatix closed this Jul 18, 2019

eregon mentioned this pull request Dec 30, 2019

Write specs for new Ruby 2.7 features and changes ruby/spec#745

Open

70 tasks

georgie84 mentioned this pull request Nov 13, 2020

Ruby 2.7 Support jruby/jruby#6464

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve performance of fiber creation by using pool allocation strategy. #2224

Improve performance of fiber creation by using pool allocation strategy. #2224

ioquatix commented Jun 2, 2019 •

edited

Loading

ioquatix commented Jun 2, 2019

ioquatix commented Jun 2, 2019

ioquatix commented Jun 2, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 5, 2019

ioquatix commented Jul 16, 2019

ioquatix commented Jul 18, 2019

ioquatix commented Jul 18, 2019

Improve performance of fiber creation by using pool allocation strategy. #2224

Improve performance of fiber creation by using pool allocation strategy. #2224

Conversation

ioquatix commented Jun 2, 2019 • edited Loading

ioquatix commented Jun 2, 2019

ioquatix commented Jun 2, 2019

ioquatix commented Jun 2, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 4, 2019

ioquatix commented Jun 5, 2019

ioquatix commented Jul 16, 2019

ioquatix commented Jul 18, 2019

ioquatix commented Jul 18, 2019

ioquatix commented Jun 2, 2019 •

edited

Loading