Fix issues preventing railsbench from running. #36

maximecb · 2021-02-08T21:09:30Z

There's a segfault in bundler preventing the required gems from installing.

# To install microjit
./configure cppflags=-DRUBY_DEBUG --prefix=$HOME/.rubies/ruby-microjit --with-opt-dir=$(brew --prefix openssl):$(brew --prefix readline):$(brew --prefix libyaml):$(brew --prefix gdbm)

https://github.com/Shopify/microjit-bench

The text was updated successfully, but these errors were encountered:

flavorjones · 2021-02-08T22:09:10Z

OK, I've reproduced this error:

NoMethodError: undefined method `gsub' for 72:Integer
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:836:in `remembered_negative_flag_deprecation'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:255:in `install'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:30:in `dispatch'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:24:in `start'
  /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:49:in `block in <top (required)>'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/friendly_errors.rb:130:in `with_friendly_errors'
  /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:37:in `<top (required)>'
  /home/flavorjones/.gem/ruby/3.1.0/bin/bundle:23:in `load'
  /home/flavorjones/.gem/ruby/3.1.0/bin/bundle:23:in `<main>'

running:

ruby 3.1.0dev (2021-02-08T20:40:36Z microjit 54c9cb051e) [x86_64-linux]
Bundler version 2.3.0.dev (2021-02-08 commit unknown)

flavorjones · 2021-02-08T22:09:43Z

@maximecb Is this the error you were seeing? It's not a segfault. If not, can you help me reproduce?

flavorjones · 2021-02-08T22:12:34Z

Ah - I see that this sometimes segfaults. Will keep going.

flavorjones · 2021-02-08T22:35:41Z

This error/segfault does not happen if I comment-out any one of the following lines from ujit_init_codegen:

ujit_reg_op(BIN(nop), gen_nop, false);
ujit_reg_op(BIN(getlocal_WC_0), gen_getlocal_wc0, false);
ujit_reg_op(BIN(getinstancevariable), gen_getinstancevariable, false);
ujit_reg_op(BIN(opt_send_without_block), gen_opt_send_without_block, true);

Conversely, the bug occurs if I comment out everything except those four calls.

My conclusion is that these four opcodes' generated code is interacting to cause this bug.

maximecb · 2021-02-08T23:15:29Z

I was trying to update bundler to fix the gsub error, and that's when I encountered the segfault (in the bundler update process).

Good idea to comment out opcodes. It's probably not an interaction per-se. It's just that uJIT stops compiling iseqs when it finds opodes it doesn't know, so commenting out some opcodes might lead to other instructions not getting compiled.

It's most likely not nop or getlocal, I think we can be fairly confident about those. That leaves getinstancevariable and opt_send_without_block.

If you uncomment this line it will print out the opcodes getting executed:
https://github.com/Shopify/ruby/blob/microjit/ujit_codegen.c#L199

flavorjones · 2021-02-09T13:52:59Z

When I uncomment those lines, here's the output from a run that ended with the exception:

...
getlocal_WC_0
opt_send_without_block
nop
getlocal_WC_0
opt_send_without_block
nop
getlocal_WC_0
opt_send_without_block
nop
getlocal_WC_0
opt_send_without_block
putself
opt_send_without_block
putself
opt_send_without_block
putself
nop
nop
compiling 0: putnil
compiling 1: setlocal_WC_0
compiling 3: getlocal_WC_0
compiling 5: branchif
--- ERROR REPORT TEMPLATE -------------------------------------------------------
# Error Report

## Questions

Please fill out answers to these questions, it'll help us figure out
why things are going wrong.

- **What did you do?**

  I ran the command `/home/flavorjones/.gem/ruby/3.1.0/bin/bundle install`

- **What did you expect to happen?**

  I expected Bundler to...

- **What happened instead?**

  Instead, what happened was...

- **Have you tried any solutions posted on similar issues in our issue tracker, stack overflow, or google?**

  I tried...

- **Have you read our issues document, https://github.com/rubygems/rubygems/blob/master/bundler/doc/contributing/ISSUES.md?**

  ...

## Backtrace

NoMethodError: undefined method `gsub' for 72:Integer
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:836:in `remembered_negative_flag_deprecation'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:255:in `install'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:30:in `dispatch'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:24:in `start'
  /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:49:in `block in <top (required)>'
  /home/flavorjones/local/lib/ruby/3.1.0/bundler/friendly_errors.rb:130:in `with_friendly_errors'
  /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:37:in `<top (required)>'
  /home/flavorjones/.gem/ruby/3.1.0/bin/bundle:23:in `load'
  /home/flavorjones/.gem/ruby/3.1.0/bin/bundle:23:in `<main>'

and here's one that ends with a segfault:

...
getinstancevariable
getlocal_WC_0
opt_send_without_block
getinstancevariable
getinstancevariable
dup
branchif
leave
nop
getinstancevariable
getlocal_WC_0
opt_send_without_block
getinstancevariable
getinstancevariable
dup
branchif
leave
nop
getinstancevariable
getlocal_WC_0
opt_send_without_block
getinstancevariable
getinstancevariable
dup
branchif
leave
/home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/parser/options.rb:50: [BUG] Segmentation fault at 0x000000000000000
0
ruby 3.1.0dev (2021-02-08T20:40:36Z microjit 54c9cb051e) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0019 p:0012 s:0133 e:000129 BLOCK  /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/parser/options.rb:50 [FINISH]
c:0018 p:---- s:0126 e:000125 CFUNC  :each
c:0017 p:0069 s:0122 e:000121 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/parser/options.rb:49 [FINISH]
c:0016 p:---- s:0113 e:000112 CFUNC  :new
c:0015 p:0125 s:0105 e:000104 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:78
c:0014 p:0058 s:0089 E:000810 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:26
c:0013 p:0022 s:0081 e:000080 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/shell.rb:45
c:0012 p:0008 s:0074 e:000073 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:55 [FINISH]
c:0011 p:---- s:0065 e:000064 CFUNC  :new
c:0010 p:0183 s:0058 e:000057 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:388
c:0009 p:0008 s:0045 e:000044 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:30
c:0008 p:0066 s:0040 e:000039 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485
c:0007 p:0008 s:0033 e:000032 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/cli.rb:24
c:0006 p:0108 s:0028 e:000027 BLOCK  /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:49
c:0005 p:0014 s:0022 e:000021 METHOD /home/flavorjones/local/lib/ruby/3.1.0/bundler/friendly_errors.rb:130
c:0004 p:0160 s:0017 E:001f18 TOP    /home/flavorjones/local/lib/ruby/gems/3.1.0/gems/bundler-2.3.0.dev/libexec/bundle:37 [FINISH]
c:0003 p:---- s:0013 e:000012 CFUNC  :load
c:0002 p:0112 s:0008 E:001890 EVAL   /home/flavorjones/.gem/ruby/3.1.0/bin/bundle:23 [FINISH]
c:0001 p:0000 s:0003 E:000d40 (none) [FINISH]

maximecb · 2021-02-09T15:38:30Z

I'm guessing the exception is because bundler is expecting some method that's been deprecated in the bleeding edge Ruby. There's probably nothing we can do about that (except maybe update bundler or manually modify its code). We could try asking around if other people are having that problem.

Segfault in leave is a little strange :O

We have two codegen functions that might help, you can place some print_str(cb, "string constant"); statements inside gen_leave. There is also print_int(cb, int_operand) and print_ptr(cb, operand). Those will generate code that prints output without disrupting anything.

gen_leave:
https://github.com/Shopify/ruby/blob/microjit/ujit_codegen.c#L1405

For example, adding print_ptr(cb, REG_CFP) on this line:
https://github.com/Shopify/ruby/blob/microjit/ujit_codegen.c#L1431

Would print the current value of the REG_CFP pointer.

flavorjones · 2021-02-09T15:56:47Z

I don't think this is a Ruby deprecation, because it runs fine with --disable-ujit set.

I'm pattern matching based on my past experience, but this looks like it may be an object being freed, and then the slot getting re-used sometimes. So on occasion an unexpected object gets gsub called on it; other times it's a use-after-free segfault. I'd like to confirm this by running valgrind, but I'm also having problems running valgrind (see my slack message from this morning).

maximecb · 2021-02-09T16:52:01Z

I'm guessing valgrind doesn't like our executable memory allocation.

This is the code that allocates executable memory: https://github.com/Shopify/ruby/blob/microjit/ujit_asm.c#L127

You could try either removing the position hint (make first argument NULL), or changing some of the flags?

flavorjones · 2021-02-09T17:11:15Z

Passing NULL as the address does allow this to run a bit further under valgrind, but it then fails in a new way that I don't know how to fix:

compiling 0: nop
compiling 1: putnil
nop

vex: priv/guest_amd64_toIR.c:20628 (dis_ESC_NONE): Assertion `sz == 2 || sz == 4' failed.
vex storage: T total 1463532320 bytes allocated
vex storage: P total 512 bytes allocated

valgrind: the 'impossible' happened:
   LibVEX called failure_exit().

host stacktrace:
==65967==    at 0x58046FFA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x58047127: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x58047390: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x580473C0: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x5805E3DA: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x58140040: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x581AC288: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x581D4FD7: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x581D5850: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x5815B13A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x5813D916: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x5813E20D: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x58060E47: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x580A68C5: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)
==65967==    by 0x580F5FD4: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/memcheck-amd64-linux)

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 65967)
==65967==    at 0x8A78081: ???
==65967==    by 0x3: ???
client stack range: [0x1FFE802000 0x1FFF000FFF] client SP: 0x1FFEFFC070
valgrind stack range: [0x1002CB6000 0x1002DB5FFF] top usage: 16264 of 1048576

maximecb · 2021-02-09T18:07:08Z

Not sure what to do about this. Maybe valgrind can't cope with the generated code.

Maybe we should wait until Alan merges his GC PR. If we're lucky it might fix this issue. If not, we should probably look for a simpler failing test.

flavorjones · 2021-02-11T14:17:24Z

So this appears to be a valgrind bug:

vex: priv/guest_amd64_toIR.c:20628 (dis_ESC_NONE): Assertion `sz == 2 || sz == 4' failed.
vex storage: T total 1463532320 bytes allocated
vex storage: P total 512 bytes allocated

where the size is 8 if there is a REX.W prefix before POPF ... but this method asserts that it's either 2 or 4. I'm building valgrind now so that it accepts 8 and we'll see what happens.

flavorjones · 2021-02-11T14:27:54Z

OK, having patched valgrind locally, this is now what I see when debugging the original problem:

...
getinstancevariable
getlocal_WC_0
opt_send_without_block
getinstancevariable
getinstancevariable
==363963== Invalid read of size 8
==363963==    at 0x8A99E0C: ???
==363963==    by 0x54FD3F: ???
==363963==    by 0x4CC3BBF: ???
==363963==    by 0x10B49B8F: ???
==363963==    by 0x2BFE1B: rb_st_insert (st.c:1098)
==363963==    by 0xA4316AC86EF22DFF: ???
==363963==    by 0xFEFFD2DF: ???
==363963==    by 0x4CC3BBF: ???
==363963==    by 0x1FFEFFD42F: ???
==363963==    by 0x4CC3BBF: ???
==363963==    by 0x4DC39CF: ???
==363963==    by 0x4E61397: ???
==363963==    by 0x10B4B2D7: ???
==363963==    by 0x33BB7E: rb_vm_exec (vm.c:2177)
==363963==    by 0x3FFFF: ???
==363963==    by 0x10BDCB1F: ???
==363963==  Address 0x121258b8 is 8 bytes after a block of size 96 alloc'd
==363963==    at 0x483B7FB: malloc (vg_replace_malloc.c:309)
==363963==    by 0x18EBB1: objspace_xmalloc0 (gc.c:10636)
==363963==    by 0x32093A: rb_obj_transient_heap_evacuate (variable.c:1403)
==363963==    by 0x312DC1: transient_heap_block_evacuate.isra.0 (transient_heap.c:731)
==363963==    by 0x312EF7: transient_heap_evacuate (transient_heap.c:805)
==363963==    by 0x312EF7: transient_heap_evacuate (transient_heap.c:774)
==363963==    by 0x360017: rb_postponed_job_flush (vm_trace.c:1694)
==363963==    by 0x2F22E0: rb_threadptr_execute_interrupts.part.0 (thread.c:2427)
==363963==    by 0x32F869: vm_pop_frame (vm_insnhelper.c:413)
==363963==    by 0x32F869: rb_vm_pop_frame (vm_insnhelper.c:422)
==363963==    by 0x32F869: vm_call_cfunc_with_frame (vm_insnhelper.c:2902)
==363963==    by 0x33B26D: vm_sendish (vm_insnhelper.c:4504)
==363963==    by 0x346A4B: vm_exec_core (insns.def:789)
==363963==    by 0x33BB7E: rb_vm_exec (vm.c:2177)

maximecb · 2021-02-11T15:18:47Z

We did fix some bugs yesterday. Can you pull the latest changes to see if that fixes it?

flavorjones · 2021-02-11T17:08:09Z

I'm going to close this, because I think the issues here are related to the broader GC challenges we're looking into.

This commit adds a check on the ep just like in the mark function. The env can contain null bytes if allocation tracing is enabled. We're seeing errors during autocompaction like this: ``` (lldb) bt 40 * thread #1, name = 'ruby', stop reason = signal SIGABRT frame #0: 0x00007f7d64b6018b libc.so.6`raise + 203 frame #1: 0x00007f7d64b3f859 libc.so.6`abort + 299 frame #2: 0x000055af5f2fefc9 ruby`die at error.c:764:5 frame #3: 0x000055af5f2ff1ac ruby`rb_bug_for_fatal_signal(default_sighandler=0x0000000000000000, sig=11, ctx=0x000055af60bc3340, fmt="") at error.c:804:5 frame #4: 0x000055af5f4bd08f ruby`sigsegv(sig=11, info=0x000055af60bc3470, ctx=0x000055af60bc3340) at signal.c:960:5 frame #5: 0x00007f7d64ebe3c0 libpthread.so.0`__restore_rt frame #6: 0x000055af5f339b0a ruby`gc_ref_update_imemo(objspace=0x000055af60b2b040, obj=0x00007f7d5b513fd0) at gc.c:9046:13 frame #7: 0x000055af5f339172 ruby`gc_update_object_references(objspace=0x000055af60b2b040, obj=0x00007f7d5b513fd0) at gc.c:9307:9 frame #8: 0x000055af5f338e79 ruby`gc_ref_update(vstart=0x00007f7d5b510010, vend=0x00007f7d5b513ff8, stride=40, objspace=0x000055af60b2b040, page=0x000055af62577aa0) at gc.c:9452:21 frame #9: 0x000055af5f337846 ruby`gc_update_references(objspace=0x000055af60b2b040, heap=0x000055af60b2b068) at gc.c:9481:9 frame #10: 0x000055af5f336569 ruby`gc_compact_finish(objspace=0x000055af60b2b040, heap=0x000055af60b2b068) at gc.c:4840:5 frame #11: 0x000055af5f335efb ruby`gc_page_sweep(objspace=0x000055af60b2b040, heap=0x000055af60b2b068, sweep_page=0x000055af63a1eb30) at gc.c:5046:13 frame #12: 0x000055af5f3355c5 ruby`gc_sweep_step(objspace=0x000055af60b2b040, heap=0x000055af60b2b068) at gc.c:5214:19 frame #13: 0x000055af5f33daf6 ruby`gc_sweep_rest(objspace=0x000055af60b2b040) at gc.c:5271:2 frame #14: 0x000055af5f33cacd ruby`gc_sweep(objspace=0x000055af60b2b040) at gc.c:5389:2 frame #15: 0x000055af5f33c21d ruby`gc_marks_rest(objspace=0x000055af60b2b040) at gc.c:7555:5 frame #16: 0x000055af5f324d41 ruby`gc_rest(objspace=0x000055af60b2b040) at gc.c:8457:13 frame #17: 0x000055af5f3297d8 ruby`garbage_collect(objspace=0x000055af60b2b040, reason=45568) at gc.c:8318:9 frame #18: 0x000055af5f344ece ruby`garbage_collect_with_gvl(objspace=0x000055af60b2b040, reason=45568) at gc.c:8632:9 frame #19: 0x000055af5f344e61 ruby`objspace_malloc_gc_stress(objspace=0x000055af60b2b040) at gc.c:10592:9 frame #20: 0x000055af5f32ced1 ruby`objspace_xmalloc0(objspace=0x000055af60b2b040, size=64) at gc.c:10767:5 frame #21: 0x000055af5f32ce11 ruby`ruby_xmalloc0(size=64) at gc.c:10988:12 frame #22: 0x000055af5f32cdac ruby`ruby_xmalloc_body(size=64) at gc.c:10997:12 frame #23: 0x000055af5f329415 ruby`ruby_xmalloc(size=64) at gc.c:12942:12 frame #24: 0x00007f7d611c4fe5 objspace.so`newobj_i(tpval=0x00007f7d5b553770, data=0x000055af639031a0) at object_tracing.c:101:35 frame #25: 0x000055af5f5b283f ruby`tp_call_trace(tpval=0x00007f7d5b553770, trace_arg=0x00007fff1016d398) at vm_trace.c:1115:2 frame #26: 0x000055af5f5b50ec ruby`exec_hooks_body(ec=0x000055af60b2b700, list=0x000055af60b2b920, trace_arg=0x00007fff1016d398) at vm_trace.c:304:3 frame #27: 0x000055af5f5b0f24 ruby`exec_hooks_unprotected(ec=0x000055af60b2b700, list=0x000055af60b2b920, trace_arg=0x00007fff1016d398) at vm_trace.c:333:5 frame #28: 0x000055af5f5b0da8 ruby`rb_exec_event_hooks(trace_arg=0x00007fff1016d398, hooks=0x000055af60b2b920, pop_p=0) at vm_trace.c:378:13 frame #29: 0x000055af5f33f8e2 ruby`rb_exec_event_hook_orig(ec=0x000055af60b2b700, hooks=0x000055af60b2b920, flag=1048576, self=0x00007f7d5b5c08c0, id=0, called_id=0, klass=0x0000000000000000, data=0x00007f7d5b513fd0, pop_p=0) at vm_core.h:1989:5 frame #30: 0x000055af5f334975 ruby`gc_event_hook_body(ec=0x000055af60b2b700, objspace=0x000055af60b2b040, event=1048576, data=0x00007f7d5b513fd0) at gc.c:2083:5 * frame #31: 0x000055af5f3342df ruby`newobj_slowpath_wb_protected [inlined] newobj_slowpath(klass=0x00007f7d5b9d19c8, flags=0x000000000000001a, objspace=0x000055af60b2b040, cr=0x000055af60b2b910, wb_protected=1) at gc.c:2284:9 frame #32: 0x000055af5f33410f ruby`newobj_slowpath_wb_protected(klass=0x00007f7d5b9d19c8, flags=0x000000000000001a, objspace=0x000055af60b2b040, cr=0x000055af60b2b910) at gc.c:2299 frame #33: 0x000055af5f333de9 ruby`newobj_of0(klass=0x00007f7d5b9d19c8, flags=0x000000000000001a, wb_protected=1, cr=0x000055af60b2b910) at gc.c:2338:11 frame #34: 0x000055af5f3227ae ruby`newobj_of(klass=0x00007f7d5b9d19c8, flags=0x000000000000001a, v1=0x000055af657d88a0, v2=0x000055af657d8890, v3=0x0000000000000000, wb_protected=1) at gc.c:2348:17 frame #35: 0x000055af5f322c5b ruby`rb_imemo_new(type=imemo_env, v1=0x000055af657d88a0, v2=0x000055af657d8890, v3=0x0000000000000000, v0=0x00007f7d5b9d19c8) at gc.c:2434:12 frame #36: 0x000055af5f5a3925 ruby`vm_env_new(env_ep=0x000055af657d88a0, env_body=0x000055af657d8890, env_size=4, iseq=0x00007f7d5b9d19c8) at vm_core.h:1363:33 frame #37: 0x000055af5f5a3808 ruby`vm_make_env_each(ec=0x000055af60b2b700, cfp=0x00007f7d6482fc90) at vm.c:801:11 frame #38: 0x000055af5f5a368d ruby`vm_make_env_each(ec=0x000055af60b2b700, cfp=0x00007f7d6482fc20) at vm.c:752:13 frame #39: 0x000055af5f5a368d ruby`vm_make_env_each(ec=0x000055af60b2b700, cfp=0x00007f7d6482fbb0) at vm.c:752:13 (lldb) f 31 frame #31: 0x000055af5f3342df ruby`newobj_slowpath_wb_protected [inlined] newobj_slowpath(klass=0x00007f7d5b9d19c8, flags=0x000000000000001a, objspace=0x000055af60b2b040, cr=0x000055af60b2b910, wb_protected=1) at gc.c:2284:9 2281 } 2282 GC_ASSERT(obj != 0); 2283 newobj_init(klass, flags, wb_protected, objspace, obj); -> 2284 gc_event_hook_prep(objspace, RUBY_INTERNAL_EVENT_NEWOBJ, obj, newobj_fill(obj, 0, 0, 0)); 2285 } 2286 RB_VM_LOCK_LEAVE_CR_LEV(cr, &lev); 2287 (lldb) p obj (VALUE) $3 = 0x00007f7d5b513fd0 (lldb) f 6 frame #6: 0x000055af5f339b0a ruby`gc_ref_update_imemo(objspace=0x000055af60b2b040, obj=0x00007f7d5b513fd0) at gc.c:9046:13 9043 { 9044 rb_env_t *env = (rb_env_t *)obj; 9045 TYPED_UPDATE_IF_MOVED(objspace, rb_iseq_t *, env->iseq); -> 9046 UPDATE_IF_MOVED(objspace, env->ep[VM_ENV_DATA_INDEX_ENV]); 9047 gc_update_values(objspace, (long)env->env_size, (VALUE *)env->env); 9048 } 9049 break; (lldb) p obj (VALUE) $4 = 0x00007f7d5b513fd0 (lldb) ```

[Bug #19793] Dummy frames are created at the top level when requiring another file. While requiring a file, it will try to convert using encodings. Some of these encodings will not respond to to_str. If method_missing is redefined on Object, then it will call method_missing and attempt raise an error. However, the iseq is invalid as it's a dummy frame so it will write an invalid iseq to the created NoMethodError. The following script crashes: ``` GC.stress = true class Object public :method_missing end File.write("/tmp/empty.rb", "") require "/tmp/empty.rb" ``` With the following backtrace: ``` frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12 frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12 frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9 frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17 frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9 frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5 frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9 frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12 frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35 frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12 frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5 frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16 frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15 frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5 ... frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12 frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20 frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12 frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18 frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17 frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15 frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21 frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12 frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15 frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9 frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11 frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19 frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5 frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12 ... frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12 frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5 ... ```

Fix crash in NoMethodError for dummy frames MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [Bug #19793] Dummy frames are created at the top level when requiring another file. While requiring a file, it will try to convert using encodings. Some of these encodings will not respond to to_str. If method_missing is redefined on Object, then it will call method_missing and attempt raise an error. However, the iseq is invalid as it's a dummy frame so it will write an invalid iseq to the created NoMethodError. The following script crashes: ``` GC.stress = true class Object public :method_missing end File.write("/tmp/empty.rb", "") require "/tmp/empty.rb" ``` With the following backtrace: ``` frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12 frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12 frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9 frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17 frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9 frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5 frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9 frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12 frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35 frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12 frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5 frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16 frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15 frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5 ... frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12 frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20 frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12 frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18 frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17 frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15 frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21 frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12 frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15 frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9 frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11 frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19 frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5 frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12 ... frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12 frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5 ... ``` --- error.c | 4 +++- test/ruby/test_require.rb | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-)

maximecb created this issue from a note in YJIT (TODO) Feb 8, 2021

maximecb assigned maximecb and flavorjones and unassigned maximecb Feb 8, 2021

maximecb moved this from TODO to In Progress in YJIT Feb 8, 2021

maximecb added the bug Something isn't working label Feb 8, 2021

flavorjones closed this as completed Feb 11, 2021

maximecb moved this from In Progress to Done in YJIT Feb 11, 2021

maximecb removed this from Done in YJIT Feb 11, 2021

flavorjones mentioned this issue Feb 16, 2021

ujit getinstancevariable code checks ivar extended table size #40

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix issues preventing railsbench from running. #36

Fix issues preventing railsbench from running. #36

maximecb commented Feb 8, 2021 •

edited

Loading

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

maximecb commented Feb 8, 2021

flavorjones commented Feb 9, 2021 •

edited

Loading

maximecb commented Feb 9, 2021

flavorjones commented Feb 9, 2021

maximecb commented Feb 9, 2021

flavorjones commented Feb 9, 2021

maximecb commented Feb 9, 2021

flavorjones commented Feb 11, 2021

flavorjones commented Feb 11, 2021

maximecb commented Feb 11, 2021

flavorjones commented Feb 11, 2021

Fix issues preventing railsbench from running. #36

Fix issues preventing railsbench from running. #36

Comments

maximecb commented Feb 8, 2021 • edited Loading

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

flavorjones commented Feb 8, 2021

maximecb commented Feb 8, 2021

flavorjones commented Feb 9, 2021 • edited Loading

maximecb commented Feb 9, 2021

flavorjones commented Feb 9, 2021

maximecb commented Feb 9, 2021

flavorjones commented Feb 9, 2021

maximecb commented Feb 9, 2021

flavorjones commented Feb 11, 2021

flavorjones commented Feb 11, 2021

maximecb commented Feb 11, 2021

flavorjones commented Feb 11, 2021

maximecb commented Feb 8, 2021 •

edited

Loading

flavorjones commented Feb 9, 2021 •

edited

Loading