Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IRReturnJump when returning from a proc returned by a lambda #6350

Closed
eregon opened this issue Aug 5, 2020 · 5 comments · Fixed by #6351
Closed

IRReturnJump when returning from a proc returned by a lambda #6350

eregon opened this issue Aug 5, 2020 · 5 comments · Fixed by #6351
Milestone

Comments

@eregon
Copy link
Member

eregon commented Aug 5, 2020

Environment Information

Provide at least:

  • JRuby version: jruby 9.2.13.0 (2.5.7)

Expected Behavior

$ ruby -e 'b=-> { Proc.new { return 42 }; }.call; p b; p b.call'
#<Proc:0x00005584560614b0 -e:1>

Actual Behavior

$ jruby -e 'b=-> { Proc.new { return 42 }; }.call; p b; p b.call'
#<Proc:0xeafc191@-e:1>
Unhandled Java exception: IRReturnJump:<Static Type[1957502751]: block []
  Static Type[1780132728]: local [b=#<Proc:0xeafc191@-e:1>]:42>
org.jruby.ir.runtime.IRReturnJump: null

Might relate to #1134.

@eregon eregon changed the title Unhandled Java exception: IRReturnJump: Unhandled Java exception: IRReturnJump Aug 5, 2020
@eregon
Copy link
Member Author

eregon commented Aug 5, 2020

Also happens for:

def m
  b = -> { Proc.new { return 42 } }.call
  b.call # returns from the method
  p :after # never reached!
end

p m
MRI:
42

JRuby 9.2.13.0:
Unhandled Java exception: IRReturnJump:<Static Type[1226622409]: block []
  Static Type[1957502751]: local [b=#<Proc:0x462d5aee@-:2>]:42>
org.jruby.ir.runtime.IRReturnJump: null

FYI The example is from oracle/truffleruby#1488 where we try to figure out what are the semantics of return inside a proc inside a lambda inside a method.

@headius headius changed the title Unhandled Java exception: IRReturnJump IRReturnJump when returning from a proc returned by a lambda Aug 5, 2020
@headius
Copy link
Member

headius commented Aug 5, 2020

Backtrace for flow-control jumps is available with -Xjump.backtrace:

$ jruby -Xjump.backtrace -e 'b=-> { Proc.new { return 42 }; }.call; p b; p b.call'
#<Proc:0x6bb7cce7@-e:1>
Unhandled Java exception: IRReturnJump:<Static Type[1800605369]: block []
  Static Type[847606512]: local [b=#<Proc:0x6bb7cce7@-e:1>]:42>
org.jruby.ir.runtime.IRReturnJump: null
                  create at org/jruby/ir/runtime/IRReturnJump.java:20
  initiateNonLocalReturn at org/jruby/ir/runtime/IRRuntimeHelpers.java:171
                  <main> at -e:1
              callDirect at org/jruby/runtime/CompiledIRBlockBody.java:141
                    call at org/jruby/runtime/IRBlockBody.java:64
                    call at org/jruby/runtime/Block.java:143
                    call at org/jruby/RubyProc.java:283
                    call at org/jruby/RubyProc$INVOKER$i$call.gen:-1
                    call at org/jruby/internal/runtime/methods/JavaMethod.java:327
            cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:354
                    call at org/jruby/runtime/callsite/CachingCallSite.java:143
       invokeOther3:call at -e:1
                  <main> at -e:1
                     run at -e:-1
     invokeWithArguments at java/lang/invoke/MethodHandle.java:710
                    load at org/jruby/ir/Compiler.java:114
               runScript at org/jruby/Ruby.java:1257
             runNormally at org/jruby/Ruby.java:1176
             runNormally at org/jruby/Ruby.java:1158
             runNormally at org/jruby/Ruby.java:1194
             runFromMain at org/jruby/Ruby.java:977
           doRunFromMain at org/jruby/Main.java:393
             internalRun at org/jruby/Main.java:287
                     run at org/jruby/Main.java:234
                    main at org/jruby/Main.java:206

I believe both of these cases should be LocalJumpError. The bug is that we are not detecting the lambda scope when determining where to return to.

The logic to generate the return jump exception detects the intervening lambda frame and sets that as the target for the jump. This is correct behavior; because the proc was created within a lambda, its return should be from the lambda level.

The subsequent logic that unrolls the stack should raise either an IRReturnJump (if the target frame is active) or a Ruby LocalJumpError (if the target frame is no longer active). Unfortunately this logic skips over the lambda and instead finds the top-level (in the top-level example) or the method (in the method example). In both cases they are still on active.

As a result, we raise an IRReturnJump, expecting it to be handled as a normal non-local return, when we should be raising a LocalJumpError.

I see from oracle/truffleruby#1488 there is some abiguity in the CRuby behavior. I would consider that behavior to be incorrect; once the proc has escaped from the lambda, its return target is no longer valid. It should not return to a different place. This is likely only effective in CRuby because of their messy handling of these jumps... there happens to be someone ready to handle any old return jump, so it just happens to work.

With a simple patch in JRuby, we get the behavior I think is correct for both cases:

$ jruby -Xjump.backtrace -e 'b=-> { Proc.new { return 42 }; }.call; p b; p b.call'
#<Proc:0x47747fb9@-e:1>
LocalJumpError: unexpected return
  <main> at -e:1
  <main> at -e:1

$ jruby -Xjump.backtrace -e 'def foo; b=-> { Proc.new { return 42 }; }.call; p b; p b.call; 42; end; p foo'
#<Proc:0x47747fb9@-e:1>
LocalJumpError: unexpected return
     foo at -e:1
     foo at -e:1
  <main> at -e:1

I will push a PR to see how it lands.

headius added a commit to headius/jruby that referenced this issue Aug 6, 2020
In the logic to look for a value return target, we find a lambda,
via IRRuntimeHelpers.initiateNonLocalReturn calling
getContainingLambda. However when we later try to detect if the
return jump should propagate as a LocalJumpError (because its
target scope is no longer active) we do not do this same search
for a lambda (see IRRuntimeHelpers.checkForLJE). This
inconsistency causes the errors from jruby#6350, because we incorrectly
detect the target as the lambda, but do the detection against its
parent scope and raise an IRReturnJump with an improper target
scope. This jump then bubbles all the way off the stack and never
becomes a LocalJumpError.

In CRuby, when the lambda becomes inactive, the returnn target
moves to the containing return target scope. This means a given
return can potentially have two different targets, depending on
whether its containing lambda is still on the stack or not.

I believe the appropriate behavior in both cases from jruby#6350 is to
raise LocalJumpError immediately. In each case, the return target
for the proc should be the lambda, but the lambda is no longer
active. The return target should not change to the containing
scope, because that's not the proper return target.

The change here adds the lambda logic to the LJE check, so that it
sees the lambda return target is no longer active and eagerly
raises the LJE.

We will need to reconcile this difference with CRuby, but since
this is such an obscure case I doubt it will affect real-world
code.
headius added a commit to headius/jruby that referenced this issue Aug 6, 2020
In the logic to look for a valid return target, we find a lambda,
via `IRRuntimeHelpers.initiateNonLocalReturn` calling
`getContainingLambda`. However when we later try to detect if the
return jump should propagate as a LocalJumpError (because its
target scope is no longer active) we do not do this same search
for a lambda (see `IRRuntimeHelpers.checkForLJE`). This
inconsistency causes the errors from jruby#6350, because we incorrectly
detect the target as the lambda, but do the detection against its
parent scope and raise an IRReturnJump with an improper target
scope. This jump then bubbles all the way off the stack and never
becomes a LocalJumpError.

In CRuby, when the lambda becomes inactive, the returnn target
moves to the containing return target scope. This means a given
return can potentially have two different targets, depending on
whether its containing lambda is still on the stack or not.

I believe the appropriate behavior in both cases from jruby#6350 is to
raise LocalJumpError immediately. In each case, the return target
for the proc should be the lambda, but the lambda is no longer
active. The return target should not change to the containing
scope, because that's not the proper return target.

The change here adds the lambda logic to the LJE check, so that it
sees the lambda return target is no longer active and eagerly
raises the LJE.

We will need to reconcile this difference with CRuby, but since
this is such an obscure case I doubt it will affect real-world
code.
@eregon
Copy link
Member Author

eregon commented Aug 6, 2020

I see from oracle/truffleruby#1488 there is some abiguity in the CRuby behavior. I would consider that behavior to be incorrect; once the proc has escaped from the lambda, its return target is no longer valid. It should not return to a different place. This is likely only effective in CRuby because of their messy handling of these jumps... there happens to be someone ready to handle any old return jump, so it just happens to work.

I agree the semantics seem confusing and potentially unintentional here.
I filed https://bugs.ruby-lang.org/issues/17105

@headius
Copy link
Member

headius commented Aug 12, 2020

Given that we have Shyouhei on our side I think it's likely that the behavior we want will be considered the right behavior. I've merged the JRuby fix that makes both cases LJE.

@headius headius added this to the JRuby 9.3.0.0 milestone Aug 12, 2020
@headius
Copy link
Member

headius commented Apr 1, 2021

Matz has approved the error condition described above, which @ko1 has a patch for in ruby/ruby#4347.

Nothing for us to do since we have already made this change for 9.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants