Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert no such method Panic to UnknownIdentifierException #7487

Merged
merged 10 commits into from
Aug 7, 2023

Conversation

JaroslavTulach
Copy link
Member

@JaroslavTulach JaroslavTulach commented Aug 3, 2023

Pull Request Description

There is a special handling of toString() and other methods in GraalVM SDK. However we need to throw standard InteropException to activate it. Converting PanicException that represents no such method error on the boundary to UnknownIdentifierException. With such change one can call .toString() on proxies wrapping Enso objects.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

  • All code follows the
    Java,
  • All code has been tested:
    • Unit tests have been written where possible.

@JaroslavTulach JaroslavTulach added the CI: No changelog needed Do not require a changelog entry for this PR. label Aug 3, 2023
@JaroslavTulach JaroslavTulach self-assigned this Aug 3, 2023
@radeusgd
Copy link
Member

radeusgd commented Aug 3, 2023

Is UnknownIdentifierException internal or does it leak into Enso as well?

IIRC we have some code relying on No_Such_Method; will it still work? Is it possible to catch the "new" UnknownIdentifierException?

I.e. will this work fine?

from Standard.Base import all

type Foo
    Instance

main =
    i = Foo.Instance
    Panic.catch No_Such_Method i.nonexistent_method caught_panic->
        IO.println "OK! I caught it, the identifier is: "+(Meta.meta caught_panic.payload.symbol . name)

@radeusgd
Copy link
Member

radeusgd commented Aug 3, 2023

Also - thanks for such a quick fix! Please don't take my comments as fussing for the sake of it - I'm just trying to understand what are the implications and if this changes how we should handle missing method errors in the libraries (in general handling such errors is most often not the best pattern, but due to lack of interfaces, it sometimes provides us useful workarounds, so we rely on this in a few places - I want to make sure we are safe in doing so).

@JaroslavTulach
Copy link
Member Author

Is UnknownIdentifierException internal or does it leak into Enso as well?

No, UnknownIdentifierException isn't "visible" from Enso code.

@radeusgd
Copy link
Member

radeusgd commented Aug 3, 2023

Is UnknownIdentifierException internal or does it leak into Enso as well?

No, UnknownIdentifierException isn't "visible" from Enso code.

Perfect! ❤️

Copy link
Member

@radeusgd radeusgd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Can we extend the tests with one last check reproducing the scenario below?

from Standard.Base import all

polyglot java import foo.Foo

type Fooable_Class
    Instance x

    foo : Integer
    foo self = self.x+100

type Not_A_Number
    Instance

main =
    i = Fooable_Class.Instance (Not_A_Number.Instance)
    method_name caught_panic = Meta.meta caught_panic.payload.symbol . name
    IO.println <| Panic.catch No_Such_Method (i.foo) method_name
    IO.println <| Panic.catch No_Such_Method (Foo.callFoo i) method_name

I'm curious what will happen if we get an Enso No_Such_Method in Enso code which itself is called from within Java. Will the No_Such_Method be converted to the Java exception when crossing the boundary? I expect not, but I'd like to check it.

I'd expect both pure-Enso and polyglot lines to print the same result. Will that be the case?

@JaroslavTulach JaroslavTulach added the CI: Ready to merge This PR is eligible for automatic merge label Aug 3, 2023
@JaroslavTulach
Copy link
Member Author

Looks good to me.
Can we extend the tests with one last check reproducing the scenario below?

I have taken code from the original bugreport #7437 and converted it into JavaInteropTest. I cannot easily integrate your code, I'd would have to rewrite it. I don't see a reason to do so as I don't believe anything is broken.

Feel free to provide/integrate an applicable diff, if you want new test that just tests something that already works.

@radeusgd
Copy link
Member

radeusgd commented Aug 3, 2023

Looks good to me.
Can we extend the tests with one last check reproducing the scenario below?

I have taken code from the original bugreport #7437 and converted it into JavaInteropTest. I cannot easily integrate your code, I'd would have to rewrite it. I don't see a reason to do so as I don't believe anything is broken.

Feel free to provide/integrate an applicable diff, if you want new test that just tests something that already works.

Ok, I will send a diff in a moment, or if you allow me I can push it onto the branch. I'm working on making this a JUnit test.

For now, I've got an Enso script:

from Standard.Base import all
import Standard.Base.Errors.Common.No_Such_Method

polyglot java import foo.Foo

type Fooable_Class
    Value

    foo : Integer
    foo self a = 
        IO.println "I'm already executing `"+self.to_text+".foo("+a.to_text+")`!"
        "".nonexistent_text_method

type Not_A_Number
    Nan

main =
    i = Fooable_Class.Value
    method_name caught_panic = Meta.meta caught_panic.payload.symbol . name
    IO.println <| Panic.catch No_Such_Method (i.foo 1000) method_name
    IO.println <| Panic.catch No_Such_Method (Foo.callFoo i) method_name

which yields

I'm already executing `Fooable_Class.Value.foo(1000)`!
nonexistent_text_method
Calling Foo.foo(2000)
I'm already executing `Fooable_Class.Value.foo(2000)`!
I'm already executing `Fooable_Class.Value.foo(2000)`!
Execution finished with an error: Unsupported operation identifier 'foo' and  object 'Fooable_Class.Value'(language: Enso, type: local.Poly_Project.Main.Fooable_Class). Identifier is not executable or instantiable.
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineException.unsupported(PolyglotEngineException.java:147)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotInteropErrors.invokeUnsupported(PolyglotInteropErrors.java:193)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ProxyInvokeNode.invokeOrExecute(PolyglotObjectProxyHandler.java:264)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ProxyInvokeNode.doCachedMethod(PolyglotObjectProxyHandler.java:214)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ProxyInvokeNodeGen.executeAndSpecialize(PolyglotObjectProxyHandlerFactory.java:308)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ProxyInvokeNodeGen.execute(PolyglotObjectProxyHandlerFactory.java:244)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ObjectProxyNode.doDefault(PolyglotObjectProxyHandler.java:150)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ObjectProxyNodeGen.executeAndSpecialize(PolyglotObjectProxyHandlerFactory.java:124)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ObjectProxyNodeGen.executeImpl(PolyglotObjectProxyHandlerFactory.java:110)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.HostToGuestRootNode.execute(HostToGuestRootNode.java:124)
        at <java> org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotObjectProxyHandler.invoke(PolyglotObjectProxyHandler.java:105)
        at <java> jdk.proxy5/jdk.proxy5.$Proxy44.foo(Unknown Source)
        at <java> foo.Foo.callFoo(Foo.java:13)
        at <enso> Main.main<arg-2>(src\Main.enso:21:47-59)
        at <enso> Panic.catch(Internal)
        at <enso> Main.main<arg-1>(src\Main.enso:21:19-72)
        at <enso> Main.main(src\Main.enso:21:5-72)

As we can see there are several issues here.

  1. We fail with a message that the Fooable_Class.Value does not support the method foo. That is not true, as can be seen in the code and logs - in fact it is executed! The issue likely comes from the fact that the No_Such_Method from "".nonexistent_text_method gets converted into the UnknownIdentifierException, affecting how the method is being resolved.
  2. Moreover, we can see that the method foo is executed by Graal 2 times! That seems not very good. If that method was doing any side effects (like here, printing to screen), they will be repeated. Of course Enso is "supposed" to be pure, but as we all know we use side effects all around the place, so we should be careful about re-running a method when resolving it.

@radeusgd
Copy link
Member

radeusgd commented Aug 3, 2023

Patch for adding a test I'm interested in:

Subject: [PATCH] Add a test checking propagation of Panics through the polyglot call.
---
Index: engine/runtime/src/test/java/org/enso/interpreter/test/JavaInteropTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/engine/runtime/src/test/java/org/enso/interpreter/test/JavaInteropTest.java b/engine/runtime/src/test/java/org/enso/interpreter/test/JavaInteropTest.java
--- a/engine/runtime/src/test/java/org/enso/interpreter/test/JavaInteropTest.java	(revision 7381176053df7dc35e38d67a2eb86240621042d0)
+++ b/engine/runtime/src/test/java/org/enso/interpreter/test/JavaInteropTest.java	(revision 93ba3c602dbc6afacc86d9ca1546ed5cde543253)
@@ -2,6 +2,7 @@
 
 import java.io.ByteArrayOutputStream;
 import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
 import java.util.List;
 
 import org.graalvm.polyglot.Context;
@@ -35,14 +36,17 @@
     out.reset();
   }
 
-  private void checkPrint(String code, List<String> expected) {
-    Value result = evalModule(ctx, code);
-    assertTrue("should return Nothing", result.isNull());
-    String[] logLines = out
+  private String[] getStdOutLines() {
+    return out
         .toString(StandardCharsets.UTF_8)
         .trim()
         .split(System.lineSeparator());
-    assertArrayEquals(expected.toArray(), logLines);
+  }
+
+  private void checkPrint(String code, List<String> expected) {
+    Value result = evalModule(ctx, code);
+    assertTrue("should return Nothing", result.isNull());
+    assertArrayEquals(expected.toArray(), getStdOutLines());
   }
 
   @Test
@@ -203,4 +207,55 @@
     assertEquals("obj.toString() = (Instance 23)", res.getArrayElement(3).asString());
     assertEquals("{(Instance 23)}.foo() = 123", res.getArrayElement(4).asString());
   }
+
+  @Test
+  public void testInterfaceProxyFailures() {
+    var code = """
+        from Standard.Base import all
+        import Standard.Base.Errors.Common.No_Such_Method
+
+        polyglot java import org.enso.example.ToString as Foo
+
+        type My_Exc
+            Error
+
+        type Fooable_Panic
+            Value
+
+            foo : Integer
+            foo self =
+                IO.println "Executing Fooable_Panic.foo"
+                Panic.throw My_Exc.Error
+
+        type Fooable_Unresolved
+            Value
+
+            foo : Integer
+            foo self =
+                IO.println "Executing Fooable_Unresolved.foo"
+                "".nonexistent_text_method
+
+        main =
+            a = Panic.catch My_Exc (Foo.callFoo Fooable_Panic.Value) (.payload)
+            b = Panic.catch No_Such_Method (Foo.callFoo Fooable_Unresolved.Value) (caught-> caught.payload.method_name)
+            [a, b]
+        """;
+
+    Value res;
+    try {
+      res = evalModule(ctx, code);
+    } catch (Exception e) {
+      System.err.println("The test code failed to execute with exception: " + e);
+      System.err.println("It has produced the following stdout so far:");
+      Arrays.stream(getStdOutLines()).forEach(System.err::println);
+      throw e;
+    }
+    assertTrue("It is an array", res.hasArrayElements());
+    assertEquals("Array with 2 elements", 2, res.getArraySize());
+    assertEquals("my panic payload message 1", res.getArrayElement(0).asString());
+    assertEquals("nonexistent_text_method", res.getArrayElement(1).asString());
+    var stdout = getStdOutLines();
+    var expectedLines = List.of("Executing Fooable_Panic.foo", "Executing Fooable_Unresolved.foo");
+    assertArrayEquals(expectedLines.toArray(), stdout);
+  }
 }

And as I have worried, it indeed fails:

[info] done compiling
[error] Test org.enso.interpreter.test.JavaInteropTest.testInterfaceProxyFailures failed: org.graalvm.polyglot.PolyglotException: Unsupported operation identifier 'foo' and  object 'Fooable_Unresolved.Value'(language: Enso, type: Unnamed.Fooable_Unresolved). Identifier is not executable or instantiable., took 0.539 sec
[error]     at com.oracle.truffle.polyglot.PolyglotEngineException.unsupported(PolyglotEngineException.java:147)
[error]     at com.oracle.truffle.polyglot.PolyglotInteropErrors.invokeUnsupported(PolyglotInteropErrors.java:193)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ProxyInvokeNode.invokeOrExecute(PolyglotObjectProxyHandler.java:264)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ProxyInvokeNode.doCachedMethod(PolyglotObjectProxyHandler.java:214)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ProxyInvokeNodeGen.execute(PolyglotObjectProxyHandlerFactory.java:228)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandler$ObjectProxyNode.doDefault(PolyglotObjectProxyHandler.java:150)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandlerFactory$ObjectProxyNodeGen.executeImpl(PolyglotObjectProxyHandlerFactory.java:105)
[error]     at com.oracle.truffle.polyglot.HostToGuestRootNode.execute(HostToGuestRootNode.java:124)
[error]     at com.oracle.truffle.polyglot.PolyglotObjectProxyHandler.invoke(PolyglotObjectProxyHandler.java:105)
[error]     at jdk.proxy2.$Proxy50.foo(Unknown Source)
[error]     at org.enso.example.ToString.callFoo(ToString.java:12)
[error]     at <enso>.Unnamed.main<arg-2>(Unnamed:27)
[error]     at <enso>.Panic.catch(Unknown)
[error]     at <enso>.Unnamed.main(Unnamed:27)
[error]     at org.graalvm.polyglot.Value.execute(Value.java:879)
[error]     at org.enso.interpreter.test.TestBase.evalModule(TestBase.java:128)
[error]     at org.enso.interpreter.test.JavaInteropTest.testInterfaceProxyFailures(JavaInteropTest.java:246)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.lang.reflect.Method.invoke(Method.java:568)
[error]     ...

The test code failed to execute with exception: org.graalvm.polyglot.PolyglotException: Unsupported operation identifier 'foo' and  object 'Fooable_Unresolved.Value'(language: Enso, type: Unnamed.Fooable_Unresolved). Identifier is not executable or instantiable.
It has produced the following stdout so far:
Executing Fooable_Panic.foo
Executing Fooable_Unresolved.foo
Executing Fooable_Unresolved.foo

[error] Failed: Total 12, Failed 1, Errors 0, Passed 11
[error] Failed tests:
[error]         org.enso.interpreter.test.JavaInteropTest
[error] (runtime / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 12 s, completed 3 sie 2023, 18:54:05

As we can see, a regular panic is propagated all fine.

But once we have a No_Such_Method error executed in some 'inner' Enso code (it could possibly be very deep in the call stack), it is actually 'mistakenly' converted into UnknownIdentifierException. Then Graal thinks that it failed to resolve foo - but that is not the case - as we can see foo was resolved and executed already. Even worse - it is being executed twice which really should not be happening.

@JaroslavTulach JaroslavTulach removed the CI: Ready to merge This PR is eligible for automatic merge label Aug 3, 2023
Copy link
Member

@radeusgd radeusgd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to verify before converting No_Such_Method that it is actually relying to the 'currently being resolved' method, on the current object, and not convert all such errors.

The fact that the error saying that nonexistent_text_method does not exist gets converted into an error that a completely different object does not support method foo is quite confusing.

But the fact that the method is actually executed twice because of this - I think we should revisit that before merging, as executing a method multiple times because of a bug in resolution seems very problematic in presence of side effects (as the test suggests).

Copy link
Member

@Akirathan Akirathan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Just a typo in docs.

@radeusgd
Copy link
Member

radeusgd commented Aug 4, 2023

Btw. interestingly the results of the JavaInteropTest seem to be not appearing in the test report uploaded on CI. I wonder if we can do sth to fix this (possibly cc @mwu-tow)?

…so-org/enso into wip/jtulach/UnknownIdentifierException_7437
@JaroslavTulach
Copy link
Member Author

JaroslavTulach commented Aug 5, 2023

Thank you for the test, Radek.

verify before converting No_Such_Method that it is actually relying to the 'currently being resolved' method, on the current object, and not convert all such errors.

Yes, that's what bd77383 does. It marks some InvokeMethodNodes as being onBoundary. Only those are then converted to UnknownIdentifierException.

All the tests are passing (feel free to commit new ones) and the code looks way better then some of the attempts tried previously. Please review.

@@ -60,7 +59,8 @@ InvokeMethodNode buildSorter(int length) {
args,
InvokeCallableNode.DefaultsExecutionMode.EXECUTE,
InvokeCallableNode.ArgumentsExecutionMode.PRE_EXECUTED,
0);
0,
true);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InteropMethodCallNode does interop call onBoundary. Thus passing true in.

Copy link
Member

@radeusgd radeusgd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

I have just one question trying to understand an aspect of the solution.

@JaroslavTulach JaroslavTulach merged commit 3b3dbc3 into develop Aug 7, 2023
24 of 27 checks passed
@JaroslavTulach JaroslavTulach deleted the wip/jtulach/UnknownIdentifierException_7437 branch August 7, 2023 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: No changelog needed Do not require a changelog entry for this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Calling toString on an Enso object implementing a Java interface causes Panic
5 participants