[native-image] C Interface vs Panama Foreign and Method Handles

[saudet]

As the author of JavaCPP, I am looking forward to a more efficient interface to native data and functions, so I have been following Panama, and more recently other projects here at Graal. I may be way ahead of my time here, but one concern I see going forward is Panama moving towards using method handles, as per this thread on the mailing list:
http://mail.openjdk.java.net/pipermail/panama-dev/2018-December/003373.html

However, it does not look like Substrate VM is going to support method handles:
https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md#invokedynamic-bytecode-and-method-handles
As far as I know, there is currently no way to support them efficiently with an AOT compiler anyway.

Panama does not appear concerned about this issue, so I am bringing it up here. As @christianwimmer mentions at #694 (comment) I understand that the plan is to support Panama, but how would it look like more concretely from a technical point of view? Speaking offline with John Rose and Maurizio Cimadamore, they made it crystal clear that Panama is never going to support static methods, for example, so I do not see an efficient way to reuse the current API at org.graalvm.nativeimage.c, which relies on static methods:
https://github.com/oracle/graal/blob/master/substratevm/src/com.oracle.svm.tutorial/src/com/oracle/svm/tutorial/CInterfaceTutorial.java

This is only one example, I am sure there are other issues that need to be addressed, but I would like to start a conversation about this now in the hope of getting things moving, and to avoid postponing issues past the point where it becomes too late to address them!

[christianwimmer]

I might not be up to date with all recent developments regarding Panama, but here is my high level idea: The Panama source code generator produces Java interfaces for C/C++ types and function definitions. These interfaces are at a fairly high level (only declarative elements and no VM specific implementation parts, and therefore also no method handles).

The Java HotSpot VM then generates implementation classes that use method handles, and other low-level VM primitives to efficiently call from Java to native.

Substrate VM cannot re-use these HotSpot specific implementation classes, but also does not need to: During image generation, we can generate our own implementation classes that are Substrate VM specific. These classes use a similar mechanism that we already use for our low-level C interface and our JNI implementation to efficiently call from Java to native. There is no need to generate Java source code that uses the annotations defined in org.graalvm.nativeimage.c.

The Substrate VM implementation of Panama will have the same limitation that we already have for many things (like reflection and JNI usage): You need to specify during image generation which Panama interfaces you are going to use. The image generator needs to prepare all implementation classes because there is no dynamic class loading.

[saudet]

That's how it will have to work, yes, and it can work, but can it work efficiently? The whole point of coming up with an alternative to JNI is to improve performance. We are not gaining any usability over what we can already do with a tool like JNR or JavaCPP, and only losing in compatibility. For example, this is how it will look like to call getpid():

        // bind unistd interface
        var u = Libraries.bind(MethodHandles.lookup(), unistd.class);
        // call getpid from the unistd.h
        System.out.println(u.getpid());

http://hg.openjdk.java.net/panama/dev/raw-file/tip/doc/panama_foreign.html

We basically need to create and call through a dummy object. Memory access using "layouts" also works the same way. Will Substrate VM be able to remove the overhead of this and achieve the same speed as org.graalvm.nativeimage.c?

[christianwimmer]

Yes, it will work efficiently on Substrate VM: the generated implementation class for unistd will have an implementation for getpid that does the same thing as a current @CFunction annotated method. The only difference is that the address for the function pointer call will come from a different location (the unistd implementation class vs. a VM-global array of all @CFunction function pointers).

[saudet]

Thanks for the explanations! It sounds like there are still a lot of indirections involved with @CFunction. To get a better grip of the situation, I ran a little benchmark here with the binaries from GraalVM 1.0-rc11 on my Fedora 27 machine and the following files:

• NativeTest.java:

// ./javac NativeTest.java
// ./native-image NativeTest --shared

import org.graalvm.nativeimage.c.function.CFunction;
import org.graalvm.nativeimage.c.function.CEntryPoint;
import org.graalvm.nativeimage.IsolateThread;

public class NativeTest {
    public native static int testJNI();

    @CFunction
    public native static int testC();

    static interface DummyInterface {
        int test();
    }
    static class DummyClass implements DummyInterface {
        public int test() {
            return testC();
        }
    }
    static DummyInterface dummy = new DummyClass();

    @CEntryPoint(name = "test")
    public static void test(IsolateThread thread) {
        System.load(System.getProperty("user.dir") + "/libNativeTest.so");

        int n = 10000000;

        for (int i = 0; i < n; i++) {
            testJNI();
        }
        long time1 = System.nanoTime();
        for (int i = 0; i < n; i++) {
            testJNI();
        }
        long time2 = System.nanoTime();
        System.out.println("JNI time = " + (time2 - time1) / n);

        for (int i = 0; i < n; i++) {
            testC();
        }
        long time3 = System.nanoTime();
        for (int i = 0; i < n; i++) {
            testC();
        }
        long time4 = System.nanoTime();
        System.out.println("C time = " + (time4 - time3) / n);

        for (int i = 0; i < n; i++) {
            dummy.test();
        }
        long time5 = System.nanoTime();
        for (int i = 0; i < n; i++) {
            dummy.test();
        }
        long time6 = System.nanoTime();
        System.out.println("dummy time = " + (time6 - time5) / n);
    }
}

• NativeTest.c:

// gcc -s -O3 NativeTest.c -shared -o libNativeTest.so -I /usr/lib/jvm/java/include/ -I /usr/lib/jvm/java/include/linux/

#include <jni.h>

jint Java_NativeTest_testJNI(JNIEnv *env, jclass cls) {
    return 42;
}

int testC() {
    return 17;
}

• NativeTestMain.c:

// gcc -s -O3 NativeTestMain.c NativeTest.so libNativeTest.so -o nativetest -Wl,-rpath,. -I.

#include <stdio.h>
#include <time.h>
#include "NativeTest.h"

int main() {
    graal_isolate_t* isolate;
    graal_isolatethread_t* thread;
    graal_create_isolate(0, &isolate, &thread);
    test(thread);

    int n = 10000000;
    for (int i = 0; i < n; i++) {
        testC();
    }
    struct timespec time1;
    clock_gettime(CLOCK_MONOTONIC, &time1);
    for (int i = 0; i < n; i++) {
        testC();
    }
    struct timespec time2;
    clock_gettime(CLOCK_MONOTONIC, &time2);
    printf("native time = %d\n", ((time2.tv_sec - time1.tv_sec) * 1000000000 + (time2.tv_nsec - time1.tv_nsec)) / n);
}

And ./nativetest typically outputs something like the following:

JNI time = 28
C time = 10
dummy time = 10
native time = 7

So this is looking pretty good, interesting. (Now if only there were a way to inline native functions...)

[saudet]

Actually, it looks like I was misunderstanding. Panama is planning on inlining native functions:
https://mail.openjdk.java.net/pipermail/panama-dev/2019-February/004152.html
@christianwimmer This could probably be implemented in Graal using its compiler intrinsics, but is such support also planned for Substrate VM?

[fernando-valdez]

Hello @christianwimmer, do you think this request applies due to the current state of the project?

[christianwimmer]

Closing this issue in favor or the roadmap item #8113