Panama JDK 22

At this time, this can be tested with: openjdk:22-slim-bookworm

$ docker pull openjdk:22-slim-bookworm

More images: https://hub.docker.com/_/openjdk/

First contact

Acquire the linker and the symbol lookup

Linker LINKER = Linker.nativeLinker();
SymbolLookup SYMBOL_LOOKUP = LINKER.defaultLookup();

get the pid of the process

var getpid = LINKER.downcallHandle(
        SYMBOL_LOOKUP.find("getpid").orElseThrow(),
        FunctionDescriptor.of(ValueLayout.JAVA_LONG) // (1)
);

1. getpid returns the pid of the process

Downcall for printf

var printf = LINKER.downcallHandle(
        SYMBOL_LOOKUP.find("printf").orElseThrow(),
        FunctionDescriptor.of(
                ValueLayout.JAVA_LONG,
                ValueLayout.ADDRESS // (1)
        )
);

1. The printf function takes a pointer to a memory address, note the carrier type of the ADDRESS layout is a MemorySegment.

try (var arena = Arena.ofConfined()) { // (1)
  var memorySegment = arena.allocateFrom(str); // (2)
  return (long) printf.invoke(memorySegment); // (3)
}

try (var scope = ResourceScope.newConfinedScope()) {
  var allocator = SegmentAllocator.nativeAllocator(scope);
  var memorySegment = allocator.allocateFrom(str); // (2)
  return (long) printf.invoke(memorySegment.address()); // (3)
}

1. Create an arena confimed to this try-with-resources block

2. Copy the Java String to an off-heap memory segment

3. Invoke the printf function

Tip

If an WrongMethodTypeException is raised The message gives hints about misuse. Exception in thread "main" java.lang.invoke.WrongMethodTypeException: handle's method type (MemorySegment)long but found (long)long handle’s method type (MemorySegment)long refers the method descriptor (or signature), the argument types are within parenthesis MemorySegement and the return type is at the end: long ; this is described with the FunctionDescriptor FunctionDescriptor.of( ValueLayout.JAVA_LONG, // (1) ValueLayout.ADDRESS // (2) ) JAVA_LONG is the return type, e.g. long ADDRESS is actually passed as MemorySegment but found (long)long refers to the types seen on the call site, e.g. (long) mh.invoke(memorySegment.address()), and follows the method descriptor convention: argument types are within parenthesis and return type is at the end. In this case, the difference is in the type of the argument, a MemorySegment is expected, but an argument of type long is passed. `MemorySegment` is expected because it is the carrier type for a ValueLayout.ADDRESS is a MemorySegment.

Caution

Mind the invoke method invoke or invokeExact, as the first is able to infer the signature, by accommodating type hierarchy, while the second requires the exact same type.

TouchId

Credits to Carl Dea.

nm touchid-swift-lib/build/lib/main/release/shared/macos/libTouchIdDemoLib.dylib

Show JAVA_LIBRARY_PATH

Load library

System.loadLibrary("TouchIdDemoLib");

Downcall to swift method

var authenticate_user = LINKER.downcallHandle(
        SYMBOL_LOOKUP.lookup("authenticate_user_touchid").get(),
        FunctionDescriptor.ofVoid()
);

// no arena needed here
authenticate_user.invoke();

BLAKE3 (ffm-blake3)

Preparations

building libblake3.so

cd c
# building the intrinsics-based implementations
gcc -c -fPIC -O3 -msse2 blake3_sse2.c -o blake3_sse2.o
gcc -c -fPIC -O3 -msse4.1 blake3_sse41.c -o blake3_sse41.o
gcc -c -fPIC -O3 -mavx2 blake3_avx2.c -o blake3_avx2.o
gcc -c -fPIC -O3 -mavx512f -mavx512vl blake3_avx512.c -o blake3_avx512.o
gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c \
    blake3_avx2.o blake3_avx512.o blake3_sse41.o blake3_sse2.o

The Gradle project is set up to invoke jextract automatically, just update the jextract_home property.

Note

Download jextract on https://jdk.java.net/jextract/ for released JDKs. Or, for unreleased JDKs, build form source https://github.com/openjdk/jextract. Choose available branch, e.g. jdk22, look at the build instructions. The build is based on Gradle and as such needs a JDK, up-to-now the version is 7.3.3, so it needs JDK 17 to run Gradle. Also, usually previous or current JDK is needed, and one of the latest LLVM distributions, they are passed on cli via specific Gradle properties (could be set in $HOME/.gradle.init.properties). sh ./gradlew \ -Pjdk22_home=$HOME/.asdf/installs/java/openjdk-22-ea+26 \ -Pllvm_home=$(brew --prefix llvm) \ clean verify Then set the path to the jextract home directory.

This is still possible to run jextract manually.

Example 1. Invoking jextract manually

simple alias to jextract

alias jextract=$HOME/opensource/jextract/build/jextract/bin/jextract

generates the blake3 mappings

jextract \
  -d ffm-blake3/build/generated/sources/jextract-blake3/java \
  --source \
  --target-package blake3 \
  -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include \
  ~/opensource/BLAKE3/c/blake3.h
  --include-typedef blake3_chunk_state
  --include-typedef blake3_hasher
  --include-constant BLAKE3_BLOCK_LEN
  --include-constant BLAKE3_CHUNK_LEN
  --include-constant BLAKE3_KEY_LEN
  --include-constant BLAKE3_MAX_DEPTH
  --include-constant BLAKE3_OUT_LEN
  --include-constant BLAKE3_VERSION_STRING
  --include-function blake3_hasher_finalize
  --include-function blake3_hasher_finalize_seek
  --include-function blake3_hasher_init
  --include-function blake3_hasher_init_derive_key
  --include-function blake3_hasher_init_derive_key_raw
  --include-function blake3_hasher_init_keyed
  --include-function blake3_hasher_reset
  --include-function blake3_hasher_update
  --include-function blake3_version

Of string

Load the library

System.load("/Users/brice.dutheil/opensource/BLAKE3/c/libblake3.so");

Open a scope

try (var arena = Arena.ofConfined()) {

}

Initialize the hasher

var hasher = blake3_hasher.allocate(scope); // (1)
blake3_h.blake3_hasher_init(hasher); // (2)

1. blake3_hasher is a specific data structure

2. blake3_hasher_init is a function that initializes the hasher

Add content to hasher

var content = arena.allocateFrom("Hello panama!\n", StandardCharsets.US_ASCII);

blake3_h.blake3_hasher_update(hasher, content, content.byteSize() - 1);

Finish hashing

var out = arena.allocate(
        MemoryLayout.sequenceLayout(
                blake3_h.BLAKE3_OUT_LEN(),
                ValueLayout.JAVA_BYTE
        )
);
blake3_h.blake3_hasher_finalize(hasher, out, blake3_h.BLAKE3_OUT_LEN());

Of file

A memory segment for a file can only be obtained from a FileChannel

try (Arena arena = Arena.ofConfined();
     FileChannel channel = FileChannel.open(path)) {

    // ...
}

Re-using the blake3 hasher from above, the byte array is reused.

Update the hasher with a segment from a memory-mapped file

var content = MemorySegment.mapFile(
      path,
      0,
      Files.size(path),
      MapMode.READ_ONLY,
      scope
);
blake3_h.blake3_hasher_update(hasher, content, content.byteSize());