akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture #986

yoppworks-andrew · 2022-09-24T19:31:16Z

Versions used

Attempted with akka-persistence-cassandra 1.0.5 and 1.0.6, and with Zulu JDKs 1.8.0_345, 11.0.15, 11.0.16, 17.0.4, 18.0.2, and 19

Expected Behavior

Running a minimal example in IntelliJ using Scala, akka-persistence-cassandra fails to start. MWE follows

// Main.scala
import akka.persistence.cassandra.testkit.CassandraLauncher

import java.nio.file.Paths

object Main extends App {

	private val cassandraDirectory = Paths.get("target", "cassandra")

	CassandraLauncher
		.start(
			cassandraDirectory = cassandraDirectory.toFile,
			configResource = CassandraLauncher.DefaultTestConfigResource,
			clean = true,
			port = 9042,
			classpath = CassandraLauncher.classpathForResources("logback-test.xml")
		)
}

// build.sbt
ThisBuild / version := "0.1.0-SNAPSHOT"

ThisBuild / scalaVersion := "2.13.9"

lazy val root = (project in file("."))
  .settings(
    name := "cassandra-jna-mwe",
    libraryDependencies := Seq(
      "com.typesafe.akka" %% "akka-persistence-cassandra" % "1.0.6",
      "com.typesafe.akka" %% "akka-persistence-cassandra-launcher" % "1.0.6"
    )
  )

(Dummy logback-test.xml must be included as well.)

Actual Behavior

Different exceptions are thrown for different JDK versions

Java 8

2022-09-24 15:21:25 ERROR o.a.c.utils.NativeLibraryDarwin  Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /private/var/folders/x8/2qjx264s0m72bv1bk5f89q600000gn/T/jna--1413260435/jna6418092536739703311.tmp: dlopen(/private/var/folders/x8/2qjx264s0m72bv1bk5f89q600000gn/T/jna--1413260435/jna6418092536739703311.tmp, 0x0001): tried: '/private/var/folders/x8/2qjx264s0m72bv1bk5f89q600000gn/T/jna--1413260435/jna6418092536739703311.tmp' (fat file, but missing compatible architecture (have (i386,x86_64), need (arm64e)))
	at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_345]
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) ~[na:1.8.0_345]
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) ~[na:1.8.0_345]
	at java.lang.Runtime.load0(Runtime.java:811) ~[na:1.8.0_345]
	at java.lang.System.load(System.java:1088) ~[na:1.8.0_345]
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) ~[cassandra-bundle.jar:1.0.6]
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) ~[cassandra-bundle.jar:1.0.6]
	at com.sun.jna.Native.<clinit>(Native.java:140) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.utils.NativeLibraryDarwin.<clinit>(NativeLibraryDarwin.java:53) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:88) [cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) [cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) [cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) [cassandra-bundle.jar:1.0.6]
2022-09-24 15:21:25 ERROR o.a.cassandra.service.StartupChecks  cassandra.jmx.local.port missing from cassandra-env.sh, unable to start local JMX service.
2022-09-24 15:21:25 ERROR o.a.c.service.CassandraDaemon  The native library could not be initialized properly. 
Exception in thread "main" java.lang.RuntimeException: Cassandra did not start within 45 seconds
...

Java 11.0.15 / 11.0.16

2022-09-24 15:24:45 ERROR o.a.c.utils.NativeLibraryDarwin  Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/x8/2qjx264s0m72bv1bk5f89q600000gn/T/jna--1413260435/jna5751740427277883523.tmp
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2633) ~[na:na]
	at java.base/java.lang.Runtime.load0(Runtime.java:768) ~[na:na]
	at java.base/java.lang.System.load(System.java:1837) ~[na:na]
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) ~[cassandra-bundle.jar:1.0.6]
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) ~[cassandra-bundle.jar:1.0.6]
	at com.sun.jna.Native.<clinit>(Native.java:140) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.utils.NativeLibraryDarwin.<clinit>(NativeLibraryDarwin.java:53) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:88) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) ~[cassandra-bundle.jar:1.0.6]
2022-09-24 15:24:45 ERROR o.a.cassandra.service.StartupChecks  cassandra.jmx.local.port missing from cassandra-env.sh, unable to start local JMX service.
2022-09-24 15:24:45 ERROR o.a.c.service.CassandraDaemon  The native library could not be initialized properly. 
Exception in thread "main" java.lang.RuntimeException: Cassandra did not start within 45 seconds
...

Java 17.0.4

Exception (java.lang.AssertionError) encountered during startup: java.lang.reflect.InaccessibleObjectException: Unable to make field private int java.io.FileDescriptor.fd accessible: module java.base does not "opens java.io" to unnamed module @610b6fe2
2022-09-24 15:27:10 ERROR o.a.c.service.CassandraDaemon  Exception encountered during startup
java.lang.AssertionError: java.lang.reflect.InaccessibleObjectException: Unable to make field private int java.io.FileDescriptor.fd accessible: module java.base does not "opens java.io" to unnamed module @610b6fe2
	at org.apache.cassandra.utils.FBUtilities.getProtectedField(FBUtilities.java:608) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:80) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:196) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) ~[cassandra-bundle.jar:1.0.6]
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private int java.io.FileDescriptor.fd accessible: module java.base does not "opens java.io" to unnamed module @610b6fe2
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354) ~[na:na]
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297) ~[na:na]
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178) ~[na:na]
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172) ~[na:na]
	at org.apache.cassandra.utils.FBUtilities.getProtectedField(FBUtilities.java:603) ~[cassandra-bundle.jar:1.0.6]
	... 4 common frames omitted
Exception in thread "main" java.lang.RuntimeException: Cassandra did not start within 45 seconds
...

Java 18.0.2 / Java 19

Exception (java.lang.UnsupportedOperationException) encountered during startup: The Security Manager is deprecated and will be removed in a future release
2022-09-24 15:28:45 ERROR o.a.c.service.CassandraDaemon  Exception encountered during startup
java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
	at java.base/java.lang.System.setSecurityManager(System.java:416) ~[na:na]
	at org.apache.cassandra.security.ThreadAwareSecurityManager.install(ThreadAwareSecurityManager.java:75) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:192) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602) ~[cassandra-bundle.jar:1.0.6]
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691) ~[cassandra-bundle.jar:1.0.6]
java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
	at java.base/java.lang.System.setSecurityManager(System.java:416)
	at org.apache.cassandra.security.ThreadAwareSecurityManager.install(ThreadAwareSecurityManager.java:75)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:192)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:602)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:691)
Exception in thread "main" java.lang.RuntimeException: Cassandra did not start within 45 seconds
...

Relevant logs

If you identified a section in your logs that explains the bug or might be important to understand it, please add it.

Reproducible Test Case

Please provide a PR with a failing test.

If the issue is more complex or requires configuration, please provide a link to a project that reproduces the issue.

The text was updated successfully, but these errors were encountered:

johanandren · 2022-09-26T07:13:06Z

Looks like the JNA lib pulled in by Cassandra does not support arm64, this SO post suggests you could potentially bump the transitive JNA dependency to a newer version that does support arm64 so that might be worth trying out: https://stackoverflow.com/questions/69486339/nativelibrarydarwin-java64-failed-to-link-the-c-library-against-jna-native-m

Another workaround would be ro run a amd64 JDK 11 instead of a native aarch64 JDK.

yoppworks-andrew · 2022-09-27T01:59:48Z

This works fine when running a local install of Cassandra (for example using brew install cassandra, overwriting the JNA jar, running with brew services start cassandra, and then connecting via cqlsh), but I'm struggling to close the gap with getting that working vs. getting akka-persistence-cassandra running.

What sort of build.sbt (or other) overrides do I need in place to get this running with CassandraLauncher, for example?

patriknw · 2022-09-27T08:06:35Z

I would recommend using docker-compose instead of the CassandraLauncher. You can find example in https://developer.lightbend.com/docs/akka-platform-guide/how-to/cassandra-alternative.html#_run_locally

Testcontainers is another alternative, and you can find inspiration in https://github.com/akka/akka-projection/tree/main/akka-projection-cassandra

yoppworks-andrew · 2022-09-27T19:29:26Z

Unfortunately, moving away from CassandraLauncher isn't an option for me.

Tested this PR locally and it seems to work #988

yoppworks-andrew · 2022-09-28T13:05:41Z

For posterity, MWE showing how to manually work around this problem on M1 / ARM64 / AArch64 Macs: https://github.com/awwsmm/apc-mwe

You can generate your own cassandra-bundle.jar by running the cassandraBundle/assembly task in akka-persistence-cassandra, then use that jar in third-party applications that depend on akka-persistence-cassandra.

johanandren · 2023-09-27T15:17:42Z

CassandraLauncher now dropped in #1054, so I'll close this. Thanks for sharing your workaround @yoppworks-andrew

yoppworks-andrew changed the title ~~akka-persistence-cassandra fails to start on arm64 (aka. aarch64 / Mac M1) architecture~~ akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture Sep 24, 2022

johanandren closed this as completed Sep 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture #986

akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture #986

yoppworks-andrew commented Sep 24, 2022

johanandren commented Sep 26, 2022

yoppworks-andrew commented Sep 27, 2022

patriknw commented Sep 27, 2022

yoppworks-andrew commented Sep 27, 2022

yoppworks-andrew commented Sep 28, 2022

johanandren commented Sep 27, 2023

akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture #986

akka-persistence-cassandra-launcher fails to start on arm64 (aka. aarch64 / Mac M1) architecture #986

Comments

yoppworks-andrew commented Sep 24, 2022

Versions used

Expected Behavior

Actual Behavior

Java 8

Java 11.0.15 / 11.0.16

Java 17.0.4

Java 18.0.2 / Java 19

Relevant logs

Reproducible Test Case

johanandren commented Sep 26, 2022

yoppworks-andrew commented Sep 27, 2022

patriknw commented Sep 27, 2022

yoppworks-andrew commented Sep 27, 2022

yoppworks-andrew commented Sep 28, 2022

johanandren commented Sep 27, 2023