Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sbt doesn't start on arm64-macos #6162

Closed
catap opened this issue Nov 23, 2020 · 66 comments
Closed

sbt doesn't start on arm64-macos #6162

catap opened this issue Nov 23, 2020 · 66 comments
Labels
Milestone

Comments

@catap
Copy link

catap commented Nov 23, 2020

steps

  1. Download and install any arm64 JDK. I've used Azul JDK build 16-ea+24
  2. java -jar /opt/local/share/sbt/sbt-launch.jar

problem

/Users/catap/Library/Caches/JNA/temp/jna8864682821277708868.tmp: cannot open `/Users/catap/Library/Caches/JNA/temp/jna8864682821277708868.tmp' (No such file or directory)
catap@Kirills-mini-m1 scala-native % ~/zulu16.0.65-ea-jdk16.0.0-ea.24-macos_aarch64/bin/java -jar /opt/local/share/sbt/sbt-launch.jar
java.lang.UnsatisfiedLinkError: /Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: dlopen(/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp, 1): no suitable image found.  Did find:
	/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: no matching architecture in universal wrapper
	/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: no matching architecture in universal wrapper
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2383)
	at java.base/java.lang.Runtime.load0(Runtime.java:746)
	at java.base/java.lang.System.load(System.java:1857)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
	at com.sun.jna.Native.<clinit>(Native.java:195)
	at org.scalasbt.ipcsocket.UnixDomainSocketLibrary.<clinit>(UnixDomainSocketLibrary.java:129)
	at org.scalasbt.ipcsocket.JNAUnixDomainSocketLibraryProvider.socket(UnixDomainSocketLibrary.java:165)
	at org.scalasbt.ipcsocket.UnixDomainServerSocket.<init>(UnixDomainServerSocket.java:109)
	at org.scalasbt.ipcsocket.UnixDomainServerSocket.<init>(UnixDomainServerSocket.java:87)
	at sbt.internal.BootServerSocket.newSocket(BootServerSocket.java:342)
	at sbt.internal.BootServerSocket.<init>(BootServerSocket.java:295)
	at sbt.xMain$.getSocketOrExit(Main.scala:118)
	at sbt.xMain$.run(Main.scala:67)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:83)
	at sbt.xMain.run(Main.scala:46)
	at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
	at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
	at xsbt.boot.Launch$.run(Launch.scala:109)
	at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
	at xsbt.boot.Launch$.launch(Launch.scala:117)
	at xsbt.boot.Launch$.apply(Launch.scala:18)
	at xsbt.boot.Boot$.runImpl(Boot.scala:41)
	at xsbt.boot.Boot$.main(Boot.scala:17)
	at xsbt.boot.Boot.main(Boot.scala)
Error during sbt execution: java.lang.UnsatisfiedLinkError: /Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: dlopen(/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp, 1): no suitable image found.  Did find:
	/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: no matching architecture in universal wrapper
	/Users/catap/Library/Caches/JNA/temp/jna12016497044725394171.tmp: no matching architecture in universal wrapper

expectation

It should work.

notes

I've opened a linked ticket sbt/ipcsocket#10, anyway I feel that it might be fixed also in sbt some how.

@eed3si9n
Copy link
Member

@catap Thanks for the report. This is somewhat expected since modern sbt uses a lot of native code to access functionalities not provided by the JVM out-of-box. For this specific error, maybe you could try:

serverConnectionType := ConnectionType.Tcp,

In general though we should make sure to provide pure-Java fallbacks for all critical features.

@eatkins
Copy link
Contributor

eatkins commented Nov 23, 2020

This might be fixed by #6165 though I'm not sure where else we may be using the jna in transitive dependencies.

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins

I've tried 13b09bc as root that I've published as 1.5.0-SNAPSHOT.

It is crashed but at another place.

catap@Kirills-mini-m1 sbt % java -Dsbt.version=1.5.0-SNAPSHOT -jar /opt/local/share/sbt/sbt-launch.jar
[info] welcome to sbt 1.5.0-SNAPSHOT (Azul Systems, Inc. Java 16-ea)
[info] loading settings for project sbt-build from plugins.sbt ...
[info] loading project definition from /Users/catap/src/sbt/project
[info] compiling 11 Scala sources to /Users/catap/src/sbt/project/target/scala-2.12/sbt-1.0/classes ...
[info] Non-compiled module 'compiler-bridge_2.12' for Scala 2.12.12. Compiling...
[info]   Compilation completed in 4.319s.
[warn] /Users/catap/src/sbt/project/SbtLauncherPlugin.scala:50:10: method zip in object IO is deprecated (since 1.3.2): Please specify whether to use a static timestamp
[warn]       IO.zip(dir.allPaths.get().flatMap(rebase), target)
[warn]          ^
[warn] /Users/catap/src/sbt/project/Scripted.scala:111:37: method toLoader in object ClasspathUtilities is deprecated (since 1.4.0): internal
[warn]     val loader = ClasspathUtilities.toLoader(classpath, noJLine)
[warn]                                     ^
[warn] /Users/catap/src/sbt/project/Scripted.scala:111:18: object ClasspathUtilities in package classpath is deprecated (since 1.4.0): internal
[warn]     val loader = ClasspathUtilities.toLoader(classpath, noJLine)
[warn]                  ^
[warn] three warnings found
[info] loading settings for project sbtRoot from build.sbt ...
[info] resolving key references (35915 settings) ...
[info]            __    __ 
[info]      _____/ /_  / /_
[info]     / ___/ __ \/ __/
[info]    (__  ) /_/ / /_  
[info]   /____/_.___/\__/ 
[info] Welcome to the build for sbt.
[info] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[info]   Java version is 16. We recommend java 8.
[info] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Couldn't load native library java.lang.UnsatisfiedLinkError: /Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: dlopen(/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib, 1): no suitable image found.  Did find:
	/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: mach-o, but wrong architecture
	/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: stat() failed with errno=17
java.lang.ExceptionInInitializerError
	at com.swoval.files.apple.FileEventMonitorImpl.<clinit>(FileEventMonitors.java:130)
	at com.swoval.files.apple.FileEventMonitors.get(FileEventMonitors.java:47)
	at com.swoval.files.ApplePathWatcher.<init>(ApplePathWatcher.java:258)
	at com.swoval.files.ApplePathWatcher.<init>(ApplePathWatcher.java:194)
	at com.swoval.files.ApplePathWatchers.get(ApplePathWatcher.java:331)
	at com.swoval.files.PathWatchers.get(PathWatchers.java:84)
	at com.swoval.files.FileTreeRepositories.get(FileTreeRepositories.java:64)
	at com.swoval.files.FileTreeRepositories.get(FileTreeRepositories.java:32)
	at sbt.internal.nio.FileTreeRepositoryImpl.<init>(FileTreeRepositoryImpl.scala:46)
	at sbt.internal.nio.FileTreeRepository$.default(FileTreeRepository.scala:40)
	at sbt.BuiltinCommands$.$anonfun$setupGlobalFileTreeRepository$1(Main.scala:954)
	at sbt.BuiltinCommands$.doLoadProject(Main.scala:945)
	at sbt.BuiltinCommands$.$anonfun$loadProjectImpl$2(Main.scala:885)
	at sbt.Command$.$anonfun$applyEffect$4(Command.scala:150)
	at sbt.Command$.$anonfun$applyEffect$2(Command.scala:145)
	at sbt.Command$.process(Command.scala:189)
	at sbt.MainLoop$.$anonfun$processCommand$5(MainLoop.scala:242)
	at scala.Option.getOrElse(Option.scala:189)
	at sbt.MainLoop$.process$1(MainLoop.scala:242)
	at sbt.MainLoop$.processCommand(MainLoop.scala:268)
	at sbt.MainLoop$.$anonfun$next$5(MainLoop.scala:162)
	at sbt.State$StateOpsImpl$.runCmd$1(State.scala:289)
	at sbt.State$StateOpsImpl$.process$extension(State.scala:325)
	at sbt.MainLoop$.$anonfun$next$4(MainLoop.scala:162)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
	at sbt.MainLoop$.next(MainLoop.scala:162)
	at sbt.MainLoop$.run(MainLoop.scala:143)
	at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:118)
	at sbt.io.Using.apply(Using.scala:27)
	at sbt.MainLoop$.runWithNewLog(MainLoop.scala:111)
	at sbt.MainLoop$.runAndClearLast(MainLoop.scala:65)
	at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:50)
	at sbt.MainLoop$.runLogged(MainLoop.scala:41)
	at sbt.StandardMain$.runManaged(Main.scala:191)
	at sbt.xMain$.$anonfun$run$8(Main.scala:102)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withIn(Console.scala:230)
	at sbt.internal.util.Terminal$.withIn(Terminal.scala:557)
	at sbt.internal.util.Terminal$.$anonfun$withStreams$1(Terminal.scala:347)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withOut(Console.scala:167)
	at sbt.internal.util.Terminal$.$anonfun$withOut$2(Terminal.scala:547)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withErr(Console.scala:196)
	at sbt.internal.util.Terminal$.withOut(Terminal.scala:547)
	at sbt.internal.util.Terminal$.withStreams(Terminal.scala:347)
	at sbt.xMain$.run(Main.scala:86)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:83)
	at sbt.xMain.run(Main.scala:46)
	at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
	at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
	at xsbt.boot.Launch$.run(Launch.scala:109)
	at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
	at xsbt.boot.Launch$.launch(Launch.scala:117)
	at xsbt.boot.Launch$.apply(Launch.scala:18)
	at xsbt.boot.Boot$.runImpl(Boot.scala:41)
	at xsbt.boot.Boot$.main(Boot.scala:17)
	at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.lang.UnsatisfiedLinkError: /Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: dlopen(/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib, 1): no suitable image found.  Did find:
	/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: mach-o, but wrong architecture
	/Users/catap/.sbt/1.0/swoval-jni/8091807251952918964-libswoval-files0.dylib: stat() failed with errno=17
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2383)
	at java.base/java.lang.Runtime.load0(Runtime.java:746)
	at java.base/java.lang.System.load(System.java:1857)
	at com.swoval.runtime.NativeLoader.loadPackaged(NativeLoader.java:140)
	at com.swoval.runtime.NativeLoader.loadPackaged(NativeLoader.java:174)
	at com.swoval.files.apple.FileEventMonitorImpl.<clinit>(FileEventMonitors.java:127)
	... 61 more
[error] java.lang.ExceptionInInitializerError
[error] Use 'last' for the full log.

@eed3si9n
Copy link
Member

Created an issue on Swoval - swoval/swoval#141

@catap
Copy link
Author

catap commented Nov 24, 2020

I made small investigation and looks like sbt-1.0.x works fine.

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

@catap if you apply the following patches to your local sbt build, I'm hoping a snapshot version might startup:

diff --git a/build.sbt b/build.sbt
index 443a3661b..2995c87f4 100644
--- a/build.sbt
+++ b/build.sbt
@@ -917,6 +917,7 @@ lazy val mainProj = (project in file("main"))
       case v if v.startsWith("2.12.") => List(compilerPlugin(silencerPlugin))
       case _                          => List()
     }),
+    libraryDependencies += "com.swoval" % "file-tree-views" % "2.1.6",
     managedSourceDirectories in Compile +=
       baseDirectory.value / "src" / "main" / "contraband-scala",
     sourceManaged in (Compile, generateContrabands) := baseDirectory.value / "src" / "main" / "contraband-scala",
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index 2dce163ae..51b43f482 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -26,7 +26,7 @@ object Dependencies {
   val launcherInterface = "org.scala-sbt" % "launcher-interface" % launcherVersion
   val rawLauncher = "org.scala-sbt" % "launcher" % launcherVersion
   val testInterface = "org.scala-sbt" % "test-interface" % "1.0"
-  val ipcSocket = "org.scala-sbt.ipcsocket" % "ipcsocket" % "1.1.0"
+  val ipcSocket = "org.scala-sbt.ipcsocket" % "ipcsocket" % "1.2.0"
 
   private val compilerInterface = "org.scala-sbt" % "compiler-interface" % zincVersion
   private val compilerClasspath = "org.scala-sbt" %% "zinc-classpath" % zincVersion

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins patch doesn't work, it crashed as it was at #6162 (comment).

I did small investigation to proof that the first chunk at the patch isn't enough:

catap@Kirills-mini sbt % rm -rf ~/.ivy
catap@Kirills-mini sbt % git diff
diff --git a/build.sbt b/build.sbt
index 443a3661b..2995c87f4 100644
--- a/build.sbt
+++ b/build.sbt
@@ -917,6 +917,7 @@ lazy val mainProj = (project in file("main"))
       case v if v.startsWith("2.12.") => List(compilerPlugin(silencerPlugin))
       case _                          => List()
     }),
+    libraryDependencies += "com.swoval" % "file-tree-views" % "2.1.6",
     managedSourceDirectories in Compile +=
       baseDirectory.value / "src" / "main" / "contraband-scala",
     sourceManaged in (Compile, generateContrabands) := baseDirectory.value / "src" / "main" / "contraband-scala",
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index 2dce163ae..51b43f482 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -26,7 +26,7 @@ object Dependencies {
   val launcherInterface = "org.scala-sbt" % "launcher-interface" % launcherVersion
   val rawLauncher = "org.scala-sbt" % "launcher" % launcherVersion
   val testInterface = "org.scala-sbt" % "test-interface" % "1.0"
-  val ipcSocket = "org.scala-sbt.ipcsocket" % "ipcsocket" % "1.1.0"
+  val ipcSocket = "org.scala-sbt.ipcsocket" % "ipcsocket" % "1.2.0"
 
   private val compilerInterface = "org.scala-sbt" % "compiler-interface" % zincVersion
   private val compilerClasspath = "org.scala-sbt" %% "zinc-classpath" % zincVersion
catap@Kirills-mini sbt % sbt publishLocal > /dev/null
catap@Kirills-mini sbt % ls ~/.ivy2/cache/com.swoval/file-tree-views/jars
file-tree-views-2.0.2.jar	file-tree-views-2.1.1.jar	file-tree-views-2.1.3.jar	file-tree-views-2.1.4.jar
catap@Kirills-mini sbt % java -version
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_275-b01)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.275-b01, mixed mode)
catap@Kirills-mini sbt %  

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

It wouldn’t be in the .ivy cache. It would be in the coursier cache.

That was mistaken. I am confused though because the patch definitely works for me. I published a local version with a snapshot version of swoval that added a println and I did see a println when I ran sbt.

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins anyway, I suprised that it has 4 different versions. Not two for sbt that I run and that I build.

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

What do you see when you run ls ~/.sbt/boot/scala-2.12/org.scala-sbt/sbt/1.5.0-SNAPSHOT? On my computer, I see the file-tree-views-2.1.6.jar. You may want to try removing ~/.sbt/boot and trying again. The cached jars in there can cause problems.

@catap
Copy link
Author

catap commented Nov 24, 2020

I move by easy way when: rm -rf ~/.sbt.

It allows to start sbt, but when I tried to load sbt current master plus your patch it throw an exception.

Anyway, it do not crash. What is very good news.

[info] welcome to sbt 1.5.0-SNAPSHOT (Azul Systems, Inc. Java 1.8.0_275)
[info] loading settings for project sbt-build from plugins.sbt ...
[info] loading project definition from /Users/catap/src/sbt/project
[info] compiling 11 Scala sources to /Users/catap/src/sbt/project/target/scala-2.12/sbt-1.0/classes ...
[info] Non-compiled module 'compiler-bridge_2.12' for Scala 2.12.12. Compiling...
[info]   Compilation completed in 5.54s.
[warn] /Users/catap/src/sbt/project/SbtLauncherPlugin.scala:50:10: method zip in object IO is deprecated (since 1.3.2): Please specify whether to use a static timestamp
[warn]       IO.zip(dir.allPaths.get().flatMap(rebase), target)
[warn]          ^
[warn] /Users/catap/src/sbt/project/Scripted.scala:111:37: method toLoader in object ClasspathUtilities is deprecated (since 1.4.0): internal
[warn]     val loader = ClasspathUtilities.toLoader(classpath, noJLine)
[warn]                                     ^
[warn] /Users/catap/src/sbt/project/Scripted.scala:111:18: object ClasspathUtilities in package classpath is deprecated (since 1.4.0): internal
[warn]     val loader = ClasspathUtilities.toLoader(classpath, noJLine)
[warn]                  ^
[warn] three warnings found
[info] loading settings for project sbtRoot from build.sbt ...
[info] resolving key references (35916 settings) ...
[info]            __    __ 
[info]      _____/ /_  / /_
[info]     / ___/ __ \/ __/
[info]    (__  ) /_/ / /_  
[info]   /____/_.___/\__/ 
[info] Welcome to the build for sbt.
Exception in thread "sbt-socket-server" java.lang.UnsatisfiedLinkError: /Users/catap/Library/Caches/JNA/temp/jna8776389905734795477.tmp: dlopen(/Users/catap/Library/Caches/JNA/temp/jna8776389905734795477.tmp, 1): no suitable image found.  Did find:
	/Users/catap/Library/Caches/JNA/temp/jna8776389905734795477.tmp: no matching architecture in universal wrapper
	/Users/catap/Library/Caches/JNA/temp/jna8776389905734795477.tmp: no matching architecture in universal wrapper
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
	at java.lang.Runtime.load0(Runtime.java:811)
	at java.lang.System.load(System.java:1088)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
	at com.sun.jna.Native.<clinit>(Native.java:195)
	at com.sun.jna.Structure.setAlignType(Structure.java:280)
	at com.sun.jna.Structure.<init>(Structure.java:197)
	at com.sun.jna.Structure.<init>(Structure.java:193)
	at com.sun.jna.Structure.<init>(Structure.java:180)
	at com.sun.jna.Structure.<init>(Structure.java:172)
	at org.scalasbt.ipcsocket.UnixDomainSocketLibrary$SockaddrUn.<init>(UnixDomainSocketLibrary.java:88)
	at sbt.internal.server.Server$$anon$1$$anon$2.$anonfun$run$1(Server.scala:74)
	at scala.util.Try$.apply(Try.scala:213)
	at sbt.internal.server.Server$$anon$1$$anon$2.run(Server.scala:63)
[warn] sbt server could not start in 10s
sbt:sbtRoot> 

@catap
Copy link
Author

catap commented Nov 24, 2020

I guess we almost here, in the world where sbt works on M1 :)

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

Great. It looks like we need one more small ipcsocket fix to avoid that particular crash. In the meantime, you should be able to workaround that with Global / serverConnectionType := ConnectionType.Tcp.

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins let make it right ;)

It doesn't block me because this machine was bought special for experiments and it makes tons of fun for last week :)

@eed3si9n
Copy link
Member

eed3si9n commented Nov 24, 2020

Yea. The check introduced in #3932 is using JNA? Maybe we can hardcode some known max length when JNI is used.

https://linux.die.net/man/7/unix

A UNIX domain socket address is represented in the following structure:

#define UNIX_PATH_MAX    108

struct sockaddr_un {
    sa_family_t sun_family;               /* AF_UNIX */
    char        sun_path[UNIX_PATH_MAX];  /* pathname */
};

macOS / BSD uses 104 characters - https://unix.stackexchange.com/questions/367008/why-is-socket-path-length-limited-to-a-hundred-chars

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

It doesn't block me because this machine was bought special for experiments and it makes tons of fun for last week :)

These experiments will prove useful for lots of future users!

@catap
Copy link
Author

catap commented Nov 24, 2020

@eed3si9n it is very bad idea to hardcode this value. Very bad.

@eed3si9n
Copy link
Member

If Linux or macOS already hardcoded it in struct sockaddr_un, in the grand scheme of things, I don't think it's in the very bad category. If we were coding this in C, it would be in the header file. I'd almost advocate we use 104 regardless of JNA or JNI since this is only a validation to display better error message when domain socket is created in a deeply nested directory.

@catap
Copy link
Author

catap commented Nov 24, 2020

@eed3si9n yes, this is header and it is used on compilation time.

Can it be changed? Yes, and SBT should follow this changes that is quite difficult to do.

From another hand I feel that I miss this constant with PATH_MAX that isn't so similar at different system and can be 4096, 255 or 1023.

I made a small grep on all available systems and looks like you right: 108 is standard value.

@catap
Copy link
Author

catap commented Nov 24, 2020

BTW with set Global / serverConnectionType := ConnectionType.Tcp I can rebuild and pass all tests from sbt!

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

sbt/ipcsocket#13

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

@catap now that you have the server running, I am curious if the thin client works. Assuming you have the sbt script installed, does sbt --client work? Or you could try the apple binary from https://github.com/sbt/sbtn-dist/releases/download/v1.4.4/sbtn-x86_64-apple-darwin-1.4.4.tar.gz. I think it's supposed to work with rosetta since it's an executable (I think dlopen doesn't work with rosetta which was why we had to publish new versions of swoval and ipcsocket) but it would be nice to have confirmation (apparently a graal binary should work: oracle/graal#2666.).

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins I assume that --client is covered via unit test?

Anyway, if you can provide a way to start it via java -jar ... I glade to test it.

Rosseta is quite tricky. It depends on version of java. If java is installed that is for x86_86 => sbt works. If java is build nativly for M1/arm64 => the hell that I explains here.

@catap
Copy link
Author

catap commented Nov 24, 2020

@eatkins about --client:

catap@Kirills-mini-m1 sbt % java -Dsbt.version=1.5.0-SNAPSHOT -jar /opt/local/share/sbt/sbt-launch.jar --client
[info] entering *experimental* thin client - BEEP WHIRR
java.lang.UnsatisfiedLinkError: /Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: dlopen(/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp, 1): no suitable image found.  Did find:
	/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: no matching architecture in universal wrapper
	/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: no matching architecture in universal wrapper
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
	at java.lang.Runtime.load0(Runtime.java:811)
	at java.lang.System.load(System.java:1088)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
	at com.sun.jna.Native.<clinit>(Native.java:195)
	at org.scalasbt.ipcsocket.UnixDomainSocketLibrary.<clinit>(UnixDomainSocketLibrary.java:129)
	at org.scalasbt.ipcsocket.JNAUnixDomainSocketLibraryProvider.socket(UnixDomainSocketLibrary.java:165)
	at org.scalasbt.ipcsocket.UnixDomainSocket.<init>(UnixDomainSocket.java:53)
	at sbt.protocol.ClientSocket$.localSocket(ClientSocket.scala:46)
	at sbt.internal.client.NetworkClient.$anonfun$waitForServer$1(NetworkClient.scala:314)
	at scala.util.Try$.apply(Try.scala:213)
	at sbt.internal.client.NetworkClient.waitForServer(NetworkClient.scala:314)
	at sbt.internal.client.NetworkClient.connectOrStartServerAndConnect(NetworkClient.scala:189)
	at sbt.internal.client.NetworkClient.init(NetworkClient.scala:223)
	at sbt.internal.client.NetworkClient.connect(NetworkClient.scala:719)
	at sbt.internal.client.NetworkClient$.clientImpl(NetworkClient.scala:1114)
	at sbt.internal.client.NetworkClient$.run(NetworkClient.scala:1232)
	at sbt.internal.client.NetworkClient$.run(NetworkClient.scala:1221)
	at sbt.xMain$.$anonfun$run$8(Main.scala:88)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withIn(Console.scala:230)
	at sbt.internal.util.Terminal$.withIn(Terminal.scala:557)
	at sbt.internal.util.Terminal$.$anonfun$withStreams$1(Terminal.scala:347)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withOut(Console.scala:167)
	at sbt.internal.util.Terminal$.$anonfun$withOut$2(Terminal.scala:547)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at scala.Console$.withErr(Console.scala:196)
	at sbt.internal.util.Terminal$.withOut(Terminal.scala:547)
	at sbt.internal.util.Terminal$.withStreams(Terminal.scala:347)
	at sbt.xMain$.run(Main.scala:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sbt.internal.XMainConfiguration.run(XMainConfiguration.scala:83)
	at sbt.xMain.run(Main.scala:46)
	at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
	at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
	at xsbt.boot.Launch$.run(Launch.scala:109)
	at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
	at xsbt.boot.Launch$.launch(Launch.scala:117)
	at xsbt.boot.Launch$.apply(Launch.scala:18)
	at xsbt.boot.Boot$.runImpl(Boot.scala:41)
	at xsbt.boot.Boot$.main(Boot.scala:17)
	at xsbt.boot.Boot.main(Boot.scala)
Error during sbt execution: java.lang.UnsatisfiedLinkError: /Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: dlopen(/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp, 1): no suitable image found.  Did find:
	/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: no matching architecture in universal wrapper
	/Users/catap/Library/Caches/JNA/temp/jna4531627309130879234.tmp: no matching architecture in universal wrapper
catap@Kirills-mini-m1 sbt %

@eatkins
Copy link
Contributor

eatkins commented Nov 24, 2020

@eatkins I assume that --client is covered via unit test?

Not really.

Anyway, if you can provide a way to start it via java -jar ... I glade to test it.

The client mode allows you to connect to an already running server to issue commands. This allows you to run sbt commands from the shell without having to start a new server, which we all know is very slow. You can see what it looks like here https://twitter.com/eed3si9n/status/1302754076841648134 and here #5620. The steps to test that don't involve the sbt script are

  1. cd to the project directory where you're going to test sbt
  2. download and extract https://github.com/sbt/sbtn-dist/releases/download/v1.4.4/sbtn-x86_64-apple-darwin-1.4.4.tar.gz this will give you a binary called sbtn
  3. start your server with java -Dsbt.version=1.5.0-SNAPSHOT as before
  4. in a separate terminal window, go to the same directory as the sbt project where the server is running
  5. run sbtn (or sbtn test or whatever command you want to run)

@catap
Copy link
Author

catap commented Nov 26, 2020

@eatkins am I right that current master should work without hack?

@eatkins
Copy link
Contributor

eatkins commented Nov 26, 2020

Not quite. The current master needs a new version of https://github.com/sbt/io which I don't think has been published yet. Because of that, you would still need the hack to add

diff --git a/build.sbt b/build.sbt
index 443a3661b..2995c87f4 100644
--- a/build.sbt
+++ b/build.sbt
@@ -917,6 +917,7 @@ lazy val mainProj = (project in file("main"))
       case v if v.startsWith("2.12.") => List(compilerPlugin(silencerPlugin))
       case _                          => List()
     }),
+    libraryDependencies += "com.swoval" % "file-tree-views" % "2.1.6",
     managedSourceDirectories in Compile +=
       baseDirectory.value / "src" / "main" / "contraband-scala",
     sourceManaged in (Compile, generateContrabands) := baseDirectory.value / "src" / "main" / "contraband-scala",

@catap
Copy link
Author

catap commented Nov 26, 2020

@eatkins let's wait when until everything is ready :)

Feel free ping me here to confirm that it is working from master branch.

@catap
Copy link
Author

catap commented Nov 26, 2020

@eatkins I just realised that master is 1.5.0. Is 1.4.x branch contains all required fixes?

@eatkins
Copy link
Contributor

eatkins commented Nov 26, 2020

Yeah, 1.4.x has all of the required fixes.

@catap
Copy link
Author

catap commented Nov 26, 2020

@eatkins yes, 1.4.3-SNAPSHOT from 1a1cca3 working very well without any patches.

I guess we can close this bug.

Congrats!

@eed3si9n eed3si9n added this to the 1.4.5 milestone Nov 26, 2020
@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n any plan to release 1.4.5?

@eatkins
Copy link
Contributor

eatkins commented Nov 27, 2020

For the 1.4.x series, we have been releasing on a roughly weekly to biweekly schedule. We just made a new release last weekend so I would imagine this release should be out at worse within the next 10-14 days.

@eed3si9n
Copy link
Member

Not sure how /opt/local/share/sbt/sbt-launch.jar was installed, and why it's not resolving but in theory but nightly build is exactly for this kind of purpose.

@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n it was installed via macports and it is sbt-launch.jar from 0.13.

I'm waiting until you released 1.4.5 to updated sbt at macports.

@eed3si9n
Copy link
Member

Could you try the launcher in sbt 1.4.4? https://github.com/sbt/sbt/releases/tag/v1.4.4 That should be able to launch both nightlies and 0.13 builds.

@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n

catap@Kirills-mini-m1 sbt-test % wget https://repo1.maven.org/maven2/org/scala-sbt/sbt-launch/1.4.4/sbt-launch-1.4.4.jar
--2020-11-27 20:01:42--  https://repo1.maven.org/maven2/org/scala-sbt/sbt-launch/1.4.4/sbt-launch-1.4.4.jar
Resolving repo1.maven.org (repo1.maven.org)... 199.232.196.209, 199.232.192.209
Connecting to repo1.maven.org (repo1.maven.org)|199.232.196.209|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1393102 (1.3M) [application/java-archive]
Saving to: ‘sbt-launch-1.4.4.jar’

sbt-launch-1.4.4.jar                          100%[==============================================================================================>]   1.33M  5.79MB/s    in 0.2s    

2020-11-27 20:01:42 (5.79 MB/s) - ‘sbt-launch-1.4.4.jar’ saved [1393102/1393102]

catap@Kirills-mini-m1 sbt-test % java -Dsbt.version=-Dsbt.version=1.5.0-20201127T081401 -jar ./sbt-launch-1.4.4.jar 
[info] [launcher] getting org.scala-sbt sbt -Dsbt.version=1.5.0-20201127T081401  (this may take some time)...

:: problems summary ::
:::: WARNINGS
		module not found: org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401

	==== local: tried

	  /Users/catap/.ivy2/local/org.scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/ivys/ivy.xml

	  -- artifact org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401!sbt.jar:

	  /Users/catap/.ivy2/local/org.scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/jars/sbt.jar

	==== local-preloaded-ivy: tried

	  file:////Users/catap/.sbt/preloaded/org.scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/ivys/ivy.xml

	==== local-preloaded: tried

	  file:////Users/catap/.sbt/preloaded/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401!sbt.jar:

	  file:////Users/catap/.sbt/preloaded/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.jar

	==== Maven Central: tried

	  https://repo1.maven.org/maven2/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401!sbt.jar:

	  https://repo1.maven.org/maven2/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.jar

	==== sbt-maven-releases: tried

	  https://repo.scala-sbt.org/scalasbt/maven-releases/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401!sbt.jar:

	  https://repo.scala-sbt.org/scalasbt/maven-releases/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.jar

	==== sbt-maven-snapshots: tried

	  https://repo.scala-sbt.org/scalasbt/maven-snapshots/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401!sbt.jar:

	  https://repo.scala-sbt.org/scalasbt/maven-snapshots/org/scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/sbt--Dsbt.version=1.5.0-20201127T081401.jar

	==== typesafe-ivy-releases: tried

	  https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/ivys/ivy.xml

	==== sbt-ivy-snapshots: tried

	  https://repo.scala-sbt.org/scalasbt/ivy-snapshots/org.scala-sbt/sbt/-Dsbt.version=1.5.0-20201127T081401/ivys/ivy.xml

		::::::::::::::::::::::::::::::::::::::::::::::

		::          UNRESOLVED DEPENDENCIES         ::

		::::::::::::::::::::::::::::::::::::::::::::::

		:: org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401: not found

		::::::::::::::::::::::::::::::::::::::::::::::



:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
[error] [launcher] unresolved dependency: org.scala-sbt#sbt;-Dsbt.version=1.5.0-20201127T081401: not found
[error] [launcher] error during sbt launcher: error retrieving required libraries
  (see /Users/catap/.sbt/boot/update.log for complete log)
[error] [launcher] could not retrieve sbt -Dsbt.version=1.5.0-20201127T081401
catap@Kirills-mini-m1 sbt-test % 

@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n and the say issue happened if I use sbt/bin/sbt-launch.jar from github archive as it expected.

@eed3si9n
Copy link
Member

I think you specified -Dsbt.version twice: java -Dsbt.version=-Dsbt.version=1.5.0-20201127T081401, but it's hitting the right repositories, so it's getting closer.

@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n I definitely need a wine!

Anyway, without a typo it doesn't work neither.

catap@Kirills-mini-m1 sbt-test % java -Dsbt.version=1.5.0-20201127T081401 -jar ./sbt-launch-1.4.4.jar  
[info] [launcher] getting org.scala-sbt sbt 1.5.0-20201127T081401  (this may take some time)...

:: problems summary ::
:::: WARNINGS
		module not found: org.scala-sbt#sbt;1.5.0-20201127T081401

	==== local: tried

	  /Users/catap/.ivy2/local/org.scala-sbt/sbt/1.5.0-20201127T081401/ivys/ivy.xml

	  -- artifact org.scala-sbt#sbt;1.5.0-20201127T081401!sbt.jar:

	  /Users/catap/.ivy2/local/org.scala-sbt/sbt/1.5.0-20201127T081401/jars/sbt.jar

	==== local-preloaded-ivy: tried

	  file:////Users/catap/.sbt/preloaded/org.scala-sbt/sbt/1.5.0-20201127T081401/ivys/ivy.xml

	==== local-preloaded: tried

	  file:////Users/catap/.sbt/preloaded/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;1.5.0-20201127T081401!sbt.jar:

	  file:////Users/catap/.sbt/preloaded/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.jar

	==== Maven Central: tried

	  https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;1.5.0-20201127T081401!sbt.jar:

	  https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.jar

	==== sbt-maven-releases: tried

	  https://repo.scala-sbt.org/scalasbt/maven-releases/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;1.5.0-20201127T081401!sbt.jar:

	  https://repo.scala-sbt.org/scalasbt/maven-releases/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.jar

	==== sbt-maven-snapshots: tried

	  https://repo.scala-sbt.org/scalasbt/maven-snapshots/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.pom

	  -- artifact org.scala-sbt#sbt;1.5.0-20201127T081401!sbt.jar:

	  https://repo.scala-sbt.org/scalasbt/maven-snapshots/org/scala-sbt/sbt/1.5.0-20201127T081401/sbt-1.5.0-20201127T081401.jar

	==== typesafe-ivy-releases: tried

	  https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt/1.5.0-20201127T081401/ivys/ivy.xml

	==== sbt-ivy-snapshots: tried

	  https://repo.scala-sbt.org/scalasbt/ivy-snapshots/org.scala-sbt/sbt/1.5.0-20201127T081401/ivys/ivy.xml

		::::::::::::::::::::::::::::::::::::::::::::::

		::          UNRESOLVED DEPENDENCIES         ::

		::::::::::::::::::::::::::::::::::::::::::::::

		:: org.scala-sbt#sbt;1.5.0-20201127T081401: not found

		::::::::::::::::::::::::::::::::::::::::::::::



:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
[error] [launcher] unresolved dependency: org.scala-sbt#sbt;1.5.0-20201127T081401: not found
[error] [launcher] error during sbt launcher: error retrieving required libraries
  (see /Users/catap/.sbt/boot/update.log for complete log)
[error] [launcher] could not retrieve sbt 1.5.0-20201127T081401
catap@Kirills-mini-m1 sbt-test % 

@eed3si9n
Copy link
Member

@catap
Copy link
Author

catap commented Nov 27, 2020

@eed3si9n 🙈

Yes, -Dsbt.version=1.5.0-bin-20201127T081401 works well with sbt-launch-1.4.4.

@eed3si9n
Copy link
Member

Awesome. Thanks for testing!

@timperrett
Copy link

Just chiming in to say thanks for doing this work folks, very timely. The following works perfectly compiling cats source for me 🙏

sbt -Dsbt.version=1.5.0-bin-20201127T081401

@hrj
Copy link

hrj commented Dec 14, 2020

Sorry for hijacking this thread, but:

In general though we should make sure to provide pure-Java fallbacks for all critical features.

  • Can I create an issue to track this?
  • Can you please enable discussions for this repo? Then we can have a longer discussion without spamming the issue tracker.

@eed3si9n
Copy link
Member

@hrj enabled Discussions.

@shi27feng
Copy link

Hi,
I got the same issue.
Would it be possible that anyone makes a summary of the steps for solving the issue? I got lost while went through the discussion.
Thank you so much!

@eed3si9n
Copy link
Member

eed3si9n commented Feb 6, 2021

The support for Apple silicon M1 (AArch64, ARM64) was added in sbt 1.4.5. In general you should use the latest stable version, which is 1.4.7 now. The version of sbt is determined both by the launcher and the project-specific declaration in project/build.properties. That's the summary.

@sbt sbt locked as resolved and limited conversation to collaborators Feb 6, 2021
skress referenced this issue in Homebrew/homebrew-core Jun 5, 2021
@eed3si9n eed3si9n added the area/arm_mac ARM (M1/M2) Mac issues label Aug 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

6 participants