Unable to compile under MacOS Big Sur

[danutsoare]

The following error is generated when compile from source under MacOS Big Sur:

$ make -C salma-hayek
Compiling Java source...
rm -f -r classes && rm -f -r .generated/classes && mkdir -p .generated/classes
/usr/Commands/javac -classpath /Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/classes -encoding UTF-8 -d .generated/classes/ -sourcepath src/ -g -deprecation -Xlint:all -Xlint:-serial -target 8 -source 8 -bootclasspath /Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home/jre/lib/rt.jar $(JAVA_SOURCE_FILES)
/bin/sh: /usr/Commands/javac: No such file or directory
make[1]: *** [/Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/java.build-finished] Error 127
make: *** [recurse] Error 1

I've tried to create the missing folder(as root) but it seems that I don't have enough privileges!

sudo mkdir -p /usr/Commands
mkdir: /usr/Commands: Operation not permitted

[philjessies]

Hi. "javac" is the Java compiler. Do you have a JDK (Java Development Kit) installed on your machine? That should include javac, among other development tools. If you do, maybe it's installing javac in a different location? I can't check myself, as I don't have a mac. Cheers, Phil

…

On Thu, 24 Dec 2020 at 12:44, Danut Soare ***@***.***> wrote: The following error is generated when compile from source under MacOS Big Sur: $ make -C salma-hayek Compiling Java source... rm -f -r classes && rm -f -r .generated/classes && mkdir -p .generated/classes /usr/Commands/javac -classpath /Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/classes -encoding UTF-8 -d .generated/classes/ -sourcepath src/ -g -deprecation -Xlint:all -Xlint:-serial -target 8 -source 8 -bootclasspath /Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home/jre/lib/rt.jar $(JAVA_SOURCE_FILES) /bin/sh: /usr/Commands/javac: No such file or directory make[1]: *** [/Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/java.build-finished] Error 127 make: *** [recurse] Error 1 I've tried to create the missing folder(as root) but it seems that I don't have enough privileges! sudo mkdir -p /usr/Commands mkdir: /usr/Commands: Operation not permitted — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#45>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6TQTPLV72IWGD5WLFGTMDSWMSQ7ANCNFSM4VIDZSDQ> .

[danutsoare]

Hi Phil,

I do have JDK kit installed and JAVA_HOME also set. It seems to point to other location than script wants.

danut@Danuts-MacBook-Pro $ which javac
/usr/bin/javac
 danut@Danuts-MacBook-Pro $ javac -version
javac 1.8.0_271

I will try to find the file that point to the wrong location and change it.

Regards,
Danut

[danutsoare]

Hi,

I've solve it with: export JAVA_COMPILER=/usr/bin/javac but now I get another error:

/Users/danut/Downloads/terminator/jessies/salma-hayek/native/Headers/PortableJni.h:13:10: fatal error: 'jni.h' file not found
#include <jni.h>
         ^~~~~~~
1 error generated.

BR,
Danut

[martindorey]

I've tried to create the missing folder

Yikes. It wasn't intended that you do that.

JAVA_HOME also set

It wasn't intended that you do that.

jessies/salma-hayek/bin/find-jdk-root.rb

Line 46 in 9612492

# * Using $JAVA_HOME -- as was common practice in Java 1.1 days -- requires

refers to that as a deprecated convention from the Java 1.1 days.

I've solve it with: export JAVA_COMPILER=/usr/bin/javac

I don't think that's the first place that our code has taken a wrong turning, so I wouldn't do that. From your output, it seems that find-jdk-root.rb found /usr/bin/javac, stripped off the last two components and returned /usr, just like it would on eg the more familiar-to-us Linux. Then

jessies/salma-hayek/lib/build/universal.make

Line 151 in 9612492

JDK_BIN_DIR.Darwin = Commands

did what looks like the wrong thing for your system. I'd have started by commenting that line out. I doubt it's the right solution but it might help to reveal what is.

The assumption for macOS at

jessies/salma-hayek/lib/build/universal.make

Line 147 in 9612492

JDK_INCLUDE.Darwin = /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/

seems to be that eg jni.h will be in:

JDK_INCLUDE.Darwin = /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/

I wonder where yours is instead. I doubt it's in /usr/include, which is what would be assumed if you commented that line out too, because that would have Just Worked already. If you conquer that barrier, I expect you'll have another, looking for jni_md.h, which is handled for other platforms with EXTRA_INCLUDE_PATH.$(TARGET_OS) assignments starting at:

jessies/salma-hayek/lib/build/universal.make

Line 326 in 9612492

EXTRA_INCLUDE_PATH.SunOS += $(JDK_INCLUDE)/solaris

Last we had a record of it working in the revision history of that file suggests it was macOS 10.14 == Mojave.

jessies/salma-hayek/lib/build/universal.make

Line 551 in 9612492

BOOT_JDK.Darwin = $(shell /usr/libexec/java_home -v $(patsubst 8,1.8,$(JAVA_MAJOR_VERSION)))

appears to run "/usr/libexec/java_home -v 1.8". https://developer.apple.com/forums/thread/666681 says that, if you've defined JAVA_HOME, then that program will break but, if you haven't, it might produce something like /Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home, which might be what we really need JDK_ROOT set to. What does ls -l /usr/bin/javac say? For an ancient Mac that I have access to, it says it's a symlink:

Ceto:~ ukdev$ ls -l /usr/bin/javac
lrwxr-xr-x 1 root wheel 75 May 19 2017 /usr/bin/javac -> /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/javac
Ceto:~ ukdev$

... where:

/System/Library/Frameworks/JavaVM.framework/Versions/Current -> A

[danutsoare]

Hi,
First piece of information:

find /Library/ -name "jni.h"
/Library//Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/jni.h
/Library//Java/JavaVirtualMachines/jdk-11.0.9.jdk/Contents/Home/include/jni.h
/Library//Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home/include/jni.h

I've used the JDK version 1.8.
The second one:

ls -l /usr/bin/javac
-rwxr-xr-x  1 root  wheel  138848 Jan  1  2020 /usr/bin/javac

Using Java 11 gives me the same message:

 danut@Danuts-MacBook-Pro  ~/Downloads/terminator/jessies   master  make -C salma-hayek
/Users/danut/Downloads/terminator/jessies/salma-hayek/lib/build/universal.make:349: Could not find /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/ - perhaps the first java on your PATH isn't in a JDK
c++ -std=c++11 -Wno-long-long -D__STDC_LIMIT_MACROS -fno-omit-frame-pointer -fno-strict-aliasing -fPIC -g -W -Wall -Wshadow -Werror -mmacosx-version-min=10.9 -stdlib=libc++ -Wno-error=deprecated-declarations  -pedantic -x objective-c++ -I/Users/danut/Downloads/terminator/jessies/salma-hayek/native/Headers -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/   -c -o /Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/native/all/java-launcher/universal_Darwin/java-launcher.o /Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/native/all/java-launcher/universal_Darwin/java-launcher.cpp
In file included from /Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/native/all/java-launcher/universal_Darwin/java-launcher.cpp:15:
In file included from /Users/danut/Downloads/terminator/jessies/salma-hayek/native/Headers/JniError.h:4:
/Users/danut/Downloads/terminator/jessies/salma-hayek/native/Headers/PortableJni.h:13:10: fatal error: 'jni.h' file not found
#include <jni.h>
         ^~~~~~~
1 error generated.
make[1]: *** [/Users/danut/Downloads/terminator/jessies/salma-hayek/.generated/native/all/java-launcher/universal_Darwin/java-launcher.o] Error 1
make: *** [recurse] Error 1
 ✘ danut@Danuts-MacBook-Pro  ~/Downloads/terminator/jessies   master  java -version
java version "11.0.9" 2020-10-20 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.9+7-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.9+7-LTS, mixed mode)
 danut@Danuts-MacBook-Pro  ~/Downloads/terminator/jessies   master  javac -version
javac 11.0.9
 danut@Danuts-MacBook-Pro  ~/Downloads/terminator/jessies   master  echo $JAVA_HOME

 danut@Danuts-MacBook-Pro  ~/Downloads/terminator/jessies   master 

Thank you!
Dan

[enh-google]

this works for me (though i now don't have access to an older macOS if you want to check it still works there):

diff --git a/salma-hayek/bin/find-jdk-root.rb b/salma-hayek/bin/find-jdk-root.rb
index 3707becd..a4b2feca 100755
--- a/salma-hayek/bin/find-jdk-root.rb
+++ b/salma-hayek/bin/find-jdk-root.rb
@@ -156,6 +156,11 @@ else
       return `java-config --jdk-home`
     end
     
+    modern_mac_dir = "/Library/Java/JavaVirtualMachines/default/Contents/Home"
+    if File.exist?(modern_mac_dir)
+      return modern_mac_dir
+    end
+
     # On FreeBSD, the 'javac' family binaries are actually symlinks in /usr/local/bin/,
     # which point to the script /usr/local/bin/javavm. The actual java version will be
     # in /usr/local/openjdk<num>.
diff --git a/salma-hayek/lib/build/universal.make b/salma-hayek/lib/build/universal.make
index f442fa79..2e345a4e 100644
--- a/salma-hayek/lib/build/universal.make
+++ b/salma-hayek/lib/build/universal.make
@@ -143,13 +143,8 @@ JDK_ROOT_SCRIPT = $(SCRIPT_PATH)/find-jdk-root.rb
 SCRIPTS_WHICH_AFFECT_COMPILER_FLAGS += $(JDK_ROOT_SCRIPT)
 JDK_ROOT := $(call findMakeFriendlyEquivalentName,$(shell ruby $(JDK_ROOT_SCRIPT)))
 
-JDK_INCLUDE.$(TARGET_OS) = $(JDK_ROOT)/include
-JDK_INCLUDE.Darwin = /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/JavaVM.framework/Versions/A/Headers/
-JDK_INCLUDE = $(JDK_INCLUDE.$(TARGET_OS))
-
-JDK_BIN_DIR.$(TARGET_OS) = bin
-JDK_BIN_DIR.Darwin = Commands
-JDK_BIN = $(JDK_ROOT)/$(JDK_BIN_DIR.$(TARGET_OS))
+JDK_INCLUDE = $(JDK_ROOT)/include
+JDK_BIN = $(JDK_ROOT)/bin
 
 # ----------------------------------------------------------------------------
 # We use our own replacement for javah(1).
@@ -320,6 +315,8 @@ SHARED_LIBRARY_LDFLAGS.Darwin += -dynamiclib
 # This doesn't hurt on Linux, indeed it generally saves having to specify nonsense like -lstdc++.
 LD = $(CXX)
 
+EXTRA_INCLUDE_PATH.Darwin += $(JDK_INCLUDE)/darwin
+
 # Note that our Solaris build assumes GCC rather than Sun's compiler.
 # GCC's -shared option, which we use on Linux, exists, but produces link
 # errors. -G, as used in Sun's tutorial examples with their own compiler works.
@@ -547,19 +544,14 @@ BOOT_JDK.FreeBSD = /usr/local/openjdk${JAVA_MAJOR_VERSION}
 # := deferred to ALTERNATE_BOOTCLASSPATH
 BOOT_JDK.Cygwin = $(call findMakeFriendlyEquivalentName,$(shell ruby -e 'require "$(JDK_ROOT_SCRIPT)"; puts(findBootJdkFromRegistry())'))
 
-# := deferred to ALTERNATE_BOOTCLASSPATH
-BOOT_JDK.Darwin = $(shell /usr/libexec/java_home -v $(patsubst 8,1.8,$(JAVA_MAJOR_VERSION)))
-
 BOOT_JDK = $(BOOT_JDK.$(TARGET_OS))
 ALTERNATE_BOOTCLASSPATH ?= $(BOOT_JDK)/jre/lib/rt.jar
 ALTERNATE_BOOTCLASSPATH := $(wildcard $(ALTERNATE_BOOTCLASSPATH))
 BOOT_JDK_MESSAGE += $(NEWLINE)
 # Something else adds the trailing period.
 BOOT_JDK_MESSAGE += You need to install a package that will give you a Java $(JAVA_MAJOR_VERSION) JDK, so we can use its rt.jar
-BOOT_JDK_ERROR = $(error $(BOOT_JDK_MESSAGE))
 # The *** does battle with filter-build-output.rb.
 BOOT_JDK_WARNING = $(warning *** $(BOOT_JDK_MESSAGE))
-# Feel free to downgrade the ERROR to WARNING in your local copy.
 BOOT_JDK_DIAGNOSTIC = $(if $(filter 8,$(JAVA_MAJOR_VERSION)),$(BOOT_JDK_WARNING))
 JAVAC_FLAGS.javac += $(if $(ALTERNATE_BOOTCLASSPATH),-bootclasspath $(ALTERNATE_BOOTCLASSPATH),$(BOOT_JDK_DIAGNOSTIC))
 

the good news though is that this mostly removes mac-specific cruft.

i've deleted BOOT_JDK_ERROR and the "feel free" comment because it seems like someone already checked in that change and just didn't update the comment, and if no-one's noticed i'm assuming we can live without that stuff.

fwiw, i'm still seeing that warning on macOS even with this change, but it doesn't seem to make any difference? do we actually need this any more? (i've long since forgotten what this -bootclasspath stuff was for, or even whether i ever knew.)

[martindorey]

someone already checked in that change

You, in 48c5e21, without much explanation.

forgotten what this -bootclasspath stuff was for

It started being treated as an error with cdce383, which links to an explanation in the review comments of e036dfa. It was to stop anyone from fixing warnings that broke compatibility with Java 6. Java 6 compatibility has since gone but presumably the analogous issue now would be that you'd make changes to fix warnings produced by Java 11 or Java 13 or whatever and break compilation for Java 8. Guess I should have generalized the elaborated warning rather than removing it in 2b39c06.

[enh-google]

someone already checked in that change

You, in 48c5e21, without much explanation.

oh. that confirms my suspicion that whoever it was had checked it in by accident then... :-)

forgotten what this -bootclasspath stuff was for

It started being treated as an error with cdce383, which links to an explanation in the review comments of e036dfa. It was to stop anyone from fixing warnings that broke compatibility with Java 6. Java 6 compatibility has since gone but presumably the analogous issue now would be that you'd make changes to fix warnings produced by Java 11 or Java 13 or whatever and break compilation for Java 8. Guess I should have generalized the elaborated warning rather than removing it in 2b39c06.

the thing i should have added was that "there is no rt.jar in macOS 11's Java 11". nor, apparently, though i hadn't noticed until now, is there an rt.jar in my Debian Java 11...

[martindorey]

Yeah, Java 9 got rid of rt.jar, but the intent here is not that you use a Java 11 rt.jar, but a Java 8 one. Then you can compile with Java 11 and fix warnings with impunity.

[enh-google]

my assumption was that anyone on Debian stale^Wstable or otherwise restricted to an LTS Java will still be screwed because they'll be on Java 11 for 6-7 years (and this trick won't work then because there's no rt.jar in 11)? (i'm assuming from the lack of a newer LTS than 11 in https://www.oracle.com/java/technologies/java-se-support-roadmap.html that upstream has no plan for another LTS until the current one is really really old, like last time, rather than the Ubuntu style of fairly frequent LTS releases.)

i think the bit i don't get/remember is why we're dicking about with rt.jar rather than just using -source/-target? as far as i can tell from https://docs.oracle.com/en/java/javase/11/tools/javac.html they haven't broken that [again, since they removed Java 5 and earlier from the list]?

[martindorey]

The lack of rt.jar in Java 11 won’t be relevant until we switch our minimum supported version from 8 to 11. Last time I looked at the upstream situation, it appeared 8 will last longer than 11.

We already use -source and -target. It wasn’t enough, as demonstrated in the links I posted here before.

[enh-google]

The lack of rt.jar in Java 11 won’t be relevant until we switch our minimum supported version from 8 to 11. Last time I looked at the upstream situation, it appeared 8 will last longer than 11.

/me re-reads https://www.oracle.com/java/technologies/java-se-support-roadmap.html ... oh, yeah. wow.

maybe golang or rust will have gotten their shit together for cross-platform ui by then? nah, probably not.

We already use -source and -target. It wasn’t enough, as demonstrated in the links I posted here before.

ah, because these have been library issues, not language issues. yeah, this is what should have been in the $(warning )...

how do you want to go forward? does anyone have a pre-Big Sur mac that we'd need to keep compatibility with? or do we want basically the patch above plus a patch that explains why we're nagging about Java 8?

[martindorey]

does anyone have a pre-Big Sur mac that we'd need to keep compatibility with?

Not that I know of... and no one's chimed in to say your patch worked or didn't for them.

do we want basically the patch above plus a patch that explains why we're nagging about Java 8?

I'm keen. If it's too hard to get the Java 8 rt.jar on the Mac (even with wget ... tar ..., put it exactly and we'll do the right thing?), then I'd certainly settle for an improved warning rather than an error.

[martindorey]

I'm keen

He's done it in 41eb995. LGTM, thanks. I guess that's this closed, unless the OP wants to tell us otherwise?