pljava build from source failing on Mac m1 - can't link with a main executable file for architecture arm64

[birk11]

Hello,

I've postgres 11 installed on my system. When building pljava using command mvn clean install, it is failing at step PL/Java backend native code with the following error.

I'm very new to pljava. Any help would be really appreciated.

java version - 11
postgres version -11
apache maven - 3.8.6

installed llvm package via Homebrew

Error Log:

`[INFO] Using compiling/linking rules for Mac OS X
ld: warning: directory not found for option '-L/opt/local/Current/lib'
ld: warning: directory not found for option '-L/opt/local/libexec/llvm-6.0/lib'
ld: warning: directory not found for option '-L/opt/local/libexec/llvm-6.0/lib'
ld: warning: directory not found for option '-L/opt/local/20180529/lib'
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/BigDecimal.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Time.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Composite.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Relation.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/PgSavepoint.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Portal.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Short.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/ErrorData.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Oid.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Invocation.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ld: warning:
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Type.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/TypeOid.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/TupleTable.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Backend.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SQLXMLImpl.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/HashMap.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Date.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/TriggerData.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Float.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SQLOutputToChunk.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Long.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SQLInputFromChunk.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/XactListener.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning:
ld: warning:
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Void.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Session.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Coerce.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/PgObject.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Boolean.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/ExecutionPlan.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Double.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SingleRowReader.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Tuple.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/VarlenaWrapper.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/TupleDesc.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/AclId.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SPI.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SubXactListener.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Exception.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Iterator.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Integer.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/UDT.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning:
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Timestamp.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Array.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Any.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/InstallHelper.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Byte.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/DualState.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/Function.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64

ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/JNICalls.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/String.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SQLOutputToTuple.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/SQLInputFromTuple.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: warning: ignoring file /Users/kishorebiradavolu/Tech/pljava/pljava-so/target/pljava-pgxs/byte_array.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
ld: can't link with a main executable file '/Library/PostgreSQL/11/bin/postgres' for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for PostgreSQL PL/Java 2-SNAPSHOT:
[INFO]
[INFO] PostgreSQL PL/Java ................................. SUCCESS [ 0.521 s]
[INFO] PL/Java API ........................................ SUCCESS [ 1.995 s]
[INFO] PL/Java backend Java code .......................... SUCCESS [ 1.724 s]
[INFO] PL/Java PGXS ....................................... SUCCESS [ 1.674 s]
[INFO] PL/Java backend native code ........................ FAILURE [ 3.036 s]
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.009 s
[INFO] Finished at: 2022-09-27T19:50:39+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.postgresql:pljava-pgxs:2-SNAPSHOT:scripted-goal (build-shared-object) on project pljava-so: Linking failed with exit code: 1 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.postgresql:pljava-pgxs:2-SNAPSHOT:scripted-goal (build-shared-object) on project pljava-so: Linking failed with exit code: 1`

[sandeep2244]

Is it resolved ? I got similar kind of issue with error such as

3 warnings generated.
ld: can't link with a main executable file '/opt/homebrew/Cellar/postgresql@14/14.8/bin/postgres' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for PostgreSQL PL/Java 1.6.4:
[INFO] 
[INFO] PostgreSQL PL/Java ................................. SUCCESS [  3.587 s]
[INFO] PL/Java API ........................................ SUCCESS [ 10.301 s]
[INFO] PL/Java backend Java code .......................... SUCCESS [  9.538 s]
[INFO] PL/Java PGXS ....................................... SUCCESS [  5.890 s]
[INFO] PL/Java backend native code ........................ FAILURE [  7.588 s]
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

[ERROR] Failed to execute goal org.postgresql:pljava-pgxs:1.6.4:scripted-goal (build-shared-object) on project pljava-so: Linking failed with exit code: 1 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.postgresql:pljava-pgxs:1.6.4:scripted-goal (build-shared-object) on project pljava-so: Linking failed with exit code: 1

I have arm64 system architecture. It's failing on project pljava-so

[jcflack]

This issue, along with #428, seems to relate to the Mac OS compiler and linker ending up with some objects built for x86_64 architecture but a postgres binary for arm64, or the other way around.

This may be a matter of not getting the right options to the compiler or linker during the build.

In pljava-so/pom.xml, there is a var configuration = [ section that currently supplies compiling and linking rules for: Linux, Mac OS X, Windows MinGW, and Windows MSVC.

It may be that the "Mac OS X" configuration needs to be either adjusted to use the right command options on both x86_64 and arm64 architecture, or split into two configurations with more specific probe functions so the right configuration is selected.

I do not have ready access to an arm64 Mac to work on that myself, and the architecture does not appear to be available in the GitHub Actions CI environment we are using. I would certainly welcome a pull request if the right options can be figured out.

[jcflack]

Recent CI runs (see, for example, c5f9ae4) include the GitHub Actions macos-14 runner, which is arm64-based, and those runs are succeeding.

However, while updating the CI scripts, I did encounter a similarly puzzling linker error that listed a lot of PostgreSQL internal symbols and said it could not find them for x86_64—and this while running on an arm64 platform.

The cause in my case turned out to be that the GitHub Actions setup-java@v1 action had installed a JDK built for x86_64. And it worked, apparently in a compatibility mode. And the compatibility mode was probably inherited by the linker spawned as a subprocess from Maven, so the linker in some way thought it should look for x86_64 symbols.

The solution was to make sure that the JDK being used was built for arm64 (in this case, by upgrading the setup-java action to a newer version smart enough to download a JDK built for the right architecture).