GH-41307: [Java] Use org.apache:apache parent pom version 31

[laurentgo]

Rationale for this change

Use/update Maven modules to org.apache:parent:31 and clean up Maven modules to remove unnecessary configuration or outdated workarounds

What changes are included in this PR?

• Add org.apache:parent:31 to org.apache.arrow:arrow-bom and org.apache.arrow.maven.plugins:arrow-maven-plugins to make them conformant with ASF standards
• Update org.apache.arrow:arrow-java-root parent to org.apache:parent:31
• Use version.* and other properties to override plugin versions defined by org.apache:parent
• Move standalone plugin versions under pluginManagement at the top level
• Cleanup redundant plugin version or configuration declaration
• Update maven-dependency-plugin to 3.6.1 and add the required overrides when necessary
• Update maven-shade-plugin to 3.5.1 (via org.apache:parent)
• Remove enforcer check for java and maven version (handled by org.apache:parent)
• Remove unnecessary mvnrepository link comments
• Remove m2e.version property check in profiles (only needed for errorprone plugin configuration which is incompatible with M2E)
• Cleanup argLine overrides for surefire/failsafe plugins
• Remove unnecessary ../pom.xml <relativePath> directives
• Remove source/target/encoding configuration properties for maven-compiler-plugin, maven-javadoc-plugin and maven-resources-plugin as it is handled by org.apache:parent and plugin themselves
• Remove unnecessary copy of codegen templates in arrow-vector module
• Remove unnecessary junit jupiter engine dependencies for surefire/failsafe plugins.

Are these changes tested?

No net new code. tested via CI jobs

Are there any user-facing changes?

None

• GitHub Issue: [Java] Update Maven modules to use latest Apache Parent (31) and plugins #41307

[github-actions]

⚠️ GitHub issue #41307 has been automatically assigned in GitHub to PR creator.

[vibhatha]

Thanks for working on this one @laurentgo
Will take a look.

[lidavidm]

@zeroshade This is a rather concerning CI flake

======================================================================
Testing C ArrowArray from file 'custom_metadata'
... with record batch #0
======================================================================
gc 41 @3.901s 0%: 0.019+0.65+0.19 ms clock, 0.077+0.098/0.17/0.34+0.76 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 4 P
gc 42 @3.920s 0%: 0.014+0.63+0.032 ms clock, 0.057+0.074/0.37/0.030+0.13 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 4 P
gc 43 @3.928s 0%: 0.010+0.28+0.18 ms clock, 0.042+0.061/0.19/0.035+0.73 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 4 P
gc 44 @3.938s 0%: 0.025+0.47+0.029 ms clock, 0.10+0.076/0.19/0.028+0.11 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 4 P
gc 45 @3.950s 0%: 0.009+0.32+0.038 ms clock, 0.036+0.062/0.23/0.070+0.15 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 4 P
runtime: marked free object in span 0x7fbb4c0bf200, elemsize=32 freeindex=0 (bad use of unsafe.Pointer? try -d=checkptr)

[lidavidm]

@laurentgo I remember this error from the last PR, there's something about the new build that seems to break the action:

Error: The template is not valid. .github/workflows/java_jni.yml (Line: 111, Col: 16): hashFiles('java/**') failed. Fail to hash files under directory '/home/runner/work/arrow/arrow'

[laurentgo]

@laurentgo I remember this error from the last PR, there's something about the new build that seems to break the action:

Error: The template is not valid. .github/workflows/java_jni.yml (Line: 111, Col: 16): hashFiles('java/**') failed. Fail to hash files under directory '/home/runner/work/arrow/arrow'

I saw that error in my previous version of my change (before the rebase) but I don't see it in the latest iteration and the arrow_jni check is now green. Do you think there's something in my change which may make things flaky?

[lidavidm]

It happens here though: https://github.com/apache/arrow/actions/runs/8776033066/job/24079021700?pr=41309

[vibhatha]

I remember this error from the last PR, there's something about the new build that seems to break the action

Are we referring to: #41174

[vibhatha]

Could we debug the CI by using something like run: ls -R java/ and making sure the directory is there? Error message suggests that it cannot find the files.

[laurentgo]

Could we debug the CI by using something like run: ls -R java/ and making sure the directory is there? Error message suggests that it cannot find the files.

We could but not sure how much it would help: from the existing output, the repository exists and is used by the build step, and I'm relatively confident that at the end of the build, the directory should still be here.

I also searched a bit on the topic and found some possible related issues on https://github.com/actions/runner/ where it could be because of the file size of file permission. Ideally we may want to trigger the workflow again in debug mode which would provide detailed logs on the action. Unfortunately I don't have the permissions for it, but maybe committers do?
If we don't, I'll probably open a new PR to debug the issue without polluting too much this PR.

[lidavidm]

I re-ran with debug logging enabled.

[vibhatha]

@laurentgo I looked into the debug logs, I am not quite sure what is wrong here. But would it be possible to check this per module? I know this change is not that big, but just a thought, would it be better if we try the change per module? Or at least a set of modules?

[lidavidm]

I mean the error is right there.

##[debug][Error: EACCES: permission denied, open '/home/runner/work/arrow/arrow/java/adapter/avro/target/arrow-avro-17.0.0-SNAPSHOT.jar'] {
##[debug]  errno: -13,
##[debug]  code: 'EACCES',
##[debug]  syscall: 'open',
##[debug]  path: '/home/runner/work/arrow/arrow/java/adapter/avro/target/arrow-avro-17.0.0-SNAPSHOT.jar'

It appears Maven is now building things that aren't readable by us for some reason?

[laurentgo]

It appears Maven is now building things that aren't readable by us for some reason?

Which is weird but things are also run inside a docker container with volume mount and the hashFiles is accessing files externally, so things can go wrong if a plugin upgrade change maybe group or other permissions for example? But that's really useful information, thanks for retriggering the action.

[laurentgo]

On my local machines all files are accessible from owner, group and others (644) but for some reason when the build is running on a github runner, permissions are slightly different:

./java/adapter/avro/target:
total 308
-rw-r--r-- 1 root root 81169 Apr 24 19:33 arrow-avro-17.0.0-SNAPSHOT-tests.jar
-rw------- 1 root root 53190 Apr 24 19:33 arrow-avro-17.0.0-SNAPSHOT.jar
-rw-r--r-- 1 root root 57327 Apr 24 19:33 bom.json
-rw-r--r-- 1 root root 49225 Apr 24 19:33 bom.xml
-rw-r--r-- 1 root root  4471 Apr 24 19:32 checkstyle-cachefile
-rw-r--r-- 1 root root 13257 Apr 24 19:32 checkstyle-checker.xml
-rw-r--r-- 1 root root   801 Apr 24 19:32 checkstyle-header.txt
-rw-r--r-- 1 root root  2369 Apr 24 19:32 checkstyle-suppressions.xml
drwxr-xr-x 4 root root  4096 Apr 24 19:33 classes
drwxr-xr-x 3 root root  4096 Apr 24 19:32 generated-sources
drwxr-xr-x 3 root root  4096 Apr 24 19:33 generated-test-sources
drwxr-xr-x 2 root root  4096 Apr 24 19:33 gradle-enterprise
drwxr-xr-x 2 root root  4096 Apr 24 19:33 maven-archiver
drwxr-xr-x 3 root root  4096 Apr 24 19:32 maven-shared-archive-resources
drwxr-xr-x 3 root root  4096 Apr 24 19:32 maven-status
drwxr-xr-x 2 root root  4096 Apr 24 19:32 test
drwxr-xr-x 5 root root  4096 Apr 24 19:33 test-classes

I see the same pattern when using maven-shade-plugin so most likely related to this plugin version change. Will keep looking into it
I misread the classifier, it is not related to the maven-shade-plugin. It's actually quite weird because permissions are "incorrect" for all main jars attached to the project, but not for test jar which is produced with the same plugin. Investigating

[laurentgo]

I believe I found the issue. Recent version of maven-jar-plugin uses an updated version of plexus-archiver which will rewrite the jar content to fix modification time if the jar contains a module-info.class file. But as Java 11 jar command does not support --date option, plexus-archiver implements a fallback which relies on Files#createTemp() method. Unfortunately, the file created by this method has only user read/write permission and no group/others permissions.

[lidavidm]

Thanks for figuring that out!

[lidavidm]

We used this since some gRPC artifacts are platform-dependent. Are the properties this plugin provides now provided by something else?

[laurentgo]

It's already declared by the top-level parent pom.xml

[lidavidm]

actually, let's run crossbow too...

[lidavidm]

@github-actions crossbow submit java

[vibhatha]

I am not aware of. I can take a look @lidavidm

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 9090e67.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 10 possible false positives for unstable benchmarks that are known to sometimes produce them.

[vibhatha]

@lidavidm I looked into the java-jars CI failure in a couple of PRs and I observed the following

Java Jars CI Failure

arrow-filesystem-test

Build C++ libraries macOS x86_64

1. https://github.com/ursacomputing/crossbow/actions/runs/8871324532/job/24354134578#step:6:11009

Build C++ libraries macOS aarch_64

1. https://github.com/ursacomputing/crossbow/actions/runs/8871324532/job/24354135400#step:6:11009

Seems like it is a failure in C++ filesystem tests.

[lidavidm]

@bkietz?

[bkietz]

It seems the registrar is getting constructed twice, which puts the registry in an error state for that scheme. I don't know how that's happening but we can reuse the deduplication logic that's already in place for merging registries

[kiszk]

@github-actions crossbow submit test-conda-python-3.10-spark-v3.5.0

[github-actions]

Revision: b613aa0

Submitted crossbow builds: ursacomputing/crossbow @ actions-285c848c41

Task	Status
test-conda-python-3.10-spark-v3.5.0