[fix](docker) skip same-file juicefs hive jar copy#61792
[fix](docker) skip same-file juicefs hive jar copy#61792xylaaaaa wants to merge 1 commit intoapache:masterfrom
Conversation
(cherry picked from commit e37531a)
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
There was a problem hiding this comment.
Pull request overview
This PR updates the thirdparty Docker harness to avoid failing the Hive setup when the discovered JuiceFS Hadoop jar is already located in Hive’s auxlib directory (same-file copy case).
Changes:
- Compute the expected Hive
auxlibtarget jar path and skip copying when source/target resolve to the same file. - Preserve existing behavior otherwise by syncing the jar into Hive
auxlib.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| target_jar="${auxlib_dir}/$(basename "${source_jar}")" | ||
| source_realpath=$(realpath "${source_jar}") | ||
| if [[ -e "${target_jar}" ]]; then | ||
| target_realpath=$(realpath "${target_jar}") | ||
| if [[ "${source_realpath}" == "${target_realpath}" ]]; then |
There was a problem hiding this comment.
This introduces a new hard dependency on the external realpath command in the Hive path. On environments where realpath isn’t available, ensure_juicefs_hadoop_jar_for_hive will fail even though the copy could succeed. Since the goal is just “same file” detection, consider using Bash’s file test [[ "$source_jar" -ef "$target_jar" ]] (same inode) to avoid external commands and handle symlink/hardlink cases more robustly.
Summary
cp -fso the Hive pipeline does not fail with a same-file copy errorTest Plan
bash -n docker/thirdparties/run-thirdparties-docker.shsource_jar == target_jarand verifiesensure_juicefs_hadoop_jar_for_hiveexits successfully withJuiceFS Hadoop jar already present in hive auxlib: juicefs-hadoop-1.3.1.jar