Skip to content

[fix](docker) skip same-file juicefs hive jar copy#61792

Open
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:fix/skip-same-file-juicefs-hive-jar-copy
Open

[fix](docker) skip same-file juicefs hive jar copy#61792
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:fix/skip-same-file-juicefs-hive-jar-copy

Conversation

@xylaaaaa
Copy link
Copy Markdown
Contributor

Summary

  • avoid copying the JuiceFS Hadoop jar when the discovered source jar is already the same file under Hive auxlib
  • compare source and target realpaths before cp -f so the Hive pipeline does not fail with a same-file copy error

Test Plan

  • bash -n docker/thirdparties/run-thirdparties-docker.sh
  • run a targeted shell harness that sets source_jar == target_jar and verifies ensure_juicefs_hadoop_jar_for_hive exits successfully with JuiceFS Hadoop jar already present in hive auxlib: juicefs-hadoop-1.3.1.jar

Copilot AI review requested due to automatic review settings March 27, 2026 02:19
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xylaaaaa
Copy link
Copy Markdown
Contributor Author

run buildall

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the thirdparty Docker harness to avoid failing the Hive setup when the discovered JuiceFS Hadoop jar is already located in Hive’s auxlib directory (same-file copy case).

Changes:

  • Compute the expected Hive auxlib target jar path and skip copying when source/target resolve to the same file.
  • Preserve existing behavior otherwise by syncing the jar into Hive auxlib.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +365 to +369
target_jar="${auxlib_dir}/$(basename "${source_jar}")"
source_realpath=$(realpath "${source_jar}")
if [[ -e "${target_jar}" ]]; then
target_realpath=$(realpath "${target_jar}")
if [[ "${source_realpath}" == "${target_realpath}" ]]; then
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces a new hard dependency on the external realpath command in the Hive path. On environments where realpath isn’t available, ensure_juicefs_hadoop_jar_for_hive will fail even though the copy could succeed. Since the goal is just “same file” detection, consider using Bash’s file test [[ "$source_jar" -ef "$target_jar" ]] (same inode) to avoid external commands and handle symlink/hardlink cases more robustly.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants