[chore](thirdparty) keep hadoop_hdfs as a symlink to hadoop_hdfs_3_4

[morningman]

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #66842, #66908

Problem Summary:

#66842 stopped building hadoop-libs 3.3.6.6 and removed the
installed/{include,lib}/hadoop_hdfs/ prefix it produced. Only 3.4.2.4 under
hadoop_hdfs_3_4/ is built now, so everything still reading the old prefix fails
against a third-party archive built from it:

consumer	reads
cloud/src/recycler/hdfs_accessor.{h,cpp}	<hadoop_hdfs/hdfs.h>
cloud/CMakeLists.txt	lib/hadoop_hdfs/native/libhdfs.a
be/src/io/hdfs_builder.cpp	hadoop_hdfs/hdfs.h
build.sh	LAST_THIRDPARTY_LIB='hadoop_hdfs/native/libhdfs.a'
run-be-ut.sh, run-cloud-ut.sh	installed/lib/hadoop_hdfs/
branch-3.0, branch-3.1, anything outside this repository	both

#66908 moves this repository's consumers onto hadoop_hdfs_3_4/, and that stays the
right fix for master. It does nothing for the release branches or for anyone outside
this repository, and it cannot land until the prebuilt archives have been rebuilt.
So build_hadoop_libs_3_4 leaves the old prefix behind as a symlink to the one build
that remains.

Why a symlink rather than a second copy

lib/hadoop_hdfs_3_4/ is 182MB per platform, and the two prefixes have always had
identical layouts - the removed build_hadoop_libs() and build_hadoop_libs_3_4()
are the same cp sequence with a different version number, so the old name fits the
new tree exactly.

The install prefix is already built this way: build-thirdparty.sh creates
installed/lib -> lib64 as a relative directory symlink before any package runs, and
every prebuilt archive carries it. That is what shows the packaging and distribution
path preserves symlinks - a prebuilt from #66842 unpacks with 55 of them.

Two details that are load-bearing

• The target is relative (hadoop_hdfs_3_4, not ${TP_INSTALL_DIR}/...), so the
  link resolves wherever the archive is unpacked rather than pointing back at the
  machine that built it.
• rm -rf runs without a trailing slash, and before ln -s. rm -rf link/
  deletes what the link points at on BSD rm and does nothing on GNU rm, while
  rm -rf link removes just the link on both. And without removing first, ln -s
  against a leftover real hadoop_hdfs/ directory - an older prebuilt, or a build-env
  image predating its removal - creates hadoop_hdfs/hadoop_hdfs_3_4 inside it and
  leaves the stale 3.3.6.6 archive in place.

Effect on the merge order of #66842 / #66908

#66842 noted a window between itself and #66908: a Linux tree picking up a freshly
built archive would not find installed/lib/hadoop_hdfs/native/libhdfs.a, which is
still what build.sh's LAST_THIRDPARTY_LIB looks for, and would start a full ~3h
third-party rebuild. Once the archives are rebuilt from this PR, the sentinel resolves
through the link and that window closes without waiting for #66908.

#66908 should still land, sentinel move included - pointing at the real directory
rather than at the shim keeps it working when the shim is eventually dropped. One line
in its release note no longer holds: anything outside this repository reading the
hadoop_hdfs/ prefix keeps working.

Release note

None

Check List (For Author)

• Test

  • Manual test (add detailed scripts or steps below)

    build_hadoop_libs_3_4 was driven end to end against a fixture install prefix
    reproducing lib -> lib64, on macOS (BSD rm/ln) and on ubuntu-22.04 (GNU
    coreutils 8.32), checking that:

    • both links are created, with the relative target, and the lib one lands in
      lib64/ where lib points;
    • build.sh's hadoop_hdfs/native/libhdfs.a sentinel and
      <hadoop_hdfs/hdfs.h> resolve through them;
    • a rerun neither nests the links nor empties the 3.4.2.4 tree behind them;
    • an installation carrying a real 3.3.6.6 hadoop_hdfs/ directory is taken over,
      with the old prefix resolving to the 3.4.2.4 archive afterwards;
    • the links survive tar -czf / tar -xzf into a different path, which is how
      the prebuilt archive is shipped.

    This PR touches thirdparty/**, so its own Build Third Party Libraries run
    builds the whole tree on ubuntu-22.04, macos-15 and macos-14, which is where the
    change actually executes.

• Behavior changed:

  • Yes.
    • installed/{include,lib}/hadoop_hdfs resolves to hadoop-libs 3.4.2.4, not
      the 3.3.6.6 it named before [chore](thirdparty) build only hadoop 3.4.2.4, build azure everywhere and slim it down, and add ccache #66842. A tree built against a new archive through
      the old prefix gets 3.4.2.4. That is the intended outcome - BE already linked
      the 3.4.2.4 archive while hdfs_builder.cpp included the 3.3.6.6 header, and
      [chore](build) move hadoop_hdfs consumers onto 3.4.2.4 and link azure on aarch64/macOS #66908 moves cloud/ to 3.4.2.4 as well - but it is a version change, not a
      restoration. Release branches keep using the archives and build-env images
      matching their own third-party tree, so their CI is unaffected.
    • This is a compatibility shim over a single build. It should be dropped once
      nothing reads the old prefix; the CHANGELOG entry says so.

• Does this need documentation?

  • No.

🤖 Generated with Claude Code

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?