Perf: remove extra allocations for hex operations

[Rich-T-kid]

Which issue does this PR close?

• Closes Avoid per-row String allocation when hex-encoding digest output (md5, sha1, sha2) #23810.

Rationale for this change

The digest functions hex-encode their output by allocating one String per row and then copying each of those into the output array (#23810)

What changes are included in this PR?

• This PR reduces per-row allocation and string-conversion overhead in the Spark crypto/hash paths by switching to the buffer-based hex encoder APIs.
• Updates call sites to re-use vector buffers.

Are these changes tested?

existing test cover behavior.

Are there any user-facing changes?

yes, encode_bytes() is being deprecated. This was apart of the public API

[Rich-T-kid]

@andygrove this is ready for review 👍

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion-common v54.1.0 (current)
       Built [  35.726s] (current)
     Parsing datafusion-common v54.1.0 (current)
      Parsed [   0.060s] (current)
    Building datafusion-common v54.1.0 (baseline)
       Built [  33.574s] (baseline)
     Parsing datafusion-common v54.1.0 (baseline)
      Parsed [   0.060s] (baseline)
    Checking datafusion-common v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.627s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure function_marked_deprecated: function #[deprecated] added ---

Description:
A function is now #[deprecated]. Downstream crates will get a compiler warning when using this function.
        ref: https://doc.rust-lang.org/reference/attributes/diagnostics.html#the-deprecated-attribute
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_marked_deprecated.ron

Failed in:
  function datafusion_common::utils::hex::encode_bytes in /home/runner/work/datafusion/datafusion/datafusion/common/src/utils/hex.rs:176

     Summary semver requires new minor version: 0 major and 1 minor checks failed
    Finished [  71.416s] datafusion-common
    Building datafusion-functions v54.1.0 (current)
       Built [  30.587s] (current)
     Parsing datafusion-functions v54.1.0 (current)
      Parsed [   0.085s] (current)
    Building datafusion-functions v54.1.0 (baseline)
       Built [  31.173s] (baseline)
     Parsing datafusion-functions v54.1.0 (baseline)
      Parsed [   0.086s] (baseline)
    Checking datafusion-functions v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.401s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  63.580s] datafusion-functions
    Building datafusion-spark v54.1.0 (current)
       Built [  58.037s] (current)
     Parsing datafusion-spark v54.1.0 (current)
      Parsed [   0.063s] (current)
    Building datafusion-spark v54.1.0 (baseline)
       Built [  58.061s] (baseline)
     Parsing datafusion-spark v54.1.0 (baseline)
      Parsed [   0.064s] (baseline)
    Checking datafusion-spark v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.339s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 117.725s] datafusion-spark

[Rich-T-kid]

We may not need to deprecate this — just swapping out how DataFusion uses this internally may be enough, and it avoids a breaking API change.

[Rich-T-kid]

local benchmarks show promising results

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.88%. Comparing base (21ad189) to head (cd5492e).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24057      +/-   ##
==========================================
- Coverage   80.88%   80.88%   -0.01%     
==========================================
  Files        1101     1101              
  Lines      375720   375787      +67     
  Branches   375720   375787      +67     
==========================================
+ Hits       303895   303947      +52     
- Misses      53729    53738       +9     
- Partials    18096    18102       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.