perf: use compact pruning for large string NOT IN lists

[kumarUjjawal]

Which issue does this PR close?

• Part of Compact IN-list pruning excludes NOT IN and NULL-containing lists #24711.

Rationale for this change

Large string NOT IN lists create long expression chains for Parquet pruning. These expressions are expensive to build and evaluate.

What changes are included in this PR?

• Use a compact sorted domain for large, non-null string NOT IN lists.

• Preserve the existing behavior for lists containing NULL.

• Add unit tests and Parquet row-group and page-pruning tests.

• Add a mixed-container regression test.

• Extend the existing benchmark and configuration documentation.

• In a local benchmark with 1,024 values, construction was about 17 times faster and evaluation was about 27 times faster.

Are these changes tested?

Yes

Are there any user-facing changes?

No public API changes

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.57143% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.53%. Comparing base (4ada0dc) to head (6241218).

Files with missing lines	Patch %	Lines
datafusion/pruning/src/pruning_predicate.rs	90.00%	7 Missing and 11 partials ⚠️
datafusion/pruning/src/string_in_list.rs	80.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24781      +/-   ##
==========================================
- Coverage   81.53%   81.53%   -0.01%     
==========================================
  Files        1123     1123              
  Lines      406041   406218     +177     
  Branches   406041   406218     +177     
==========================================
+ Hits       331049   331192     +143     
- Misses      55631    55652      +21     
- Partials    19361    19374      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[adriangb]

@sunchao would you be interested in reviewing @kumarUjjawal's work?