datafusion/substrait: enable unicode_expressions in dev-dependencies to fix substring planning test
#17584
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
not planned under Substrait whenunicode_expressions` feature is disabled #17582Rationale for this change
A test failure was observed when building the DataFusion Substrait integration with only the
nested_expressionsfeature enabled indatafusion/substrait/Cargo.toml. TheUnicodeFunctionPlannerwas not registered whenunicode_expressionswas disabled, causing theSUBSTR/substringfunction to be unplannable during Substrait logical plan conversion. This produced the error shown in the test:This PR updates the dev-dependency for
datafusionwithindatafusion/substrait/Cargo.tomlto include theunicode_expressionsfeature so that thesimple_scalar_function_substrtest can run reliably and the Unicode function planner is available in test builds.This change is limited to test/build configuration (dev-deps) and does not alter runtime behavior for downstream users. It ensures the test-suite accurately reflects the capabilities exercised by the tests and prevents false negatives during CI.
What changes are included in this PR?
datafusion/substrait/Cargo.tomldev-dependency fordatafusionto add theunicode_expressionsfeature alongsidenested_expressions.Before:
After:
No source code changes are required; this is a test/build configuration change.
Are these changes tested?
Yes — the change is intended to fix existing unit/integration tests that exercise
SUBSTR/substringplanning under the Substrait converter. In particular, thesimple_scalar_function_substrtest which failed with theNotImplementederror should now run with theUnicodeFunctionPlannerregistered and pass.CI: this change is expected to make CI green for the Substrait test suite where the previous configuration caused a failure. Please run the repository test matrix locally or rely on CI to validate across all relevant feature combinations.
If maintainers prefer not to enable
unicode_expressionsin dev-deps, an alternative is to mark the test as conditionally skipped whenunicode_expressionsis not enabled or to add a fallback planning path for ASCII-only substring expressions. Both are noted below.Are there any user-facing changes?
unicode_expressionsexplicitly.Test plan / How to validate
datafusion/substrait/Cargo.toml).cargo test -p datafusion-substrait --test substrait_integrationsimple_scalar_function_substrtest passes and that no other tests regress.Context
This issue was uncovered here