Skip to content

Be able to cast struct to have non-nullable inner field #22709

@Jefffrey

Description

@Jefffrey

Is your feature request related to a problem or challenge?

these work:

> select arrow_cast(struct(10), 'Struct("c0": nullable Int64)');
+--------------------------------------------------------------------+
| arrow_cast(struct(Int64(10)),Utf8("Struct("c0": nullable Int64)")) |
+--------------------------------------------------------------------+
| {c0: 10}                                                           |
+--------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.065 seconds.

> select arrow_cast(10, 'List(non-null Int64)');
+----------------------------------------------------+
| arrow_cast(Int64(10),Utf8("List(non-null Int64)")) |
+----------------------------------------------------+
| [10]                                               |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.010 seconds.

this doesn't work, when field is specified to be non-nullable inside a struct:

> select arrow_cast(struct(10), 'Struct("c0": non-null Int64)');
Optimizer rule 'simplify_expressions' failed
caused by
This feature is not implemented: Unsupported CAST from Struct("c0": Int64) to Struct("c0": non-null Int64)

Describe the solution you'd like

see if its possible to implement this support? it seems to work for lists but not structs, so not sure if this is a limitation or theres a valid reason to not allow this

Describe alternatives you've considered

No response

Additional context

i think somewhere around here in the codebase?

pub fn cast_with_target_field(
expr: Arc<dyn PhysicalExpr>,
input_schema: &Schema,
target_field: FieldRef,
cast_options: Option<CastOptions<'static>>,
) -> Result<Arc<dyn PhysicalExpr>> {
let expr_type = expr.data_type(input_schema)?;
let cast_type = target_field.data_type();
if expr_type == *cast_type && is_default_target_field(&target_field) {
return Ok(Arc::clone(&expr));
}
let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) {
// Allow casts involving structs (including nested inside Lists, Dictionaries,
// etc.) that pass name-based compatibility validation. This validation is
// applied at planning time (now) to fail fast, rather than deferring errors
// to execution time. The name-based casting logic will be executed at runtime
// via ColumnarValue::cast_to.
can_cast_named_struct_types(&expr_type, cast_type)
} else {
can_cast_types(&expr_type, cast_type)
};
if !can_build_cast {
return not_impl_err!("Unsupported CAST from {expr_type} to {cast_type}");
}
Ok(Arc::new(CastExpr::new_with_target_field(
expr,
target_field,
cast_options,
)))
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions