Is your feature request related to a problem or challenge?
these work:
> select arrow_cast(struct(10), 'Struct("c0": nullable Int64)');
+--------------------------------------------------------------------+
| arrow_cast(struct(Int64(10)),Utf8("Struct("c0": nullable Int64)")) |
+--------------------------------------------------------------------+
| {c0: 10} |
+--------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.065 seconds.
> select arrow_cast(10, 'List(non-null Int64)');
+----------------------------------------------------+
| arrow_cast(Int64(10),Utf8("List(non-null Int64)")) |
+----------------------------------------------------+
| [10] |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.010 seconds.
this doesn't work, when field is specified to be non-nullable inside a struct:
> select arrow_cast(struct(10), 'Struct("c0": non-null Int64)');
Optimizer rule 'simplify_expressions' failed
caused by
This feature is not implemented: Unsupported CAST from Struct("c0": Int64) to Struct("c0": non-null Int64)
Describe the solution you'd like
see if its possible to implement this support? it seems to work for lists but not structs, so not sure if this is a limitation or theres a valid reason to not allow this
Describe alternatives you've considered
No response
Additional context
i think somewhere around here in the codebase?
|
pub fn cast_with_target_field( |
|
expr: Arc<dyn PhysicalExpr>, |
|
input_schema: &Schema, |
|
target_field: FieldRef, |
|
cast_options: Option<CastOptions<'static>>, |
|
) -> Result<Arc<dyn PhysicalExpr>> { |
|
let expr_type = expr.data_type(input_schema)?; |
|
let cast_type = target_field.data_type(); |
|
if expr_type == *cast_type && is_default_target_field(&target_field) { |
|
return Ok(Arc::clone(&expr)); |
|
} |
|
|
|
let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) { |
|
// Allow casts involving structs (including nested inside Lists, Dictionaries, |
|
// etc.) that pass name-based compatibility validation. This validation is |
|
// applied at planning time (now) to fail fast, rather than deferring errors |
|
// to execution time. The name-based casting logic will be executed at runtime |
|
// via ColumnarValue::cast_to. |
|
can_cast_named_struct_types(&expr_type, cast_type) |
|
} else { |
|
can_cast_types(&expr_type, cast_type) |
|
}; |
|
|
|
if !can_build_cast { |
|
return not_impl_err!("Unsupported CAST from {expr_type} to {cast_type}"); |
|
} |
|
|
|
Ok(Arc::new(CastExpr::new_with_target_field( |
|
expr, |
|
target_field, |
|
cast_options, |
|
))) |
|
} |
Is your feature request related to a problem or challenge?
these work:
this doesn't work, when field is specified to be non-nullable inside a struct:
Describe the solution you'd like
see if its possible to implement this support? it seems to work for lists but not structs, so not sure if this is a limitation or theres a valid reason to not allow this
Describe alternatives you've considered
No response
Additional context
i think somewhere around here in the codebase?
datafusion/datafusion/physical-expr/src/expressions/cast.rs
Lines 384 to 416 in 5c92390