Skip to content

[CH] Fix flatten nullable inner array row handling #12174

@lgbo-ustc

Description

@lgbo-ustc

Backend

CH (ClickHouse)

Bug description

For Array(Nullable(Array(T))), the code scans all nested elements and returns input_rows_count default nullable rows as soon as it sees one null inner array. This makes unrelated rows null. Example: [[[1], null], [[2]]] should only make the first row null under Spark-style flatten semantics, but the current implementation returns both rows as null.

Gluten version

No response

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions