Skip to content

[VL] Rework the workaround for empty schema batch #7600

@zhztheplayer

Description

@zhztheplayer

Description

Velox backend uses a very specific way including some workarounds to handle empty schema batches. Over time, this part of code gets bloated by one workaround over another. We'd review all the relevant code and do a complete refactor.

Note it's easier to come up with a solution after we removed Spark 3.2 support. See #540. But we don't wait for that and do the essentials in advance.

Sub-tasks:

  1. UT [GLUTEN-7600][VL] Prepare test case for the removal of workaround code for empty schema batches #7601
  2. Removal of the rules [GLUTEN-7600][VL] Remove EmptySchemaWorkaround #7620
  3. Changes to RAS [GLUTEN-7600][VL] Simplify offload rules in RAS #7646

Related historical issue list (incomplete):

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions