Skip to content

FilterExec empty projection is changed to full projection after serde #21871

@askalt

Description

@askalt

Consider the following places in the physical plan serde code.

serialization:

projection: exec.projection().as_ref().map_or_else(Vec::new, |v| {
v.iter().map(|x| *x as u32).collect::<Vec<u32>>()

deserealization:

let projection = if !filter.projection.is_empty() {
Some(
filter
.projection
.iter()
.map(|i| *i as usize)
.collect::<Vec<_>>(),
)
} else {
None
};

If we consider FilterExec with an empty projection (vec![]), then it is serialized as an empty vector. Then the deserialization code makes a FilterExec with None as projection, which semantics is a full projection. The problem is that None and Some(vec![]) have the same proto representation.

The simplest solution:

  1. Make (0..fields.len()).collect() vec when None projection is serialized.
  2. Make None if this vec is met in the deserialization code.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions