drop columns gives wrong results or errors for tables with "holes"

[IanManske]

Describe the bug

drop columns will only keep columns present in the first row. So, if the rows after the first have columns not present in the first, these columns will also be dropped. Similarly, if the rows after the first do not have all the retained columns of the first row, the command errors with nu::shell::column_not_found.

How to reproduce

1. [{a: 1}, {b: 2, c: 3}] | drop column gives:

╭───┬───────────────────╮
│ 0 │ {record 0 fields} │
│ 1 │ {record 0 fields} │
╰───┴───────────────────╯

2. [{a: 1, b: 2}, {c: 3}] | drop column gives:

Error: nu::shell::column_not_found

  × Cannot find column
   ╭─[entry #2:1:1]
 1 │ [{a: 1, b: 2}, {c: 3}] | drop column
   · ───────────┬──────────┬
   ·            │          ╰── value originates here
   ·            ╰── cannot find column 'a'
   ╰────

Expected behavior

1. [{a: 1}, {b: 2, c: 3}] | drop column should give:

╭───┬────┬────╮
│ # │ a  │ b  │
├───┼────┼────┤
│ 0 │  1 │ ❎ │
│ 1 │ ❎ │  2 │
╰───┴────┴────╯

or

╭───┬────┬────╮
│ # │ b  │ c  │
├───┼────┼────┤
│ 0 │ ❎ │ ❎ │
│ 1 │  2 │  3 │
╰───┴────┴────╯

2. [{a: 1, b: 2}, {c: 3}] | drop column should give:

╭───┬────┬────╮
│ # │ a  │ b  │
├───┼────┼────┤
│ 0 │  1 │  2 │
│ 1 │ ❎ │ ❎ │
╰───┴────┴────╯

or

╭───┬────┬────╮
│ # │ a  │ c  │
├───┼────┼────┤
│ 0 │  1 │ ❎ │
│ 1 │ ❎ │  3 │
╰───┴────┴────╯

Screenshots

No response

Configuration

key	value
version	0.86.0
branch	makepkg
commit_hash	5d8763e
build_os	linux-x86_64
build_target	x86_64-unknown-linux-gnu
rust_version	rustc 1.73.0 (cc66ad468 2023-10-03) (Arch Linux rust 1:1.73.0-1)
cargo_version	cargo 1.73.0
build_time	2023-10-18 08:45:09 +00:00
build_rust_channel	release
allocator	mimalloc
features	dataframe, default, extra, sqlite, trash, which, zip
installed_plugins

Additional context

No response

[amtoine]

yeah that sounds like a bug, good catch 😮