Extracting data from partial tables results in error

[martinrode]

Describe the bug
Using "from json" my data contains array of objects. Not all of these objects have the same properties. Extracting a partially available property using "get object.rare_prop" results in an error.

This makes "from json | get ..." a lot less usable for me as our data is diverse and I want to be able to use nushell to analyze that data more easily.

> curl ... | from json 
───┬───────────┬──────────┬─────────────┬──────────┬──────────┬─────────────┬───────────┬─────────────┬─────────────┬─────────────
 # │ _basetype │   user   │ _generated_ │ _groups  │   _acl   │ _system_rig │  _owner   │ _collection │ _created_at │ _updated_at 
   │           │          │   rights    │          │          │     hts     │           │ _pin_codes  │             │             
───┼───────────┼──────────┼─────────────┼──────────┼──────────┼─────────────┼───────────┼─────────────┼─────────────┼─────────────
 0 │ user      │ [row 30  │ [row delete │ [table 0 │ [table 0 │ [row ]      │ [row      │ [table 0    │ 2021-03-18T │ 2021-03-18T 
   │           │ columns] │ read write] │ rows]    │ rows]    │             │ _basetype │ rows]       │ 11:03:13.57 │ 11:03:13.57 
   │           │          │             │          │          │             │ user]     │             │ 6225+01:00  │ 7399+01:00  
 1 │ user      │ [row 30  │ [row delete │ [table 0 │ [table 0 │ [row        │ [row      │ [table 0    │ 2021-03-18T │ 2021-03-18T 
   │           │ columns] │ read write] │ rows]    │ rows]    │ system.fron │ _basetype │ rows]       │ 11:03:13.58 │ 11:03:24.54 
   │           │          │             │          │          │ tend_featur │ user]     │             │ 8234+01:00  │ 8302+01:00  
   │           │          │             │          │          │ es          │           │             │             │             
   │           │          │             │          │          │ system.sear │           │             │             │             
   │           │          │             │          │          │ ch          │           │             │             │             
   │           │          │             │          │          │ system.user │           │             │             │             
   │           │          │             │          │          │ ]           │           │             │             │             
───┴───────────┴──────────┴─────────────┴──────────┴──────────┴─────────────┴───────────┴─────────────┴─────────────┴─────────────

───┬───────────┬──────────┬─────────────┬──────────┬──────────┬────────────┬───────────┬────────────┬────────────┬────────────┬────────────
 # │ _basetype │   user   │ _generated  │ _groups  │   _acl   │ _system_ri │  _owner   │ _collectio │ _created_a │ _updated_a │ _archived_ 
   │           │          │   _rights   │          │          │    ghts    │           │ n_pin_code │     t      │     t      │     at     
   │           │          │             │          │          │            │           │     s      │            │            │            
───┼───────────┼──────────┼─────────────┼──────────┼──────────┼────────────┼───────────┼────────────┼────────────┼────────────┼────────────
 2 │ user      │ [row 30  │ [row        │ [table 0 │ [table 0 │ [row       │ [row      │ [table 0   │ 2021-03-18 │ 2021-03-18 │ 2021-03-18 
   │           │ columns] │ delete read │ rows]    │ rows]    │ system.sea │ _basetype │ rows]      │ T11:03:13. │ T11:03:24. │ T11:03:24. 
   │           │          │ write]      │          │          │ rch        │ user]     │            │ 597015+01: │ 704966+01: │ 704965+01: 
   │           │          │             │          │          │ system.use │           │            │ 00         │ 00         │ 00         
   │           │          │             │          │          │ r]         │           │            │            │            │            
───┴───────────┴──────────┴─────────────┴──────────┴──────────┴────────────┴───────────┴────────────┴────────────┴────────────┴────────────

───┬───────────┬──────────┬─────────────┬──────────┬──────────┬─────────────┬───────────┬─────────────┬─────────────┬─────────────
 # │ _basetype │   user   │ _generated_ │ _groups  │   _acl   │ _system_rig │  _owner   │ _collection │ _created_at │ _updated_at 
   │           │          │   rights    │          │          │     hts     │           │ _pin_codes  │             │             
───┼───────────┼──────────┼─────────────┼──────────┼──────────┼─────────────┼───────────┼─────────────┼─────────────┼─────────────
 3 │ user      │ [row 30  │ [row delete │ [table 0 │ [table 0 │ [row ]      │ [row      │ [table 0    │ 2021-03-18T │ 2021-03-18T 
   │           │ columns] │ read write] │ rows]    │ rows]    │             │ _basetype │ rows]       │ 11:03:13.60 │ 11:03:13.60 
   │           │          │             │          │          │             │ user]     │             │ 4045+01:00  │ 4632+01:00  
───┴───────────┴──────────┴─────────────┴──────────┴──────────┴─────────────┴───────────┴─────────────┴─────────────┴─────────────

/Users/martin> curl ... | from json | get _archived_at
2021-03-18T11:03:24.704965+01:00error: Unknown column
   ┌─ shell:50:179
   │
50 │ curl -s -X 'GET' -d '' -H 'Content-Type: application/json' -H 'X-Easydb-Token: 9058a983-b1df-4c83-a970-7a96e2c96337' 'http://localhost/api/v1/user?type=easydb' | from json | get _archived_at
   │                                                                                                                                                                                   ^^^^^^^^^^^^
   │                                                                                                                                                                                   │
   │                                                                                                                                                                                   There isn't a column named '_archived_at'
   │                                                                                                                                                                                   Perhaps you meant '_created_at'? Columns available: _basetype, user, _generated_rights, _groups, _acl, _system_rights, _owner, _collection_pin_codes, _created_at, _updated_at

I think "get" should output a "" or "" or "" for values which cannot be found

[martinrode]

turns out I can use "select" which omits empty fields.

Not sure if this is a worthwhile fix / change for get. So I leave the ticket open, just in case... Personally I will use select, so I don't need get to be fixed / changed.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[sophiajt]

This part of the design is intentional so that someone working with data columns knows that the column is partial. Like you found, you can select out the column, and then us commands like compact if you want to remove the blanks before processing.

This may be more of a documentation thing, so we explain why this case errors, rather than a code issue. Closing this one, but I've created a documentation bug for the docs.