COMPASS-3826 COMPASS-3947: Import preview table with project and type customization

[imlucas]

Description

Adds a preview table of the file you are importing with checkboxes to exclude fields and select to specify a different type.

Motivation and Context

Using Import to Reshape Apple Health Data Export

• https://www.ryanpraski.com/apple-health-data-how-to-export-analyze-visualize-guide/
• xml → csv delimiter:;

Default to CSV:

Tweak delimiter so it looks right:

Specified field transforms:

const fields = [
  { path: 'type', checked: false, type: 'String' },
  { path: 'sourceName', checked: true, type: 'String' },
  { path: 'sourceVersion', checked: true, type: 'Number' },
  { path: 'creationDate', checked: true, type: 'Date' },
  { path: 'startDate', checked: true, type: 'Date' },
  { path: 'endDate', checked: true, type: 'Date' }
];

Import done:

CRUD has my enriched types:

And the schema view is super pretty:

Types of changes

• Minor (non-breaking change which adds functionality)

Open questions

• Change to utils/import/*.js instead of utils/import-*.js?
• The preview could have partial objects if spill over into next buffer. Can we back pressure against the input source for real instead of this hacky impl?
• move EJSON.deserialize() from parsers to apply-import-type-and-projection?
• better name for apply-import-type-and-projection?

[lrlna]

Left a few comments on the UX side of things!

In response to your comments from above:

Change to utils/import/.js instead of utils/import-.js?

I like the import-*.js, but that could be just be me!

The preview could have partial objects if spill over into next buffer. Can we back pressure against the input source for real instead of this hacky impl?

Replied in comments! tl;dr: i think peek-stream is meant to make sure we don't deal with partial objects.

move EJSON.deserialize() from parsers to apply-import-type-and-projection?

Not sure which EJSON.deserialize() you mean? There is a comment for progressStream in the parsers file, but I think it makes sense to have that with the parsers. Since when you parse, you have progress.

better name for apply-import-type-and-projection?

I think with the style of import-*.js style, would be good to have it as import-apply-type-and-projection. But otherwise it's nice and descriptive and I right away understand what's going on.