BREAKING(csv): honour fieldsPerRecord when mapping rows to objects#7141
Open
fibibot wants to merge 1 commit into
Open
BREAKING(csv): honour fieldsPerRecord when mapping rows to objects#7141fibibot wants to merge 1 commit into
fieldsPerRecord when mapping rows to objects#7141fibibot wants to merge 1 commit into
Conversation
…parse()` and `CsvParseStream` When `skipFirstRow` or `columns` is set together with `fieldsPerRecord: -1` (or `fieldsPerRecord` left undefined, which is the same variable-length mode), `convertRowToObject` previously threw "The record has X fields, but the header has Y fields" unconditionally on length mismatch, ignoring the `fieldsPerRecord` setting. It now respects the mode: when variable-length records are permitted, short rows yield `undefined` for missing header keys and extra fields beyond the header list are dropped. Strict modes (`fieldsPerRecord >= 0`) keep the existing length check. The mapped-row value type widens from `Record<string, string>` to `Record<string, string | undefined>` (and the same shift in `RecordWithColumn`) so the static type reflects the runtime behaviour. This is a TS-level breaking change for callers reading values cookie-style without `noUncheckedIndexedAccess`. Fixes #6434.
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7141 +/- ##
=======================================
Coverage 94.61% 94.61%
=======================================
Files 634 634
Lines 51830 51838 +8
Branches 9341 9342 +1
=======================================
+ Hits 49037 49045 +8
Misses 2218 2218
Partials 575 575 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #6434.
CsvParseStreamandparse()documentfieldsPerRecord: -1(or undefined,the default) as allowing variable-length records, and that worked when
emitting
string[][]. OnceskipFirstRow: trueorcolumns: [...]was set,parsing threw
Syntax error on line N: The record has X fields, but the header has Y fieldseven withfieldsPerRecord: -1, becauseconvertRowToObjectthrew unconditionally on length mismatch, ignoring thefieldsPerRecordsetting.This change makes
convertRowToObjectaccept anallowVariableLengthflagthat both call sites compute from the active
fieldsPerRecordmode (thestream's
#fieldsPerRecord === "ANY"; for the syncparse(),options.fieldsPerRecord === undefined || options.fieldsPerRecord < 0).Runtime behaviour:
fieldsPerRecordundefined or negative): short rowsyield
undefinedfor missing header keys, extra fields are dropped.fieldsPerRecord >= 0): unchanged. The existingexpected N fields but got Mcheck still fires beforeconvertRowToObject, and the residual length-vs-header check still throwswhen
columnsdiffers in length from a positivefieldsPerRecord.Type widening (BREAKING)
The mapped-row value type widens from
Record<string, string>toRecord<string, string | undefined>(and the same shift insideRecordWithColumn) so callers see the runtime possibility ofundefined.This is a TS-level breaking change for callers reading values cookie-style
without
noUncheckedIndexedAccess. The issue calls this out; wideningunconditionally was preferred over gating on
fieldsPerRecordbecause thedefault mode (no
fieldsPerRecordset) is variable-length, so any callerusing
skipFirstRoworcolumnswith the defaultfieldsPerRecordcan nowobserve
undefinedand the type should reflect that.Test plan
deno fmt --check csv/deno lint csv/deno test -A --doc csv/(72 tests / 199 steps, all passing)fieldsPerRecorddoesn't work withskipFirstRoworcolumns#6434(skipFirstRow / columns / both) in both
csv/parse_test.tsandcsv/parse_stream_test.ts.fieldsPerRecorderror paths still tested.Closes bartlomieju/orchid-inbox#88