Skip to content

Commit

Permalink
csv: allow csv records with varying lengths, padding with empties
Browse files Browse the repository at this point in the history
Sometimes trailing empty fields are omitted entirely (including the
commas) in CSV records. (I see this in exported Google spreadsheets.)
Now we don't raise an error in this case, instead we automatically pad
any "short" records with empty fields. Not yet well tested.
  • Loading branch information
simonmichael committed Oct 8, 2019
1 parent 6dcddad commit eff1b31
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions hledger-lib/Hledger/Read/CsvReader.hs
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -222,14 +222,17 @@ validateCsv numhdrlines (Right rs) = validate $ drop numhdrlines $ filternulls r
where where
filternulls = filter (/=[""]) filternulls = filter (/=[""])
validate [] = Right [] validate [] = Right []
validate rs@(first:_) validate rs@(_first:_)
| isJust lessthan2 = let r = fromJust lessthan2 in Left $ printf "CSV record %s has less than two fields" (show r) | isJust lessthan2 = let r = fromJust lessthan2 in
| isJust different = let r = fromJust different in Left $ printf "the first CSV record %s has %d fields but %s has %d" (show first) length1 (show r) (length r) Left $ printf "CSV record %s has less than two fields" (show r)
-- | isJust different = let r = fromJust different in
-- Left $ printf "the first CSV record %s has %d fields but %s has %d"
-- (show first) length1 (show r) (length r)
| otherwise = Right rs | otherwise = Right rs
where where
length1 = length first
lessthan2 = headMay $ filter ((<2).length) rs lessthan2 = headMay $ filter ((<2).length) rs
different = headMay $ filter ((/=length1).length) rs -- length1 = length first
-- different = headMay $ filter ((/=length1).length) rs


-- -- | The highest (0-based) field index referenced in the field -- -- | The highest (0-based) field index referenced in the field
-- -- definitions, or -1 if no fields are defined. -- -- definitions, or -1 if no fields are defined.
Expand Down

0 comments on commit eff1b31

Please sign in to comment.