Skip to content

v0.9.6 - open a CSV that breaks the standard

Choose a tag to compare

@Kenzy-Zero Kenzy-Zero released this 05 Aug 12:21
· 5 commits to main since this release

Found by using kenze for real work: a plain Spark-written CSV could not be opened by any documented route.

Added

  • --no-strict-csv - read a CSV that breaks the CSV standard: mixed and line endings, or a stray quote in an unquoted field. Files written by Spark (part-00000-<uuid>-c000.csv) do this routinely, and until now kenze could not open them at all - the parser fails in its state machine before there is any row to skip, so neither --skip-bad-lines nor --errors could rescue the file. Both failed byte-identically to the plain call.
    • It is a separate flag on purpose. --skip-bad-lines drops a row it cannot parse and --errors hands it back to you; relaxing the standard instead accepts a non-conforming row and silently discards any fields past the header. Folding that into the existing flags would have made a broken file look clean.
    • Shell: set strict-csv off. Python: strict=False.

Fixed

  • The global CSV flags now reach the read-only commands. profile, peek, stats, plot, count and validate never accepted --skip-bad-lines or --errors - the flags parsed and were then dropped before the command ran. These are the commands you reach for first on a file that will not open.
  • kenze check no longer dies on the file it is diagnosing, and now names which kind of damage it found.
  • CSV errors now name a flag you can actually type. DuckDB reports the fix in its own terms (strict_mode=false), which no kenze user can pass.

Full notes in CHANGELOG.md.