Skip to content

format: a CSV can be written in the dialect you were handed - #36

Merged
donislawdev merged 1 commit into
mainfrom
format/csv-settings
Sep 1, 2026
Merged

format: a CSV can be written in the dialect you were handed#36
donislawdev merged 1 commit into
mainfrom
format/csv-settings

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

Adds delimiter, line_ending and header to the csv format.

tfg generate --format csv --size 64kb --set delimiter=semicolon --set line_ending=crlf

Separators are named rather than typed - comma, semicolon, tab, pipe - so tab and pipe need no escaping on a command line or in a recipe.

No existing file changes by a byte. The defaults are comma, lf and a header, which is what this tool has always written. Checked against a binary built from main: four seeds by three sizes, twelve of twelve identical.

These are the three ways a real CSV differs before its contents do. A European spreadsheet exports with semicolons, anything written on Windows ends its rows with CRLF, and a table dumped out of a database has no header. All three are CSV and all three break a reader that assumed the other thing.

Two things that are not obvious from the list of settings

The filler has to emit the separator, not a comma. The description is a quoted field, and it is quoted only because it carries the separator. A filler that kept dropping commas would leave a semicolon file looking perfect - right size, right separators between the fields, every row the same width - while never exercising the quoted path the setting exists to test. Nothing else here would have seen it, so the guard counts separators in the closing row: six fields need five, and any more are inside the quotes.

The dialect moves the floor. A CRLF row is a byte longer and a header is a whole line.

lf crlf
with a header 117 B 119 B
without one 76 B 77 B

The separator does not move it, because all four are one byte. The registry announces the floor for the settings left alone and Plan works out the real one - the same arrangement the log format uses.

The checker is told the dialect, not left to guess

oracle.Strict now takes optional settings. A checker that sniffed the separator would agree with a file that used the wrong one: it would split on whatever it found and report a tidy table either way. Whether the file uses the separator that was ordered is a question for a guard reading the manifest, and there is one.

That the checker is not a rubber stamp is asked directly - a comma file handed to it as semicolon has to come back refused, and does.

What was checked

  • D11 against a main binary: 12 of 12 byte for byte
  • all 16 dialects: exact size over 24 runs each, and every one through the structural checker
  • the floor each dialect announces is accepted, and one byte below it is refused
  • the separator, the row ending and the header are read back out of the file, not the manifest
  • the manifest records the separator as the character while the recipe names it as a word
  • a value outside the closed set exits 4
  • 39 guard subtests, no skips; the whole affected guard set green
  • ten mutations, all caught
  • end to end through a recipe, not only through flags

New guards: TestTheCSVDialectIsInTheFileAndMovesTheFloor, TestEveryCSVDialectIsWellFormed, TestTheCSVManifestRecordsTheDialectAsFacts.

Still to come for this format

The quote character and escaping, and the column count. A row count disagrees with the exact size contract for the reason an entry count does in a log, and encoding waits for the piece that does TXT, where it is the headline setting rather than one of several.

🤖 Generated with Claude Code

Adds delimiter, line_ending and header to the csv format. Separators are
named rather than typed - comma, semicolon, tab and pipe - so tab and pipe
need no escaping on a command line or in a recipe. The defaults are comma,
lf and a header, which is what this tool has always written, so no
existing file changes by a byte: checked against a binary built from main,
four seeds by three sizes, twelve of twelve identical.

These are the three ways a real CSV differs before its contents do. A
European spreadsheet exports with semicolons, anything written on Windows
ends its rows with CRLF, and a table dumped out of a database has no
header. All three are CSV and all three break a reader that assumed the
other thing.

Two things are not obvious from the list of settings.

The filler has to emit the SEPARATOR rather than a comma. The description
is a quoted field and it is quoted only because it carries the separator,
so a filler that kept dropping commas would leave a semicolon file looking
perfect - right size, right separators between the fields, every row the
same width - while never exercising the quoted path the setting exists to
test. Nothing else here would have seen that, so the guard counts
separators in the closing row: six fields need five, and any more are
inside the quotes.

And the dialect moves the floor. A CRLF row is a byte longer and a header
is a whole line, so the four floors are 117, 119, 76 and 77 B rather than
one number. The registry announces the floor for the settings left alone
and Plan works out the real one, the same arrangement the log format uses.

The structural checker is now TOLD the dialect rather than sniffing it,
which needed Strict to take optional settings. A checker that guessed the
separator would agree with a file that used the wrong one - it would split
on whatever it found and report a tidy table either way. Whether the file
uses the separator that was ordered is a question for a guard reading the
manifest instead, and there is one. That the checker is not a rubber stamp
is asked directly: a comma file handed to it as semicolon has to come back
refused.

Still to come for this format: the quote character and escaping, and the
column count. A row count disagrees with the exact size contract for the
reason an entry count does in a log, and encoding waits for the piece that
does TXT, where it is the headline setting rather than one of several.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit a531708 into main Sep 1, 2026
18 checks passed
@donislawdev
donislawdev deleted the format/csv-settings branch September 1, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant