Skip to content

format: a log can be six shapes, and it knows what time it is - #28

Merged
donislawdev merged 2 commits into
mainfrom
log-entry-formats
Sep 1, 2026
Merged

format: a log can be six shapes, and it knows what time it is#28
donislawdev merged 2 commits into
mainfrom
log-entry-formats

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

Adds entry_format to the log format - apache-combined (the default, unchanged), apache-common, nginx, syslog, plain, json-lines - and six settings beside it: timestamps, rate, methods, status_mix, ip_version, line_ending.

The text group had no settings at all until now, so this is the first, and the shape the rest will copy.

Measured, not remembered

Every template came from a real file, and two would have been wrong otherwise:

shape what measuring corrected
nginx its default log_format ends with $http_x_forwarded_for, so a real line carries one more quoted field than "combined"
apache the image's own default is common - no referrer, no agent
syslog a file carries no <PRI> - that is on the wire - and no RFC 3164 style line exists on this machine, so ISO is what a tester sees

Sources and samples: docs/MVP-FORMATS.md §5.1a.

Breaking, with a way back that is pinned

The clock advances now, one second apart by default. That moves the bytes of every log, so it is under ### Breaking with the version untouched - the animated GIF's pattern.

timestamps=fixed reproduces the old file to the byte, and there is a hash for it rather than a promise: the value log_8kib carried before this change is now pinned as log_8kib_the_way_back.

The oracle now detects the shape

It is handed a format id and a path, never a recipe - so it takes the shape from the first entry and holds every line to that one. Asking each line only to be valid on its own would pass a file that changed shape half way down.

Five deliberate breakages, five caught: truncated line, two shapes in one file, mixed endings, missing final newline, octet above 255. JSON lines goes through Python's own json module, giving this format its first reader that is not a regular expression of ours.

Three defects found after the code was written

A window could not produce syslog or JSON lines at all. A menu cannot be empty - it opens on its declared default - so a window sends every setting it draws. The first version refused a setting that could do nothing whenever the key arrived. The command line never showed it, because there an unset flag is an absent key, and every test I had written used the command line's shape of a request. Reported from a screenshot.

The folded summary line listed the format's whole declaration rather than what anybody chose, for the same reason. Unnoticed while formats declared one or two settings; with seven it ran off the edge of the window. Boxes people type into are left alone - there an empty box and a typed default genuinely differ (defaulted, untouchable rule 5).

syslog missed its size in about one file in ten. The line counted its process id as four digits always, and it runs from 100 to 9998. Only the last entry is built to a length, so the miss needed that entry to draw a short pid. Measured before the repair: 35 files out of 360 - and this project's own size guard could not see it, because it asks each format with its settings left alone, which for this one is apache-combined.

Guards

Nine mutations, all caught, including one per defect above. New guards: every shape hits its size across seeds, every shape is reachable when every setting is sent the way a window sends them, the clock advances by exactly one tick per entry, the way back holds one instant, an impossible pair is refused naming both halves, the label is a line its own reader accepts, and the folded line names what was chosen.

.gitignore gains /tfg-out/ - the window proposes it by default, so running the program from a checkout was filling the repository with generated files.

donislawdev and others added 2 commits September 1, 2026 08:10
Adds entry_format to the log format, with apache-combined staying the default,
and six more settings beside it: timestamps, rate, methods, status_mix,
ip_version and line_ending. The text group had no settings at all until now, so
this is the first of them and the shape the rest will copy.

Every template came from a real file rather than from a specification recalled,
and two of them would have been wrong otherwise. A real nginx writes one more
quoted field than "combined" does, because its default log_format ends with
$http_x_forwarded_for. Apache's own default is common, with no referrer and no
agent at all. A third thing, less obvious: a syslog FILE carries no priority in
angle brackets - that belongs on the wire - and no line in the RFC 3164 style
exists on this machine at all, so the ISO form is what a tester actually sees.
Sources and samples are in docs/MVP-FORMATS.md section 5.1a.

The clock advances now, which moves the bytes of every log and is listed under
Breaking. timestamps=fixed reproduces the old file to the byte, and there is a
pinned hash for it rather than a promise - the value log_8kib carried before
this change is now log_8kib_the_way_back.

A setting that could do nothing for the chosen shape is refused, naming both
halves, rather than accepted and ignored. What "asked for" means took a
correction, below.

The structural checker is handed a format id and a path and never a recipe, so
it takes the shape from the first entry and holds every other line to that one.
Asking each line only to be valid on its own would pass a file that changed
shape half way down. Five deliberate breakages, five caught: a truncated line,
two shapes in one file, mixed endings, a missing final newline and an octet
above 255. JSON lines is checked by Python's own json module, which gives this
format its first reader that is not a regular expression of ours.

Three defects found after the code was written, each worth its own note:

  A window could not produce syslog or JSON lines AT ALL. A menu cannot be
  empty - it opens on its declared default - so a window sends every setting it
  draws, and the first version refused a setting that could do nothing whenever
  the KEY arrived. The command line never showed it, because there an unset
  flag is an absent key, and every test written before the report had the
  command line's shape. A value equal to the default is not something anybody
  asked for, and it cannot disagree with the shape.

  The folded summary line listed the format's whole declaration rather than
  what somebody chose, for the same reason. Nobody noticed while formats
  declared one or two settings. With seven the line ran off the edge of the
  window. Boxes people type into are left as they were: there an empty box and
  a typed default really do differ.

  syslog missed its size in about one file in ten. The line counted its process
  id as four digits always, and it runs from 100 to 9998. Only the last entry
  is built to a length, so the miss needed that entry to draw a short pid.
  Measured before the repair: 35 files out of 360, and this project's own size
  guard could not see it because it asks each format with its settings left
  alone.

Nine mutations, all caught, including one for each of those three.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gates called parseOptions at 32 decision points against a ceiling of 22 and
86 lines against 79. Those ceilings are today's measurement and only ever go
down, so the branches come out rather than the number going up.

One reader per setting now, run in order - and the order is the point, since
the shape and the clock have to be settled before anything can be asked whether
it disagrees with them.

Two more things the same run named. The check that refuses a chosen setting for
a shape that cannot use it was written twice as an if inside an if, so it is one
helper now. And writing an address nested a condition inside a loop inside a
condition, which splits cleanly by family - the two halves had nothing in common
but the name.

No behaviour moves: every pinned hash is unchanged and the mutation patterns all
still find their line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit 6355609 into main Sep 1, 2026
30 of 31 checks passed
@donislawdev
donislawdev deleted the log-entry-formats branch September 1, 2026 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant