format: a log can be six shapes, and it knows what time it is - #28
Merged
Conversation
Adds entry_format to the log format, with apache-combined staying the default, and six more settings beside it: timestamps, rate, methods, status_mix, ip_version and line_ending. The text group had no settings at all until now, so this is the first of them and the shape the rest will copy. Every template came from a real file rather than from a specification recalled, and two of them would have been wrong otherwise. A real nginx writes one more quoted field than "combined" does, because its default log_format ends with $http_x_forwarded_for. Apache's own default is common, with no referrer and no agent at all. A third thing, less obvious: a syslog FILE carries no priority in angle brackets - that belongs on the wire - and no line in the RFC 3164 style exists on this machine at all, so the ISO form is what a tester actually sees. Sources and samples are in docs/MVP-FORMATS.md section 5.1a. The clock advances now, which moves the bytes of every log and is listed under Breaking. timestamps=fixed reproduces the old file to the byte, and there is a pinned hash for it rather than a promise - the value log_8kib carried before this change is now log_8kib_the_way_back. A setting that could do nothing for the chosen shape is refused, naming both halves, rather than accepted and ignored. What "asked for" means took a correction, below. The structural checker is handed a format id and a path and never a recipe, so it takes the shape from the first entry and holds every other line to that one. Asking each line only to be valid on its own would pass a file that changed shape half way down. Five deliberate breakages, five caught: a truncated line, two shapes in one file, mixed endings, a missing final newline and an octet above 255. JSON lines is checked by Python's own json module, which gives this format its first reader that is not a regular expression of ours. Three defects found after the code was written, each worth its own note: A window could not produce syslog or JSON lines AT ALL. A menu cannot be empty - it opens on its declared default - so a window sends every setting it draws, and the first version refused a setting that could do nothing whenever the KEY arrived. The command line never showed it, because there an unset flag is an absent key, and every test written before the report had the command line's shape. A value equal to the default is not something anybody asked for, and it cannot disagree with the shape. The folded summary line listed the format's whole declaration rather than what somebody chose, for the same reason. Nobody noticed while formats declared one or two settings. With seven the line ran off the edge of the window. Boxes people type into are left as they were: there an empty box and a typed default really do differ. syslog missed its size in about one file in ten. The line counted its process id as four digits always, and it runs from 100 to 9998. Only the last entry is built to a length, so the miss needed that entry to draw a short pid. Measured before the repair: 35 files out of 360, and this project's own size guard could not see it because it asks each format with its settings left alone. Nine mutations, all caught, including one for each of those three. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gates called parseOptions at 32 decision points against a ceiling of 22 and 86 lines against 79. Those ceilings are today's measurement and only ever go down, so the branches come out rather than the number going up. One reader per setting now, run in order - and the order is the point, since the shape and the clock have to be settled before anything can be asked whether it disagrees with them. Two more things the same run named. The check that refuses a chosen setting for a shape that cannot use it was written twice as an if inside an if, so it is one helper now. And writing an address nested a condition inside a loop inside a condition, which splits cleanly by family - the two halves had nothing in common but the name. No behaviour moves: every pinned hash is unchanged and the mutation patterns all still find their line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
entry_formatto the log format -apache-combined(the default, unchanged),apache-common,nginx,syslog,plain,json-lines- and six settings beside it:timestamps,rate,methods,status_mix,ip_version,line_ending.The text group had no settings at all until now, so this is the first, and the shape the rest will copy.
Measured, not remembered
Every template came from a real file, and two would have been wrong otherwise:
log_formatends with$http_x_forwarded_for, so a real line carries one more quoted field than "combined"common- no referrer, no agent<PRI>- that is on the wire - and no RFC 3164 style line exists on this machine, so ISO is what a tester seesSources and samples:
docs/MVP-FORMATS.md §5.1a.Breaking, with a way back that is pinned
The clock advances now, one second apart by default. That moves the bytes of every log, so it is under
### Breakingwith the version untouched - the animated GIF's pattern.timestamps=fixedreproduces the old file to the byte, and there is a hash for it rather than a promise: the valuelog_8kibcarried before this change is now pinned aslog_8kib_the_way_back.The oracle now detects the shape
It is handed a format id and a path, never a recipe - so it takes the shape from the first entry and holds every line to that one. Asking each line only to be valid on its own would pass a file that changed shape half way down.
Five deliberate breakages, five caught: truncated line, two shapes in one file, mixed endings, missing final newline, octet above 255. JSON lines goes through Python's own
jsonmodule, giving this format its first reader that is not a regular expression of ours.Three defects found after the code was written
A window could not produce syslog or JSON lines at all. A menu cannot be empty - it opens on its declared default - so a window sends every setting it draws. The first version refused a setting that could do nothing whenever the key arrived. The command line never showed it, because there an unset flag is an absent key, and every test I had written used the command line's shape of a request. Reported from a screenshot.
The folded summary line listed the format's whole declaration rather than what anybody chose, for the same reason. Unnoticed while formats declared one or two settings; with seven it ran off the edge of the window. Boxes people type into are left alone - there an empty box and a typed default genuinely differ (
defaulted, untouchable rule 5).syslog missed its size in about one file in ten. The line counted its process id as four digits always, and it runs from 100 to 9998. Only the last entry is built to a length, so the miss needed that entry to draw a short pid. Measured before the repair: 35 files out of 360 - and this project's own size guard could not see it, because it asks each format with its settings left alone, which for this one is
apache-combined.Guards
Nine mutations, all caught, including one per defect above. New guards: every shape hits its size across seeds, every shape is reachable when every setting is sent the way a window sends them, the clock advances by exactly one tick per entry, the way back holds one instant, an impossible pair is refused naming both halves, the label is a line its own reader accepts, and the folded line names what was chosen.
.gitignoregains/tfg-out/- the window proposes it by default, so running the program from a checkout was filling the repository with generated files.