Hold one engine against another without averaging reads into writes - #122
Merged
Conversation
A mixed workload's speedup over a rival is two questions wearing one number. On reads almost all the latency is the engine's own work, so a ratio there says what the two engines are worth. On writes a durable commit costs one flush of the drive and no engine goes below it, so two correct engines on one disk converge however good either is. LinkBench is thirty percent durable commits, which caps a blended ratio for it at the drive rather than at either engine. So gate --rival splits it. Reads are gated as a ratio, --read-speedup, default ten. Writes are gated in units of one durable sync on the volume the run wrote to, --commit-syncs, default two at the median, and against the rival's count at both statistics. One sync is the floor since a durable commit has to flush, and two is what a writer pays arriving an instant after a flush began. Two things it has to get right to be honest. Both sides are divided by one number when both runs are on one machine, the cheaper of the two probes, because otherwise the two columns of one table are in different units and whichever engine drew the dearer probe gets a smaller count for free. This laptop probed the same volume at 3.02ms under one run and 4.35ms under another, which is a wider spread than the gap between passing the ceiling and failing it. And where a flush never reaches a device the write half declines to rule rather than deciding: WSL2 on a virtual disk probes at 316ns, which would read a healthy 132us commit as four hundred syncs of work. The comparison prints whether it passed or not. A run that beat a rival by a hundred times said something worth reading, and a gate that only prints its failures throws it away.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
zu#78 asks for linkbench at ten times a rival at a concurrency of eight, and that number cannot be reached by any engine on any disk. LinkBench is thirty percent durable commits. A commit costs one flush of the drive, no engine goes below it, and two correct engines on one volume converge there however good either is. Even at a perfect write p50 of one sync the weighted ratio caps at 12.5x, and at the honest floor of two syncs it caps at 6.5x. A single blended number for that workload is measuring the drive and calling it the engine.
This adds
gate --rival other.json, which splits the comparison into the two questions it was.Reads are gated as a ratio against the rival,
--read-speedup, default ten. That is where a ratio means something, because almost all of a read's latency is the engine's own work.Writes are gated in units of one durable sync on the volume the run wrote to, which is the part of a commit the engine decides. At or under
--commit-syncs, default two, at the median, and no more syncs a commit than the rival at either statistic. One sync is the floor since a durable commit has to flush. Two is what a writer pays arriving an instant after a flush began and waiting that one out before its own. A median above two is commits that are not grouping, which is the engine's to fix and not the drive's.Two things the split has to get right to be worth trusting.
When both runs are on one machine both sides are divided by one number, the cheaper of the two probes. The probe is already a median of fifteen flushes and it still moves: this laptop came back at 3.02ms under one run and 4.35ms under another, a spread wider than the gap between passing the ceiling and failing it. Dividing each engine by its own probe would print two columns of one table in two different units and hand whichever engine drew the dearer probe a smaller count for free.
Where a flush never reaches a device the write half declines to rule. WSL2 on a virtual disk probes a durable sync at 316ns, which is a memory copy, and it would read a healthy 132us commit as four hundred syncs of work. A run there is reported indeterminate, not failed.
The comparison prints whether it passed or not, because a run that beat a rival by a hundred times said something worth reading.
Here is a real one, linkbench smoke at count 8000 and a concurrency of eight, one engine per invocation, on an M4 where a durable sync is a real F_FULLFSYNC:
The read half clears ten times by two orders of magnitude. The write half is where the old blended number was going to die, and in syncs it reads as what it is rather than as a failure to be fast.
Nine tests cover the split, the two gates, the shared unit, the two hosts case, the unprobed volume, the volume that does not flush, and the printed form.
go build ./...andgo test ./...pass,gofmt -lis clean.