Performance and Benchmarks

Performance & Benchmark Methodology

The C extension is the gem's selling point, so the numbers matter — and so does being able to reproduce them. This page covers how they're measured and what they mean. The full data set and every prototype comparison is in the research log §7–§8; the per-release numbers are in the CHANGELOG.

Reproducing the benchmarks

The benchmark/ directory holds scripts that measure the engine from different angles. They are not packaged with the gem.

bundle install                              # pulls benchmark-ips, benchmark-memory (dev deps)
bundle exec rake compile
bundle exec ruby benchmark/vs_pure_ruby.rb  # head-to-head vs pure-Ruby gsub, same patterns
bundle exec ruby benchmark/throughput.rb    # MB/s on a log line, JSON, 1MB and 10MB logs
bundle exec ruby benchmark/scaling.rb       # runtime vs input size (1KB → 50MB) — linearity
bundle exec ruby benchmark/per_pattern.rb   # per-pattern scan cost over a 1MB payload

See benchmark/README.md for what each script measures.

Methodology (how the numbers stay honest)

• Realistic payload. ~1 MB of mostly-noise lorem ipsum text with sensitive strings embedded at ~1 per 5,600 bytes — a realistic "log file" workload where the scan almost never fires. Construction is in research log §7.1.
• The pure-Ruby baseline reads the same patterns. The comparison isn't against a strawman: the baseline is a gsub loop over DataRedactor::BUILTIN_PATTERN_SOURCES — literally the same pattern set the C engine uses, via Ruby's own Onigmo. Equivalent work, different engine.
• Correctness gate. Every payload size passes a check that the redaction count matches the pure-Ruby gsub result (a superset relation on match spans — see research log §7.3). A faster wrong answer doesn't count.
• Timing. Benchmark.realtime averaged over iterations, single-process, sequential. Numbers are stable to within ~10% across runs. Absolute MB/s is machine-dependent; run it on your own hardware.

Results (0.10.0 — v19 engine)

vs a pure-Ruby gsub loop over the same patterns:

Payload	v19 engine	Pure-Ruby gsub	Ratio
log line (168 B)	41 µs	71 µs	1.7× faster
JSON blob (~580 B)	81 µs	132 µs	1.6× faster
8 log lines (1.3 KB)	175 µs	399 µs	2.3× faster
100 log lines (17 KB)	2.0 ms	4.6 ms	2.3× faster
1 MB log	138 ms	294 ms	2.1× faster
10 MB log	1.44 s	—	6.9 MB/s

The previous engine (per-pattern regexec) was 4.25× slower than pure Ruby on the 1 MB payload — a ~9× swing to v19.

Linear scaling (the important curve)

The single-pass engine is O(N): a 10× bigger payload takes ~10× longer and MB/s holds flat. The old per-pattern regexec engine was O(N²) and fell off a cliff on large inputs.

Size	Time	MB/s
1 KB	0.14 ms	7.1
100 KB	13.4 ms	7.3
1 MB	142 ms	7.0
10 MB	1.42 s	7.0
50 MB	7.14 s	7.0

Absolute MB/s is machine-dependent, but the flat curve is not — that's the claim worth reproducing. There are no published benchmarks for comparable Ruby PII-redaction gems, so these are absolute (vs pure-Ruby gsub), not a gem-to-gem head-to-head.

Why it's fast (in one line)

Single pass over the buffer, per-pattern lazy DFA transition caching, and two merge passes that collapse the digit/IBAN pattern classes. The full story — including the ideas that didn't pan out — is in C Engine Internals and the research log.