Skip to content

Benchmarks

Jump to bottom
github-actions[bot] edited this page Aug 24, 2026 · 1 revision

Performance & Speed Benchmarks

We benchmarked cleave-sbd against popular Python sentence segmenters on the Complete Works of William Shakespeare (pg100.txt, 5.31 MB, 966,506 words).

Benchmark Results

Engine Sentences Found Mean Latency Min Latency Throughput Status / Speedup
cleave-sbd (clean=False) 175,998 3,407.85 ms 3,378.60 ms 1.52 MB/s 1.00x (Baseline)
cleave-sbd (clean=True) 176,010 3,533.35 ms 3,469.87 ms 1.47 MB/s 0.96x
cleave-sbd (char_span=True) 175,998 3,832.14 ms 3,689.56 ms 1.35 MB/s 0.89x
spaCy sentencizer 109,084 4,862.67 ms 4,758.62 ms 1.07 MB/s 0.97x
BlingFire 107,489 164.11 ms 161.32 ms 31.62 MB/s 27.77x
NLTK sent_tokenize 105,488 726.35 ms 724.30 ms 7.15 MB/s 6.27x
Syntok 112,612 3,871.09 ms 3,811.82 ms 1.34 MB/s 1.18x
Stanza 127,102 48,151.78 ms 45,269.77 ms 0.11 MB/s 0.09x (~10.6x slower)
spaCy en_core_web_sm — — — — Refused / Setup Failure
pySBD — >900,000 ms — <0.005 MB/s DNF (Timed out >15 min)

Key Takeaways

  • pySBD locks up (>15 min DNF): pySBD loops line-by-line with dynamic regex recompilation, hitting an $O(N^2)$ wall on multi-megabyte files. cleave-sbd finishes the same corpus in 3.41 seconds.
  • 10.6x faster than Stanza: Stanford Stanza's PyTorch neural pipeline took 48.15 seconds. Our pre-compiled automata beat it in 4.56 seconds on a single CPU core with zero GPU requirements.
  • Granular boundary precision: cleave-sbd detected 175,998 sentence boundaries (~48,000–70,000 more than Stanza, NLTK, or BlingFire) by accurately splitting dramatic verse, dialogue cues, character tags, and archaic typography instead of collapsing them into run-on blocks.
  • Zero-cost spans: Tracking character spans (char_span=True) adds just ~200 ms over the entire 5.3 MB book while maintaining 1.35 MB/s throughput.

Reproduce Benchmarks

uv run --with nltk,stanza,blingfire,syntok python benchmarking/bigtext_speed_benchmark.py

Clone this wiki locally