Repository navigation
Version 2.6.0
What's Changed
memtier_benchmark 2.6.0 adds --request-timeout, a sliding-window tail-latency row, a Memcached meta protocol mode and a native Windows build, and fixes many --test-time, reconnect, retry and cluster edge cases.
Read Upgrade notes first: several changes move benchmark numbers or reject command lines that 2.5.x accepted.
Upgrade notes
Command lines and exit codes
--requestis ambiguous (#903). It now abbreviates both--requestsand--request-timeout; use the full name.- Empty and out-of-range numbers are errors (#699, #721, #748, #761, #668). Empty values such as an unset
--max-reconnect-attempts=$VARor--max-retries=$VAR(previously 0) are rejected; pass0explicitly. Wrapped and overflowing values, ports outside 1-65535 and--reconnect-backoff-factorinfinity or underflow are rejected too.--multi-key-getis capped at 16777214 and--data-offsetat 536870911. --uri(#764, #720). Credentials are percent-decoded, so a literal%XXin a password must be written%25XX; bracketed IPv6 hosts work.--data-import(#767). Keys or values over 512 MiB are rejected and--generate-keysvalidates the file at startup.- Runs that exited 0 can now fail. Write errors on any output file or stdout (exit 1; a closed pipe reader such as
| headis exempt) (#729, #760, #717), verification mismatches or a verification pass with no replies (#771), connection rejections counted as ops (#897; exit 1, or 2 under unlimited reconnects after--connection-stage-timeout), and a closed stdout with no--out-file(#736). - Signals. SIGTERM is handled like Ctrl+C (#706).
Output consumers
- New fields. Top-level
FATAL_ERRORon failed runs (#25),TotalsErrors,Retry AttemptsandRetried Opswith--retry-on-error(#801),Unanswered requestsinRuntime(#669), and library versions (#823). - Changed values. Empty latency histograms keep their JSON keys and write
0.000(#793). Percentile keys are no longer truncated or colliding (#517), duplicate--commandkeys get a" #N"suffix (#588, #631), and invalid UTF-8 becomes U+FFFD (#741, #751).--uriand-avalues are redacted in--json-out-fileand--show-config(#735).
Build and runtime requirements
- libevent 2.0.11 or newer (#812). The default-on Prometheus exporter needs 2.1.4 or newer; on 2.1.1 to 2.1.3 use
--disable-prometheus(#823). libtool is no longer required (#762). - A failed
evthread_use_pthreads()is now a fatal error instead of a warning that could hang later (#823). memtier also warns when the loaded libevent or OpenSSL has a different major.minor than the one it was built against.
New Features
- Request stall timeout:
--request-timeout=SECS(default0, off) fails connections whose requests are never answered, for example a proxy that completes the TCP handshake but stops callingaccept(). It is a stall timeout, not a per-request deadline: a connection that keeps receiving reply data never trips it, a connection that never answers is detected afterSECS, and one that goes silent after answering withinSECSto2*SECS. A timeout is a connection error (reconnected under--reconnect-on-error, otherwise the run fails). Do not set it below the length of a legitimate blocking command such asBLPOPorWAIT. (#903) - Sliding-window tail latency:
--latency-window=SECS(default 60,0disables, maximum 3600) under--realtime-latenciesadds awin<N>srow with the percentiles of every sample from the last N seconds. Rows show their sample count (n=), and percentiles from too few samples are flagged*. Also fixes theoverallrow dropping samples recorded between a client's last snapshot and its per-second reset, a bug since 2.4.0. (#386, #824) - Configurable progress display:
--progress-interval=SECS(positive whole seconds, default 1; 0 is rejected) renders progress every SECS seconds while supervision and export keep their cadence. (#129) - Memcached meta protocol:
--protocol=memcache_metafor SET, GET and multi-GET against Memcached 1.6.10 and newer;--wait-ratioand--commandare rejected at startup in this mode. (#363) - Abstract Unix sockets:
--unix-socket @NAMEconnects to Linux abstract-namespace sockets (name up to 107 bytes; use./@NAMEfor a filesystem path that starts with@). (#257) - Native Windows build: build and run on Windows 10+ with MSYS2 UCRT64 and the autotools build. Unix sockets are rejected on Windows. (#553)
- Cursor walks for more types:
--scan-incremental-iterationfollows cursors forHSCAN,SSCANandZSCANas well asSCAN(standalone, single command, pipeline 1). GeneratedMATCHarguments now stay fixed for the whole walk, including forSCAN. (#539) - Structured failure summary: with
--json-out-file, a failed run records a top-levelFATAL_ERRORstring and keeps the configuration and results already written. Successful runs and graceful interrupts do not gain the field; JSON notices on stderr appear only with--debug. (#25) - Retry counters in JSON:
TotalsgainsErrors,Retry AttemptsandRetried Opsunder--retry-on-error. (#801) - RESP2 without
HELLO: with--protocol=resp2, a server that answers-ERR unknown command 'HELLO'(Redis older than 6.0, other RESP servers) is treated as RESP2 instead of stalling.--protocol=resp3still fails against such a server, now with a clearer message. (#884)
Bug Fixes
--test-time, reconnects and retries
- Timed-run end times: clients end at the later of their deadline and last reply, send no workload past the deadline, and exit
--clients-startruns at--test-time; the JSONStart time,Finish timeandTotal durationaverage all threads, not the last one. (#869, #876, #892, #722, #742, #723, #789) - Setup-only clients: a client that reconnects repeatedly but never submits workload no longer adds its setup interval to
Total duration, and a late first connect no longer ends a--reconnect-on-errorrun with zero requests. (#821, #865, #714) - Dead server: with
--reconnect-on-errorand unlimited attempts (the default), the--test-timedeadline wins over further reconnects and the run exits normally; a finite--max-reconnect-attemptsstill exits 1 when exhausted. (#666, #873, #812) --connection-stage-timeout: measured in microseconds, soNcan no longer abort after just over N-1 seconds; a worker that already ended is not also reported as an abort; exit 2 if any--run-countrun hits it. (#860, #891, #752)- Hangs on rejected setup: a run whose AUTH, SELECT or HELLO is rejected on only some connections, or on reconnect, now ends or reconnects instead of hanging; a retried
CLUSTER SLOTSreplaces a hang. (#750, #630, #667) - Connection rejections and errors:
-ERR Protocol error,-ERR max number of clientsand-DENIED ... protected modeare not counted as operations, ordinary error replies count as connection progress (#897, #788), and connection errors and Ctrl+C of clients that never connected are reported (#707, #698, #730). - Retry backoff: each retry waits its own
retry-backoff-ms * factor^resends; fractional factors are floored at 100 ms (or--retry-backoff-msif smaller),nan,inf, overflowing and empty factors are rejected. (#700, #727, #694, #692) - Retry accounting: a
--requestsrun no longer stops early, exits 0 having sent fewer commands than requested, or fails with "client(s) stopped before reaching their stop condition" when retries remain after the last request; in-flight requests dropped on disconnect no longer stall clients. (#559, #556) - Worker threads and transactions: worker threads are no longer silently re-run after connection errors (#546); a
--transactionblock is no longer continued after itsMULTIis lost to a reconnect, transaction requests are not retried, and planned--reconnect-intervalreconnects finish the rotation first (#607, #634, #615, #807, #606, #893).
Cluster mode
- Redirects and slot maps:
-MOVEDretries go to the node the reply names,-ASKretries sendASKINGfirst, empty hosts resolve to the replying connection, andCLUSTER SLOTSfollowsMOVEDeven with a full pipeline; refreshes no longer happen inside an openMULTI, twice concurrently or on every-READONLY, and back off per client onCLUSTERDOWN(1 s doubling to 8 s). (#622, #600, #787, #591, #576, #618, #662, #777, #877) - Connections and partial maps: only a dropped cluster connection is reconnected, and once; synchronous dial failures are retried; a refresh that retires its own connection no longer aborts the run; partial slot maps no longer leave workloads spinning, and a seed owning none of the keys no longer stalls. (#560, #574, #575, #584, #799, #612)
- Replica reads and statistics: keyless
--commandrequests stay off replicas unless replica reads are allowed, strict--read-preference=secondarywith no replicas names the cause, retried redirects count once and arbitrary-command redirects count inMOVED/secandASK/sec. (#887, #901, #605, #593, #582, #594, #583, #878)
Output, JSON and statistics
--run-count: BEST and WORST no longer always show run 1 and prefer complete runs (a run with a failed worker carriesInterrupted: true); the aggregateTotalsblock has a real histogram,KB/secand miss-rate warning, and latency is weighted by completed samples. (#571, #794, #572, #581, #573, #586, #804, #757)- Latency and counters: built-in
WAITreports real latency in JSON (#805); run statistics and hit, miss, MOVED and ASK counters are 64-bit, so totals no longer wrap on 32-bit and Windows builds (#552, #659). - JSON validity: strings, keys and non-UTF-8 bytes are escaped or replaced, deep-percentile keys no longer truncate or collide, and the writer is closed on early exit, leaving parseable (possibly configuration-only) JSON, so still check the exit status. (#555, #741, #751, #517, #588, #631, #772)
- Redaction:
--uricredentials and-aare redacted (user:***,***) in--json-out-fileand--show-config, so rerunning from the JSON needs-a/--uriagain. (#735, #784) - Prometheus:
memtier_latency_secondsrecords every completed operation once (#665); invalid UTF-8 in labels is replaced (#763); a bind failure reports the real reason (#872). --failed-keys-fileand HDR output:--commandrequests record their key, backslashes are escaped, long keys are not truncated, and HDR file names are sanitized with a_Nsuffix for repeats. (#599, #632, #629)- Output errors and standard streams: a write error fails the run and an unwritable
--out-fileor--failed-keys-fileis a fatal startup error (#729, #760, #717); closed stdin, stdout or stderr are reopened on/dev/null, and with stdout closed and no--out-filememtier fails at startup instead of losing results (#736).
Option validation
- Numeric options: wrapped, overflowing and empty values are rejected, including ports and the
--uridatabase number; leading whitespace and a single+are still accepted,--key-minimum=0and--reconnect-backoff-factor=0are accepted. (#699, #721, #748, #761, #782, #668) --uri: percent-decoded credentials and bracketed IPv6 hosts; a host-less--uriwarns and uses localhost. (#764, #720)- Data options: malformed
--data-importfiles fail fast instead of hanging or crashing (#767);--data-verifyand--verify-onlyexit non-zero on mismatches, missing keys, error replies or no verification replies (#771);__data__is sized by its command's key under--data-size-pattern=S,__scan_cursor__is sent as a constant, and__monitor_line@__placeholders are filled before sending, fixing a cluster hang (#759, #754, #660, #795).
Protocol parsing, libevent and crashes
- RESP parsing: lengths and counts the parser cannot represent are rejected, and RESP3 pushes, attributes and nested aggregates no longer read past their slot or corrupt the next reply; miss tracking counts RESP3 booleans, and a
SCANreply whose cursor is not a bulk string restarts the iteration. (#832, #846, #859, #882, #881, #854) - Multi-channel pub/sub:
SUBSCRIBE,PSUBSCRIBE,UNSUBSCRIBEandPUNSUBSCRIBEwith several channels complete at their last reply (one per channel) instead of aborting, and their byte and request counts change;UNSUBSCRIBEwith no arguments is still counted as one reply. (#875, fixes #515) - libevent: TLS error reporting is fixed on libevent 2.1 and 2.2 (a plaintext connection error on 2.2.2-alpha could corrupt memory), libevent 2.1.8's error callback from a failing connect is handled, and a failed bufferevent creation reports an error instead of aborting. (#822, #855, #823)
- Memcache text parsing: reply keywords are compared only within the line, fixing a one-byte over-read on every GET reply. (#857)
- Memory safety and crash reports: TCP keepalive and no-delay options are always applied (#541, also in 2.5.2), URI credentials and hostnames are owned correctly (#544, also in 2.5.2), the Unix socket listen queue is retried without blocking the event loop (#258),
READONLYframes are fully written (#820), the crash report header avoidslocaltime()and stdio (#755), and--versionand crash-report Git metadata refresh on rebuild (#778).
Build & Configuration Improvements
- libevent minimum 2.0.11: raised from 2.0.10 for the core, OpenSSL and pthread packages, because 2.0.10 returns monotonic uptime from
event_base_gettimeofday_cached()on Linux, which broke deadline-clamped reconnect timing. (#812) - Prometheus exporter needs libevent 2.1.4: the documented and configured minimum is corrected from 2.1.1 to 2.1.4 (
evhttp_request_set_on_complete_cb()first appeared in 2.1.4-alpha); use--disable-prometheusto build on older 2.1.x. (#823) - libtool no longer required, and the spurious
LT_INIT: command not foundmessage is gone. (#762) - OpenSSL 4.0: the build is fixed. Also in 2.5.2. (#548)
- Bash completion now installs under
--prefix(before, an absolute pkg-config path was used when bash-completion was installed, so non-root prefix installs andmake distcheckfailed); use--with-bash-completion-dir=DIRor--without-bash-completion-dir. (#765)
Packaging
- APT repository: Debian 13 (trixie) and Ubuntu 26.04 LTS (resolute) are in the APT release matrix, and Ubuntu armhf/arm64 package builds use a reachable ports mirror. Also in 2.5.1. (#512)
- Grafana: a Prometheus dashboard at
grafana/dashboards/prom-memtier.jsonfor live runs with--prometheus-port. Also in 2.5.1. (#510) - Vendored HdrHistogram_c is upgraded to 0.12.0 (unmodified upstream) with SHA-256 checksums verified by
make check. (#563)
Developer Tooling & CI
- Compatibility and sanitizers: libevent 2.1.12, 2.1.13 and 2.2.2-alpha compatibility CI and a Rocky Linux 8 job (libevent 2.1.8), unit tests in the sanitizer workflows, repaired coverage builds, sharded and more reliable CI, and a stricter test harness (#545, #855, #900, #853, #844, #781, #792, #814, #818, #899, #896, #841, #863). Legacy OpenSSL and Debian Bullseye builds and CI on
2.*branches were restored (#540, #550; also in 2.5.2). - Opt-in Claude PR review and issue triage workflows, hardened to remove the model's Bash tool grant (#520-#524, #533), and shared Claude/Codex agent instructions (#770).
- Docs and small items: Docker instructions repository name, a README typo,
MANAGER.md(#518, #815, #791); and cheaper per-second latency summaries (#564).
Thanks
New contributors since 2.5.2: @dnlzrgz (#518), @thweetkomputer (#519), @Daniel-Issen (#546), @darth-modder (#552), @sam-obeid (#363), @wanghenshui (#129) and @hamidzr (#815). Also thanks to @zumo64 (#510, 2.5.1) and @collinfunk (#548, 2.5.2).
Full Changelog: 2.5.2...2.6.0