perfcached 0.6.6

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

This page covers 0.6.3 to 0.6.6: 0.6.3, 0.6.4 and 0.6.5 were not published and have no pages of their own. Coming from 0.6.2, read all of it. The ChangeLog has the detail.

The first release published only after it was proven. 0.6.6 is the commit v0.6.6-rc3 (7d28c44), green on both CI systems and run for 72 hours on a three-node test fleet under staging traffic before this tag was put on it. Nothing was rebuilt for this page: the signed tarball below is the build that ran there. On that fleet it rolled in one node at a time over 0.6.4, which had rolled in over 0.6.2.

The dashboard leaves the daemon

• perfcached-dashboard serves the status page from its own process: a client of the fleet's native door through libperfd, one encrypted connection per member, so a dashboard under attack stalls no daemon. The page is the daemon's, byte for byte, under the same script-hash CSP.
• Its HTTP layer is libmicrohttpd. TLS and HTTP/2 belong to a reverse proxy in front; contrib/ carries HAProxy and nginx front ends for several dashboards, and trusted_proxy names the proxies whose X-Forwarded-For is believed.
• A POST needs a custom request header, a JSON body under body_max and the session token; each action asks for the fleet's enable secret once per browser page; wrong secrets lock the address out. Several dashboards sharing a session_key honour each other's session tokens.
• Its own package, with its own user and a disabled unit. The daemon's own page stays, deprecated, and goes in 0.7.0.
• Wrong enable secrets are counted per source address on the native door: after enable_lockout_after of them within enable_lockout_s (5 in 60 s by default) that address's enable is refused for the window.

Signed builds and the node's half of an upgrade (0.6.5)

• A release tarball comes with a detached ed25519 signature. perfcached -K <tarball> checks it against the release key built into the binary, or [upgrade] trusted_keys for builds you sign yourself.
• upgrade_stage receives a signed build over a privileged connection (perfcli upgrade-stage <tarball> <version>), verifies it, and passes it to every member that can take one; upgrade_status lists what is staged.
• An optional root helper, perfcached-upgrade, with its systemd units installed but not enabled, switches a node to a staged build and switches it back if the new one does not report ready. The Debian packages do not carry it.
• A node refuses to start on state a newer build wrote - a newer snapshot format, a WAL record of an unknown type, a newer cluster-id file - and says so, instead of reading it as damage.

Settings a fleet must agree on, compared (0.6.5)

• Each member's keepalive carries the settings the fleet must share: [wal] fsync, whether an HTTP token is set, the [alerts] thresholds, the tombstone limits, and each read-through collection's write settings. A member that differs from the most common value is named on every node as a config.<setting> alert.
• Every member reports its build; the status page shows each node's version and the header says "mixed versions" during a roll, where the cards of the members still on the older build blink amber.
• The tombstone table's limits are alerts: cluster.tombstones when deletes are dropped before tombstone_retain_s is up, cluster.peer_absent when a member returns after an absence longer than deletes are kept.

The restart reconcile has a fast path (0.6.4)

• A node back from a restart asks a peer that was up throughout for every delete and expiry made since its snapshot began; when the peer can vouch for that window, the pass ends in seconds instead of asking about every replayed key. Otherwise it asks key by key as before, and stats.cluster.reconcile_fast_why says why.
• An eagerly replicated key that expires is remembered with the retained deletes, so an older, longer-lived copy cannot come back. Size tombstone_retain_max for deletes plus expiries.
• Asking key by key is paced to the free pending-request slots, between 64 and reconcile_rate_max a second; stats, /metrics and the page show an open pass's progress.

Write-through to Redis, hardened (0.6.3 - 0.6.5)

• Under upstream_writes = through every node follows every change Redis makes under the collection's prefixes (broadcast tracking): a write another Redis client makes to a key written through here drops the node's copy, and the next read fetches Redis's value.
• A node's restart no longer empties the fleet's write-through cache, and a node no longer answers nil for a key Redis holds when an invalidation lands between a record and its no-fall mark.
• The write queue is bounded (upstream_write_queue, 10,000): past it a write is refused at once, and one with under a quarter of its timeout left is answered "not applied" without being sent.
• The writer has its own credential (upstream_write_user, upstream_write_password), sends plain writes pipelined, and the native door forwards add, sub, hash writes and mset under through.
• upfill fills the keys a collection owned before the switch to through, compares each with what Redis already holds, and says what Redis's values replaced - never a value, only lengths and expiries. Sweeps are paced by bytes as well as keys (upstream_drain_mb_rate), and the writer thread's CPU share is on the page.

libperfd 0.6.6

• The admin verbs, typed: perfd_stats, perfd_members and perfd_clients answer the trees the status page reads, perfd_enable raises privilege and raises it again by itself after a re-dial, and perfd_upfill / perfd_updrain run, check or stop a sweep.

Files

The signed tarball: perfcached-0.6.6.tar.gz with perfcached-0.6.6.tar.gz.sig - the exact build that soaked on the test fleet, x86_64, built on Debian 13 (glibc 2.41) with libcurl mail. It unpacks to one prefix directory (bin/, lib/, include/, share/) for /opt, and it is what upgrade_stage and the perfcached-upgrade helper take. Check it with the binary you already run (0.6.5 or later):

perfcached -K perfcached-0.6.6.tar.gz        # GOOD, the release key 5646e8e4 (built in)

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64:

• perfcached: the clustered daemon, every cluster mode.
• perfcached-standalone: the single-node edition, built from the same tree without the cluster code. A [cluster] section in its config is refused rather than ignored.
• perfcached-dashboard (new): the status page's own process, unit installed disabled. Not for Ubuntu 20.04: its libmicrohttpd (0.9.66) predates the API (0.9.71) the dashboard is written for. Run the dashboard on another machine for a 20.04 fleet - it reaches the members over the native door.
• libperfd0: the client library, libperfd.so.0.
• libperfd-dev: its header, static archive, libperfd.so link and libperfd.pc.

The two daemon packages install the same files and replace each other. They do not need libperfd0: install it, or libperfd-dev, for your own programs.

sudo apt install ./perfcached_0.6.6-1+deb13_amd64.deb
sudo apt install ./perfcached-dashboard_0.6.6-1+deb13_amd64.deb    # on the machine that shows the page
sudo apt install ./libperfd0_0.6.6-1+deb13_amd64.deb ./libperfd-dev_0.6.6-1+deb13_amd64.deb   # to build against libperfd

Each distribution's packages were built for it: Debian 13 natively (gcc 14.2, glibc 2.41), Debian 12 (gcc 12.2), Ubuntu 24.04 (gcc 13.3) and Ubuntu 20.04 (gcc 9.4, debhelper from focal-backports) each in a container of that distribution. Every build ran noisetest, the selftest and a deliberately broken-lock build that must fail. Each set was then checked in a clean container of its distribution:

• the version and revision perfcached -V reports (0.6.6 (7d28c44)), and the systemd unit;
• the daemon starting and answering ping, set and get over its encrypted native door from perfcli (on epoll: the containers' default seccomp profile refuses io_uring);
• a program built against libperfd-dev through pkg-config, linking libperfd.so.0 and reporting 0.6.6;
• the shipped config passing perfcached -C, a [cluster] section refused by the standalone edition, and the two editions replacing each other;
• the dashboard package: its binary's libraries, its unit file and the example config;
• an upgrade over the published 0.6.0 packages.

Alpine 3.24 (musl), x86_64, both editions, as tarballs of the tree's make install layout: binaries in usr/bin (the dashboard among them), libperfd (shared and static) with perfd.h and libperfd.pc, the example config, and the docs. No systemd unit or OpenRC service is shipped; run the daemon under your own supervisor.

apk add libsodium libcurl libmicrohttpd
tar -xzf perfcached-0.6.6-alpine3.24-x86_64.tar.gz
cp -a perfcached-0.6.6-alpine3.24-x86_64/usr perfcached-0.6.6-alpine3.24-x86_64/etc /

Built with gcc 15.2 in a clean Alpine container behind the same three gates, then checked in a second clean container: -V and the musl interpreter, the dashboard's libraries, the round trip, the libperfd program, the shipped config and the [cluster] refusal.

libperfd for Alpine: libperfd and libperfd-dev as .apk, signed with the perfcached release...

perfcached 0.6.2

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

This page covers 0.6.2 and 0.6.1, which has no page of its own. The ChangeLog has the detail.

No packages are attached to this release. Build it from the source archive below (README, Install: a C compiler, make and libsodium), or stay on the 0.6.0 packages.

A rolling restart from 0.4.5. One thing to do first, from 0.6.0 or earlier: a read-through collection's password moved from [secrets] upstream into the collection, as upstream_password. 0.6.1 and later refuse the old key, and perfcached -C says where it goes.

0.6.2

Write-through to Redis

• upstream_writes = through sends every client write on a read-through collection to the upstream's master first: the listed upstream whose ROLE says master, or, when two say so, the one the serving replicas follow. The written keys are read back in the same round trip, so the fleet then holds what Redis holds: a value with Redis's expiry, a deleted key as gone, and a list, set, sorted set or stream left to Redis. The client gets Redis's own reply.
• With no master reachable every write is refused (the critical alert write.<collection>) and reads go on. The master is looked for at start and after a failure, backing off to every 10 s; ROLE is asked every 10 s, and RedisJSON every 60 s while it is missing - never once per key.
• At the RESP door every write command is forwarded as sent; perfcached's EX on JSON.SET becomes an EXPIRE after it. EVAL, EVALSHA and RL.HIT write only this node and are refused. At the native door: set, setnx, setxx, del, expire, jset and jdel.
• JSON writes need RedisJSON on the master. Without it they are refused here, with no round trip, and a warning alert says so.
• Pub/sub is not bridged: Redis's channels and the fleet's stay separate.

Commands perfcached does not know

• Under through, upstream_unknown = pass (the default) lets Redis classify a command perfcached does not implement (COMMAND INFO, COMMAND GETKEYS): a write goes to the master with its keys read back, a read to a serving replica. The reply is relayed byte for byte, up to upstream_pass_max_kb (1024); a larger one is refused, never cut.
• Refused whatever Redis says: admin, dangerous, blocking, pub/sub, transaction and script commands, another database, writes that name no key, and keys outside upstream_prefixes. refuse keeps "unknown command".

Stats and the status page

• /stats upstream.write_through has the target, its state, RedisJSON, and the writes by outcome and by command as sent to Redis, with the last eight errors in Redis's own words and the slowest write in five minutes. The page shows them under the collection, grouped with its upstreams.
• A command that waited for an upstream (a read-through miss, a write-through write) is counted once in the commands table; it was counted twice.
• The slow log and the alerts fold like the commands table, and the commands table sorts by calls or by its slowest call. Every card holds its content at desktop widths.

The HTTP door

• A peer refused by http_allow can no longer stall a worker. A silent refused connection held a worker for 200 ms, and one sending a byte every 150 ms held it indefinitely; the 403 is now sent and the socket drained without waiting. Every release from 0.3.0 has this, so update a node whose HTTP port is reachable from outside its allow-list.
• http_rate (20 requests a second per client address, loopback exempt) answers a faster address 429 Too Many Requests. http_cache_ms (1000) builds /stats and /metrics at most once in that long, however many pages are open.
• A browser can open a page behind [secrets] http: the 401 asks for its login, and Basic with the token as the password (any user name) is accepted beside Bearer.

0.6.1

Alert mail a person can read

• The subject is <STATE>: <title> on <host> - CRITICAL, WARNING, INFO or RESOLVED - in at most 78 characters, and From names the host. Each mail has a plain and an HTML part: per alert what happened, its status, when it started, what the node does by itself and what to do. A resolve answers the problem's first mail.
• Levels say how soon to act: a clean start is info (not mailed at the default level), an unclean start and a WAL loss are warnings, a third loss within an hour and a heal stuck for ten minutes are critical.

Several Redis upstreams for one collection

• Up to eight. upstream_role = replica reads only from servers whose ROLE says replica (upstream_master_fallback = yes lets the master serve while none does). upstream_mode = balance sends each fetch to the one with the fewest in flight; failover keeps one active, the next taking over when it stops and the primary returning after upstream_preempt_hold_s. Each upstream has its own tracking.
• The upstream password belongs to the collection, upstream_password, so one perfcached can front several Redis servers.
• No ring of read-throughs: an upstream that is itself a perfcached is refused unless upstream_allow_perfcached = yes, and then a key crosses one perfcached hop at most.

I/O

• The page has an io card: the backend running (io_uring or epoll), a fallback in amber with its reason, and the busiest worker's share of the last five minutes.
• A node on epoll whose busiest worker stays over [alerts] epoll_busy_pct (80) for five minutes is warned, with the change to make.
• A pub/sub subscriber that keeps up is no longer closed as slow when its worker drains a large backlog of big messages on io_uring: a drain stops at 1 MB as well as at 1,024 messages.

perfcached 0.6.0

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

This page covers 0.6.0. 0.5.7.1, 0.5.8 and 0.5.9 have pages of their own without packages: coming from 0.5.6.6, the last release with packages, read those too. The ChangeLog has the detail.

A rolling restart. From 0.4.5, 0.6.0 joins a fleet one node at a time; use SET IFEQ/IFNE and DELEX once every node runs 0.5.0 or later. A config that uses [mail] (0.5.9) is refused by an older release: to roll back, put the old config back with the old binary.

io_uring by default

• The workers receive into provided buffers, with one multishot receive per connection, and send their replies with the turn's single io_uring_enter(): no read() and no write() a request.
• At start the daemon tries io_uring for real and falls back to epoll where the kernel (before 6.0), kernel.io_uring_disabled or a container's seccomp profile refuses it. The reason is logged and shown in /stats (io.backend), and perfcached -C says so beforehand. [daemon] io = epoll forces epoll.
• On 16-vCPU VMs with multiqueue NICs, against epoll: 41% more RESP and 63% more native requests a second one at a time, 12% more pipelined, and a third less CPU a request.
• On epoll, a short read ends a wakeup's reads: one read() a request instead of two.
• PRODUCTION.md section 7: one NIC queue per CPU on the nodes and the clients, irqbalance, and the daemon unpinned.

A fast writer with a slow reader is throttled, not closed

• At 4 MB of replies unread, a client's further requests wait and its socket is not read, until the backlog drains below 1 MB. One that reads nothing for output_stall_s (default 30) meanwhile is closed as a slow consumer.
• The log says throttling client <ip:port> the first time, and the disconnect line counts the throttles. /stats (clients.output) and /metrics count them. A slow consumer's close now says so; it used to read "peer closed".

Busy polling, opt-in

• busy_poll_us and busy_poll_prefer, per worker (kernel 6.9+), off by default. Its effect on perfcached is not measured yet: measure it on your own NIC before you keep it.

Stats

• /stats io and /metrics count the system calls against the work they moved: waits, reads and writes, io_uring enters, completions, receives and sends, and UDP datagrams a call. perfcached_io_worker_ops_total has each worker's I/O.

Packages

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64:

• perfcached: the clustered daemon, every cluster mode.
• perfcached-standalone: the single-node edition, built from the same tree without the cluster code. A [cluster] section in its config is refused rather than ignored.
• libperfd0: the client library, libperfd.so.0.
• libperfd-dev: its header, static archive, libperfd.so link and libperfd.pc. A program built against it links the shared library by default.

The two daemon packages install the same files and replace each other. They do not need libperfd0: install it, or libperfd-dev, for your own programs.

sudo apt install ./perfcached_0.6.0-1+deb13_amd64.deb
sudo apt install ./libperfd0_0.6.0-1+deb13_amd64.deb ./libperfd-dev_0.6.0-1+deb13_amd64.deb   # to build against libperfd

Each distribution's packages were built on that distribution: Debian 13 (gcc 14.2), Debian 12 (gcc 12.2, in a container), Ubuntu 20.04 (gcc 9.4) and Ubuntu 24.04 (gcc 13.3). Every build ran noisetest, the selftest and a deliberately broken-lock build that must fail. Each set was then checked in a clean container of its distribution:

• the version and revision perfcached -V reports, and the systemd unit;
• the daemon starting and answering ping, set and get over its encrypted native door from perfcli, on epoll under the container's default seccomp profile and on io_uring without it;
• a program built against libperfd-dev through pkg-config, linking libperfd.so.0;
• the shipped config passing perfcached -C, a [cluster] section refused by the standalone edition, and the two editions replacing each other;
• an upgrade over the published 0.5.6.6 perfcached package.

Alpine 3.24 (musl), x86_64, both editions, as tarballs of the tree's make install layout: binaries in usr/bin, libperfd (shared and static) with perfd.h and libperfd.pc, the example config, and the docs. No systemd unit or OpenRC service is shipped; run the daemon under your own supervisor.

apk add libsodium libcurl
tar -xzf perfcached-0.6.0-alpine3.24-x86_64.tar.gz
cp -a perfcached-0.6.0-alpine3.24-x86_64/usr perfcached-0.6.0-alpine3.24-x86_64/etc /

Built with gcc 15.2 in a clean Alpine container behind the same three gates, then checked in a second clean container: -V and the musl interpreter, the round trip on epoll and on io_uring, the libperfd program, the shipped config and the [cluster] refusal.

libperfd for Alpine: libperfd and libperfd-dev as .apk, signed with the perfcached release key. Trust the key once (its public half is attached here and in contrib/alpine/keys), then install:

cp yury@kfi.net.au-6aca2bb6.rsa.pub /etc/apk/keys/
apk add ./libperfd-0.6.0-r0.apk ./libperfd-dev-0.6.0-r0.apk

Checked in a clean container that trusts only that key: both install, pkg-config reports 0.6.0, a program links the shared or the static library and pings a 0.6.0 daemon, and without the key apk refuses them.

SHA256SUMS covers every file on this page.

Known

• In a container under Docker's or podman's default seccomp profile, io_uring is refused and the daemon runs on epoll. perfcached -C and the log say which.
• Busy polling's effect is not measured yet.
• No lists, sets, sorted sets or streams yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment. spread mode has not yet carried production traffic.
• In eager mode a delete made on one side of a network partition can be undone when the partition heals.

perfcached 0.5.9

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

Superseded by 0.6.0. This page was added afterwards for the record: no packages are attached - install the latest release. The source archives below are this tag's tree.

This page covers 0.5.9. The ChangeLog has the detail.

A rolling restart. From 0.4.5, 0.5.9 joins a fleet one node at a time; use SET IFEQ/IFNE and DELEX once every node runs 0.5.0 or later. A config that uses [mail] is refused by an older release: to roll back, put the old config back with the old binary.

A node raises its own alerts, and mails them

• Failed or healing, records the WAL lost, a slow fsync, writes refused for want of memory, an arena near its ceiling, a read-through upstream unreachable or its tracking off, its own start (critical after an unclean stop). Each is logged, kept - the last 100 in /stats alerts and on the status page's alerts card - and counted on /metrics.
• With [mail] and a build with an SMTP client (PC_MAIL=libcurl; the release packages have one), alerts are mailed: STARTTLS or TLS, AUTH with the password in [secrets] mail, the same alert at most once per min_interval_s unless it got worse, everything within digest_s in one mail, and the clear when it recovers. perfcli -E ... mailtest sends a test mail and prints the server's answer. tls_verify = no keeps TLS for a relay whose certificate cannot be verified, with a warning at every config check.
• In a fleet, [mail] on one node is enough: the nodes elect the lowest-numbered one that can mail and relay their alerts to it, and what happens to the fleet itself - a member silent, failed, gone or missing, the fleet below a majority - is mailed once, by that node.

libperfd 0.6.0

• perfd_get_buf, perfd_jget_buf and perfd_hget_buf read into the caller's buffer and allocate nothing. The SONAME stays libperfd.so.0.

Build flavors

• make menuconfig chooses release, debug or asan (sanitizers linked statically); a non-release build says so in -V, /stats and the page header. contrib/systemd/cores.conf keeps a crashed node's core; asan.conf routes a sanitizer's report beside it.
• The perfcached packages depend on libcurl (the alert mail).

perfcached 0.5.8

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

Superseded by 0.6.0. This page was added afterwards for the record: no packages are attached - install the latest release. The source archives below are this tag's tree.

This page covers 0.5.8. The ChangeLog has the detail.

A rolling restart. From 0.4.5, 0.5.8 joins a fleet one node at a time; use SET IFEQ/IFNE and DELEX once every node runs 0.5.0 or later.

libperfd 0.5.0: a shared library

• libperfd.so.0 beside the static archive: an update with the same SONAME replaces it in place. The ABI carries what that needs - perfd_opts starts with its size (PERFD_OPTS_INIT, perfd_opts_init()), errors carry a kind (perfd_error_kind()), values are released with perfd_free_value(), the library logs through perfd_set_log(), and only perfd_* is exported. A pkg-config file, libperfd.pc; make install-lib.
• The archive is position-independent, so it links into shared objects such as plugins and modules.
• test/abitest.sh holds the library to its SONAME: a caller built against the frozen 0.5.0 header runs against the new library, and no exported symbol disappears.

Packages and the licence boundary

• libperfd0 and libperfd-dev .deb; libperfd and libperfd-dev .apk for Alpine, signed with the release key.
• libperfd's archive no longer carries a GPL object from the daemon: its MIT boundary is now the set of files the compiler reads to build it, checked on every push.

Asterisk

• func_perfd links the installed libperfd (found through pkg-config, or --with-perfd=DIR) instead of compiling a copy: the patch goes from 22 files to 8, and the module compiles under developer mode.

perfcached 0.5.7.1

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

Superseded by 0.6.0. This page was added afterwards for the record: no packages are attached - install the latest release. The source archives below are this tag's tree.

This page covers everything since 0.5.6.6: 0.5.7 and 0.5.7.1, which replaced it the same day. The ChangeLog has both.

A rolling restart. From 0.4.5, 0.5.7.1 joins a fleet one node at a time; use SET IFEQ/IFNE and DELEX once every node runs 0.5.0 or later.

A rolling restart waits for the witness a reconcile needs

• A node does not call itself safe to restart while it is the last live witness of a peer's reconcile: stats.cluster.restart.waits_for names the peers. (0.5.7 waited while any peer reconciled; 0.5.7.1 waits only where its restart would leave the reconcile without a witness.)
• A reconcile left with no witness at all - every peer restarted after the node went down - says so once, after 30 seconds, with the remedy, and counts the time in stats.cluster.reconcile_no_witness_s. A rolling restart that went on regardless used to leave the first node's reconcile waiting for good.
• The status page's header says when a reconcile is in trouble: red "reconcile has no witness - restart this node", amber "do not restart: node N needs it to finish its reconcile". /members carries each member's reconciling and sole_witness.

The memory guard and huge pages

• The guard no longer counts a huge-page arena twice. On a 2 GB node with arena_mb = 1024 on a 512-page pool it refused every new client once the arena had given memory back. Boot says when a pool is outside the limit; /stats clients.memory.hugetlb_pool.

Read-through probes without a fetch

• EXISTS, TTL, PTTL and TYPE of a key the collection does not hold answer from one round trip of TYPE and PTTL, and a hash or JSON document is remembered as present for the next ones - they used to fetch and store the whole document. /stats upstream.fill_present, present_hits.

Connect storms

• A worker turn runs at most 8 Noise handshakes and parks the rest, and accepts at most 64 connections: at ~6,000 encrypted connects a second, an established client's p99 went from 92-107 ms to 33-37 ms. /stats clients.handshakes.

perfcached 0.5.6.6

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

This page covers everything since 0.5.5: 0.5.6 and its point releases up to 0.5.6.6. The ChangeLog has each one.

A rolling restart. 0.5.6.6 joins a 0.4.5 or later fleet one node at a time: restart the members first and the master last, and read the fleet between nodes. The storage formats and the wire epoch are unchanged, and clients built on libperfd 0.4.0 keep working. Two things to know before you start:

• Check the memory guard on huge-page nodes first - see Known below. A node whose arena lives in a reserved hugetlb pool may need memory_floor = off.
• A config that uses a new key is refused by an older release (memory_floor, stats_reset, upstream_writes). To roll back, put the old config back with the old binary.

The daemon protects itself from the out-of-memory killer

On a 1 GB node, 50,000 idle RESP connections used to get the daemon killed. Now:

• A memory guard reads, once a second, the memory the node may still use: MemAvailable, or the headroom of a memory-limited cgroup (v2 or v1), whichever is tighter, less the arena's uncommitted part. Below the floor, new clients are refused exactly as at max_clients. Open connections and /stats are untouched, and refusing stops once a further 5% of the limit is free.

  [daemon]
  memory_floor = 15%      # the default; kept within 128 MB .. 4 GB.  Or: 512 MB | 2 GB | off
  

  A 300 MB cgroup now holds 24,457 of 25,000 connections and refuses the rest.
• max_clients, when unset, counts memory as well as descriptors: the smaller of the two wins. The boot line and /stats clients.max_basis say which. A configured max_clients above what memory holds stands, with a warning; the guard protects.
• An idle connection holds no buffers. After two quiet seconds its input and output buffers are released (14 KB down to 2.2 KB per idle client), and they grow back on its next request. Once a burst of closed or idle connections settles, the freed heap is handed back to the system, so the daemon's memory falls back after a mass disconnect.
• A busy client is no longer slowed by idle ones. Every request used to pay a walk over all of its worker's connections (with 20,000 idle: 60 -> 750 us a PING). Now it does not.
• Encrypted native clients past the limit are refused before the Noise handshake, with the RESP door's -ERR max number of clients reached. A storm of refused clients no longer costs a handshake each: 12,000 clients against a limit of 2,000 take a third less daemon CPU. A library that does not read the line sees its handshake fail, as on any close. A plaintext native door (loopback, unix socket) still answers an error frame to the first request.
• New counters: /stats clients.memory and clients.buffers. New /metrics: perfcached_memory_limit_bytes, _available_bytes, _floor_bytes, perfcached_memory_refusing, perfcached_clients_refused_memory_total, perfcached_conn_buffers_released_total, perfcached_heap_trims_total. The status page shows a red "memory floor" chip while the node is refusing.

libperfd 0.4.2

• 0.4.1: a blocking connect takes the key from the process-wide cache. It used to derive it on every connect: 150 ms and 64 MB each. The derive is single-flight across threads, and perfd_psk_prepare() derives before fork() so a forking server's children inherit it. Rebuild clients that connect often.
• 0.4.2: fits a musl thread's 128 KB default stack. A connect from a thread on Alpine overflowed it.

Fixes

• A crash on the metrics door (0.5.6 - 0.5.6.2): a connection that reached http_timeout in the very moment its request or close arrived was freed and then handed its event, a use-after-free ending in SIGSEGV. Seen under a monitor plus a Prometheus scraper.
• WAL, standalone edition (every standalone release since 0.4.0): a key that expired, was reaped and was written again on another thread could come back after a crash with its expiry instead of the new value. The clustered edition is not affected.
• WAL: an empty node restarted more times than its WAL has segments came back FAILED ("wal: FULL at startup").
• "perfcached ready" is logged once every worker's listeners are open; a client that connected the moment it appeared could be refused.
• A record moved back from the overflow chain expires on time.
• A native client with no secret is told so at once, instead of being taken for a long handshake.
• The per-door connection and request totals are counted to 64 bits.

Read-through

• Tracking stays on one server. Behind a round-robin proxy over several replicas, a fetching connection could land on another server than the tracking one, and its copies would never see a change. Each connection now reads its server's run_id. One that reached another server is not tracked; its copies live at most upstream_negative_ms, and stats counts them (upstream.tracking_other_server). The upstream user needs +info; the GUIDE's ACL line has it.
• upstream_writes = refuse: a read-through collection clients can only read. Every write to it, on every door, is refused with one message, and so are a drain sweep and a restore into it.
• Read-through works on the RESP door under a mapped db index (resp_collections mapping SELECT n onto a named collection).
• It counts what clients got: a read answered with a value is a hit, an absent answer a miss. The status page adds a line per read-through collection: fetched from the upstream (found / absent, average fetch time), answered from fetched copies, and answered absent from the negative cache.
• The drain sweep reads untracked, so the upstream no longer remembers every swept key.
• The <name>.nofall companion follows its parent's autoscale.
• tools/redis-replica-interop.sh runs read-through against a real Redis replica in throwaway containers.

Stats and the status page

• Every count on the status page is the last five minutes, kept by the daemon (win_<field> in /stats), so the window is true from the moment the page opens. Gauges stay current; maxima are the last five minutes'.
• /stats resets on a schedule: daily at local midnight by default (stats_reset = off | 1d .. 7d, stats_reset_at = HH:MM). The page's "reset stats" resets the whole fleet at one moment. /metrics is never reset, so Prometheus sees true counters across resets and collection resizes.
• perfcached_command_latency_seconds has one bucket per octave (3 us .. 262 ms): 18 rows per command instead of 129. Every bound is an edge of the daemon's own histogram, so the counts are exact.
• The header names hosts in trouble: it blinks red (failed) or amber (any other state but ready) and counts the fleet ("fleet 3/3 ready").
• One rule for collection names wherever one is made: 1-32 characters of A-Z a-z 0-9 _ -, not starting with -. A name from before the rule is kept and named in the log.

Known

• The memory guard counts a hugetlb arena twice. With the arena on MAP_HUGETLB huge pages, the pool is already outside MemAvailable, but the guard also subtracts the arena's uncommitted part. A node whose arena is large against its RAM can refuse every new client once the arena has handed memory back. For example: 2 GB of RAM, arena_mb = 1024 on a 512-page hugetlb pool. At a fresh start the arena commits the whole pool, so it shows up later, not at boot.
  • Check the boot line arena: ... reserved on MAP_HUGETLB huge pages and /stats clients.memory.budget.
  • On such nodes set memory_floor = off (the 0.5.6.4 behaviour) until the fix.
  • THP and 4K arenas are counted correctly.
• Read-through: KEYS and SCAN see only what the collection holds. proxy, shard and spread collections refuse an upstream. In a cluster, a read on another node in the instant before a change reaches it may answer the upstream's older value.
• No sorted sets, lists or streams yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment. spread mode has not yet carried production traffic.
• In eager mode a delete made on one side of a network partition can be undone when the partition heals.

Packages

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64, in both editions:

• perfcached: the clustered daemon, every cluster mode.
• perfcached-standalone: the single-node edition, built from the same tree without the cluster code. A [cluster] section in its config is refused rather than ignored.

The two install the same files under the same names and replace each other, so moving between editions is one apt install. Each package was built in a clean container and install-checked in a second one: the version and revision, the edition perfcached -V reports, libperfd 0.4.2, the shipped config passing perfcached -C (also with memory_floor set), and the systemd unit. For the standalone package it also checked that a [cluster] section is refused and that installing perfcached over it swaps the edition.

Alpine 3.24 (musl), x86_64, both editions, as tarball...

perfcached 0.5.5

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

A rolling restart. 0.5.5 joins a 0.4.5 or later fleet one node at a time: restart the members first and the master last, and read the fleet between nodes. The storage formats, the wire epoch and libperfd (0.4.0) are unchanged. Read-through is off unless a collection names an upstream, so a fleet that does not use it runs exactly as before.

Read-through: take a keyspace over from Redis

A collection can name an UPSTREAM - Redis, Valkey or another perfcached - that a node reads on a local miss and never writes. Move the clients to perfcached and it answers from its own collection; a key it does not hold yet is read from the upstream the first time anyone asks. No big-bang copy, and no way back into Redis. The GUIDE has the whole story; in short:

[secrets]
upstream = <the password perfcached sends to the upstream>

[collection 0]
upstream = redis://reader@10.0.0.12:6379/0
upstream_prefixes = tok:*, cid:*

• One way only. perfcached sends the upstream the handshake and reads - TYPE, PTTL, GET, HGETALL, JSON.GET, and SCAN for the drain sweep - and nothing else. Point it at a replica, as a user that can only read; the GUIDE has the ACL line.
• Only the keys you list fall through: upstream_prefixes is required (Redis glob patterns, * for all of them).
• Strings, hashes and RedisJSON documents are carried, with the upstream's TTL. A list, set, sorted set or stream answers WRONGTYPE and is not stored.
• A read waits for one fetch and answers from a node-local copy; concurrent misses for one key cause one fetch. A change - INCR, HSET, JSON.* path writes, DEL, EXPIRE and the rest, on either door - first takes the upstream's value over as a real write (logged, replicated), then runs: INCR of an upstream 41 answers 42. A plain overwrite (SET) does not wait.
• A key the fleet has written or deleted never reads through again. perfcached cannot delete the upstream's copy, so a companion collection, <name>.nofall, created and replicated with the collection, remembers it for as long as the upstream's copy may live.
• The upstream's own changes reach perfcached through Redis client tracking (RESP3, Redis 6 or later): a key the upstream changes, deletes or expires drops out of the fetched copies at once. Losing a tracking connection drops every fetched copy.
• When the upstream fails, a read answers an error (or a miss, with upstream_on_error = miss); a change always answers the error - it never runs on a guess.
• The drain sweep finishes the move: perfcli -E <enable> updrain <collection> scans the upstream and takes over every key the fleet does not already hold, has not written and has not deleted, at most upstream_drain_rate keys a second (5000). Run it on one node; the writes replicate. Then remove upstream and upstream_prefixes on every node and restart them.
• stats carries an upstream block per collection - fetches and fills by type, promotions, invalidations, errors, timeouts, fetch time, and the drain sweep's progress.

Tested against a scripted upstream that fails the run on any write - on one node and in a three-node eager fleet - and against redis 8.10.2 as a read-only ACL user, where MONITOR shows perfcached sending only the handshake and reads and ACL LOG stays empty (tools/redis-interop.sh, a throwaway container).

Known

• Read-through: KEYS and SCAN see only what the collection holds; proxy, shard and spread collections refuse an upstream; and in a cluster, a read on another node in the instant before a change reaches it may answer the upstream's older value - the same window in which an eager fleet without an upstream would answer a miss.
• No sorted sets, lists or streams yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment. spread mode has not yet carried production traffic.
• In eager mode a delete made on one side of a network partition can be undone when the partition heals.

Packages

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64, in both editions:

• perfcached - the clustered daemon, every cluster mode.
• perfcached-standalone - the single-node edition, built from the same tree without the cluster code; a [cluster] section in its config is refused rather than ignored. Read-through works in both.

The two install the same files under the same names and replace each other, so moving between editions is one apt install. Each package was built in a clean container and install-checked in a second one: the version and revision, the edition perfcached -V reports, the shipped config passing perfcached -C, the systemd unit, and for the standalone package that a [cluster] section is refused and that installing perfcached over it swaps the edition.

perfcached 0.5.1

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

A rolling restart. 0.5.1 joins a 0.4.5, 0.4.6 or 0.5.0 fleet one node at a time: restart the members first and the master last, and read the fleet between nodes. The storage formats, the wire epoch and libperfd (0.4.0) are unchanged; an older node ignores the two new lane messages. Every node keeps the identity it has - see below for switching to one derived from the machine.

A node's identity comes from the machine

• With a state directory and the default identity_source = auto, the identity is derived from the platform - the VM's DMI product uuid, or inside a container (where that uuid is the host's) a NIC's MAC - keyed by a salt the node mints once into its state directory (node-salt). A disk cloned onto another VM comes up as a new member rather than a twin; two daemons on one machine are two members; a state directory moved to another path keeps its identity.
• An identity already on disk with no node-salt beside it - one an earlier release minted at random, or derived under a site_salt since removed - is kept, with a WARNING naming the source the machine offers. To switch, stop the node and delete node-identity (or set identity_source = dmi / mac); it then joins as a new member, so on a fleet holding data do it one node at a time.
• product_uuid is readable only by root and the daemon runs as perfcached: the shipped clustered unit copies it into the service's runtime directory first (RuntimeDirectory=perfcached and a root ExecStartPre=-+ step). A unit written for an earlier release needs those two lines for the DMI source.

autoscale = floor

• A fourth mode beside auto, warn and off: the collection grows by key count as auto does, and shrinks only down to the size it was given - its buckets_log2, a create's size, or an explicit resize. An empty collection sized for the keys it expects keeps that size instead of being shrunk a minute after the start. /stats carries autoscale_floor_log2 for every collection.

Cluster

• A departing master's goodbye is no longer taken for a UDP block. Its members measured its silence from the machine's boot and, while its TCP lane still answered during its shutdown snapshot, moved all their traffic to it onto TCP - for good.
• A forced lane now comes back: the node asks the peer by UDP every [cluster] lane_retry_s (default 30) and returns to UDP once the answer proves both directions (/stats cluster.lane.udp_restored). A peer that leaves takes its forced lane with it.

Store

• A SCAN, a snapshot or a resize copy that walked a table's overflow leg while the leg drained back into the buckets could miss a record: the walk found its place again by position after records ahead of it had left. It now resumes after the last record it handed over that is still in the chain - a record may be handed over twice, never skipped.

Known

• No sorted sets, lists or streams yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment. spread mode has not yet carried production traffic.
• In eager mode a delete made on one side of a network partition can be undone when the partition heals.

Packages

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64, in both editions:

• perfcached - the clustered daemon, every cluster mode.
• perfcached-standalone - the single-node edition, built from the same tree without the cluster code; a [cluster] section in its config is refused rather than ignored.

The two install the same files under the same names and replace each other, so moving between editions is one apt install. Each package was built in a clean container and install-checked in a second one: the version and revision, the edition perfcached -V reports, the shipped config passing perfcached -C, the systemd unit (the clustered one with the runtime directory and the DMI copy step), and for the standalone package that a [cluster] section is refused and that installing perfcached over it swaps the edition.

perfcached 0.5.0

A multithreaded, self-clustering cache for Linux, with its own encrypted protocols and a Redis-compatible door. New here? Start with the README.

A rolling restart. 0.5.0 joins a 0.4.5 or 0.4.6 fleet one node at a time: restart the members first and the master last, and read the fleet between nodes. The storage formats and the wire epoch are unchanged, and libperfd 0.3.0 clients keep working. Use SET ... IFEQ/IFNE and DELEX once every node runs 0.5.0; the Symfony lock scripts are refused until it does.

Alpine Linux

• Alpine (musl) is a supported runtime. On Alpine 3.24 (musl 1.2.6, its own malloc) the daemon serves 0.78-0.95 of the glibc build's rate where the server is the bottleneck, and is ahead of redis:8-alpine on every measured row - INCR, SET/GET of 256 B and 1 KB values, a 7:1 mix, JSON.SET and JSON.GET, at 4 and 12 workers - by 1.5x to 14.9x.
• bench/Containerfile.alpine builds a 55 MB image. CI builds both editions on Alpine and runs the standalone suites there.
• The allocator is a build choice: make menuconfig, or PC_ALLOC = libc | mimalloc | jemalloc. libc stays the default everywhere (the packages below use it); perfcached -V names a non-default allocator, and /stats carries process.allocator.

No heap per request

• The JSON path allocates nothing after warm-up: JSON.SET, GET, NUMINCRBY, ARRAPPEND, DEL and the hash forwards made three to four allocations each. On musl one of them was a 256 KB buffer mapped and unmapped on every command, which held JSON.SET at 25-30k a second; it now runs at 1.3M at 4 workers.
• libperfd 0.4.0 allocates nothing per call on a warm handle. The API and the wire are 0.3.0's, so a rebuild is all an existing client needs; perfd_get and perfd_jget still return a malloc'd result, as documented.

Compare-and-set, and Symfony Lock

• SET ... IFEQ <value> | IFNE <value> and DELEX <key> [IFEQ|IFNE <value>], in Redis 8.4's spelling and with its replies. IFDEQ/IFDNE are refused by name: perfcached keeps no value digest. Native methods setifeq, setifne, delifeq, delifne, and in libperfd perfd_set_if and perfd_del_if.
• EVAL, EVALSHA and SCRIPT LOAD | EXISTS | FLUSH for the lock scripts of Symfony's Lock component (5.4 to 7.4): they are recognised by SHA1 and run natively, as one atomic step on the key's holder. Any other script is refused.

Proxy collections

• A request may ask where its key lives (an opt-in flag), and the reply names the node that holds it. libperfd's route_hints option remembers that per key and sends the next request for the key straight there; a stale entry costs one extra hop, never a wrong answer. Off by default.
• A GET, DEL or EXPIRE that a stale locator steered to a node no longer holding the key now asks the fleet once. Before, the GET answered nil, the DEL answered 0 and left the key alive on its new holder, and the EXPIRE left it without a TTL.

Faster

• A SET that rewrites an unchanged value compares the two a word at a time and prefetches the stored one (on musl, SET of 256 B +7%, of 1 KB +15%).
• The reply writer's small appends and its numbers no longer go through memcpy: on musl a GET hit made seven memcpy calls, now three.

Known

• No sorted sets, lists or streams yet. Keys hold strings, counters, JSON documents or hashes (a hash is one value, under 58,000 bytes).
• discovery = unicast (clouds, Kubernetes) is tested in lab network namespaces that drop IP fragments and block UDP, not yet in a real cloud deployment. spread mode has not yet carried production traffic.
• In eager mode a delete made on one side of a network partition can be undone when the partition heals.

Packages

Debian 12 and 13, Ubuntu 20.04 and 24.04, amd64, in both editions:

• perfcached - the clustered daemon, every cluster mode.
• perfcached-standalone - the single-node edition, built from the same tree without the cluster code; a [cluster] section in its config is refused rather than ignored.

The two install the same files under the same names and replace each other, so moving between editions is one apt install. Each package was built in a clean container and install-checked in a second one: the version and revision, the edition perfcached -V reports, the shipped config passing perfcached -C, the systemd unit, and for the standalone package that a [cluster] section is refused and that installing perfcached over it swaps the edition.