v0.3.27

Seven pull requests and no milestone has closed, so this is a patch.

Most of it is M8 compatibility work: MONITOR, the DEBUG container, the five persistence and role commands, and a password. The one to read if you run more than one thread is the accept fix, which is a one line change to the server loop that decides which thread a connection lands on. A file written by 0.3.26 opens unchanged under this version and a file written by this version opens under 0.3.26. No record kind was added.

Fixed

• yodb serve --threads 16 frequently ran a whole benchmark on one thread. Every worker has the same listener in its own poller, and a worker that was woken looped on accept until the door said nothing was waiting. Clients do not arrive one at a time. A benchmark opens its two hundred and fifty six connections at once, every worker is idle at that moment, and whichever one won the wakeup drained the entire backlog into itself while the rest found nothing. A connection belongs to the thread that accepted it for as long as it is open, so the split was decided once, by a race, at the only moment when every thread was idle. A worker now takes one connection per ready event and goes back to the poller. The doors are level triggered, so a door with more waiting is ready again immediately and the next worker round the loop takes the next one. The cost is one extra wakeup per connection, paid once per connection rather than once per command. It was found in a cache-bench sweep on a 32 core box rather than by reading: GET throughput at eight and sixteen threads had a coefficient of variation between 0.40 and 0.77 where four of the six rival servers in the same cells on the same box sat between 0.00 and 0.02, and sixteen threads came out slower than eight. That makes those numbers measurements rather than good, and the re-measurement is still to come.
• COMMAND INFO was sending arrays where the reference sends sets, for the flags, the acl categories and the three empty fields on the end. It only shows on RESP3 and it showed on every command in the table. It turned up because the compare written for the DEBUG work looks at the wire bytes rather than at what a client decoded them into, which is the whole argument for writing compares that way.
• modularity panicked on a grouping whose labels ran above the node count. Any grouping of the nodes is meant to be accepted, since the labels mean nothing beyond which nodes share one, but the renumbering step sized its table by the node count and then indexed it by the label. Every caller inside the crate hands over labels that are already below the count, so nothing internal ever saw it, and a caller who numbered their groups by a hash or a database id hit it on the first call. Sizing by the largest label would let one of those callers ask for sixteen gigabytes, so the labels are ranked instead, and only when they need to be: the check is one pass, the fast path is what a level of Louvain runs twice a level, and the other path costs a sort of the distinct labels and a binary search a node.

Added

• MONITOR. Every command the server runs, echoed to whoever asked for it, in Redis's format byte for byte: the time to the microsecond, the database and the address in brackets, then the command and every argument quoted the way sdscatrepr quotes them. A simple string on RESP2 and RESP3 alike rather than a push, which is why redis-cli monitor works against either. The database reported is the one the connection is on after the command ran. A command that never reached its body is not on the feed and a command that failed inside its body is. The six script commands are reported before they run rather than after, because a script's own calls come back through the same place and a script whose effects arrived in front of the EVAL that caused them would be unreadable. Administrative commands are kept off it per subcommand rather than per container, off 8.10.1's own table, so CLIENT ID is on the feed and CLIENT LIST is not. The line is rendered once however many monitors are watching and handed to each monitor's thread through the mailbox the pub/sub path already uses, so the feed never writes into a reply buffer another thread owns. A server nobody is watching pays one relaxed load of a zero per command. A monitor may not touch the keyspace, which reads like an accident of Redis's implementation and is load bearing, since a monitor is exempt from CLIENT PAUSE and would otherwise have a way around its own pause that nothing else has. Seventy cases against 8.10.1 on the same machine, zero differences.
• The DEBUG container, eleven subcommands of it. DEBUG is the command a test suite talks to rather than a client, and Redis's own suite leans on it hard enough that a lot of it does not run at all against a server that has none. This is the half that is about this server: HELP, PROTOCOL, ERROR, LOG, SLEEP, POPULATE, SET-ACTIVE-EXPIRE, PAUSE-CRON, DICT-RESIZING, SET-SKIP-CHECKSUM-VALIDATION and QUICKLIST-PACKED-THRESHOLD. PROTOCOL is the one client libraries reach for, since it is the only way to make a server send a type on purpose, and all thirteen names answer the exact bytes 8.10.1 answers on both protocols, including the two nobody guesses: on RESP3 attrib is the attribute followed by a string and push is the string first and the push second, and on RESP2 push is the only one of the thirteen that comes back as an error. A knob that is remembered and read by nothing is worse than no knob, so three of the four gate something real: SET-ACTIVE-EXPIRE gates the background expiry sweep without making an expired key readable, PAUSE-CRON gates the whole maintenance slice the shard loop runs between batches, and DICT-RESIZING gates arena compaction, which is the nearest thing here to the dictionary resize a real server turns off, both being the background reclaim of room a table no longer needs. All four are read the reference's way, so anything unreadable means nought and a test that sends the word true to one of them has turned it off. 196 cases against 8.10.1 over both protocols, inside MULTI, through a monitor and with the notification channel open, 194 matching byte for byte, and the two that do not are one older gap seen once per protocol.
• SAVE, BGSAVE, BGREWRITEAOF, LASTSAVE and ROLE. The snapshot writer already produced a real RDB image for BACKUP, so the work was putting it behind the commands a client reaches for and making the file land safely: the image is written to a temp file, synced, and renamed over dump.rdb, so a reader never sees half a file. A real redis-server starts on the file yo writes and answers all fourteen probes identically across every type, and redis-check-rdb reads it and says it looks OK. BGSAVE writes the file before it answers rather than forking, so no save is ever in progress, and BGREWRITEAOF counts itself and writes nothing because there is no append only file. INFO persistence reports the twelve fields a monitoring rule reads and leaves out everything about a fork or a load, on the principle that a missing field is a client falling back rather than a client believing a zero.
• AUTH, requirepass and the NOAUTH gate. A server can be given a password with CONFIG SET requirepass or the new --requirepass flag on yodb serve, and a connection that has not authenticated is refused everything. There is one password and it belongs to a user called default, because requirepass on a real server is a thin layer over the ACL. Any other user name is refused with the same WRONGPASS a wrong password gets, since the difference between a name that does not exist and a password that was wrong is a list of user names. The rule worth reading twice is who starts out let in: the flag is decided when the connection is accepted rather than when it sends its first command, so setting a password does not lock out the clients already connected, including the one that just set it, and does lock out everything that connects after. That is the reference's behaviour and it is not what anybody would guess. The compare is written out byte by byte with the lengths folded in rather than returned on early, so how long a guess took does not say how much of it was right. A monitor is shown AUTH with every argument after the name replaced, the user name included, because a user name a client got wrong is very often a password typed one field up.

Changed

• The thirty tests that open a Lua state are skipped under Miri, which is what every one of the six yo-resp shards in the nightly deep run was failing on. A Lua state lives inside LuaJIT, which is C compiled by a build script, and Miri interprets Rust, so it stops at the first call across. There is nothing to shrink and nothing an interpreter could check even in principle, and the reason string on these says so rather than borrowing the wording from the counts that were made smaller for the same run, so a reader does not come away thinking this one could be shrunk too. The list came out of a run rather than out of reading names, which matters because one of them reads as though it stops short of Lua and runs a script at the end. Native runs are unchanged, and everything on this side of the boundary is still interpreted, including the five script libraries that are Rust here rather than the C a real server links.