v8.5.37 — a credential that follows its name in a route

v8.5.37 - A credential that follows its name in a route

removeFromPath() saw only the SHAPE rule: a segment that looks like a token.
It missed the other half, which a client's UrlScrubber had - a credential named
by the segment in front of it. /token/abc, /api_key/xyz, /reset-password/short:
the value need not look like anything, because the route already said what it
is.

Promoted, and narrowed on the way. The client matched the naming segment
against BOTH its lists, so a path segment called code, key, hash, pin, otp or
sig redacted whatever followed - and /code/at lost its country. Those names
are query-only here for exactly that reason, so this reads the substring list
alone: token, password, secret, jwt, bearer, signature, api_key. A route
segment carrying one of those is naming a credential; the others are ordinary
words in a path.

Pinned in the shared corpus, which grew a paths group carrying both halves -
what the rule catches and what must survive it, /de/pre-und-onboarding/
included.

v8.5.36 — sess names no bug, so it comes back out

sess names no bug, so it comes back out

v8.5.35 added sess to the query list because a client redacted it. That was
checked afterwards, which is the wrong order round, and the check came back
empty:

• the client and other CMS sites set session.use_trans_sid = 0; the id rides
  a cookie (session.name = PHPSESSID)
• no ?sess= anywhere across the estate
• the only occurrence in the estate was one client's own denylist, which looks
  copied from a generic list rather than derived from its code

A shared security list should carry rules it can point at a bug for. This one
could not — and addQueryNames(['sess']) is exactly the extension point for an
application that ever does emit one.

What the corpus pins now

The decision, not the rule: ?sess=s-9 survives a scrub, with the reason
beside it, so the next person to consider adding it finds the answer rather
than the question.

Unaffected

Everything else from v8.5.34/35 stands: jwt, bearer and signature on the
substring list, otp, pin and hash query-only. A plain Logger still
covers every other name on the client's two lists, with auth deliberately
query-only (as a substring it eats author) and addresses masked by value with
the domain kept.

v8.5.35 — sess and signature, so a plain Logger covers westbahn outright

sess and signature — a plain Logger now covers a client outright

Two more names off a client's OTEL denylists, and a correction: the first pass
left sess out for a reason that did not hold.

The correction

sess was rejected on the grounds that it would redact session_id, which the
console stores as a first-class column and correlates by. It would not.
queryNames is matched against a query parameter name only, inside
removeFromUrl() — never against a field in a bag. So ?sess=abc is redacted
and the column is untouched.

signature moves from the exact query list to the substring list, where
the client had it: no ordinary field name contains the word, and the exact form
missed webhook_signature and x_signature.

Where that leaves the two lists

client denylist name	answered by php-library
key, code, sig, otp, pin, hash, sess	query list
api_key, apikey	~api[_-]?key~i
password, passwd, pwd, token, secret	substring list
jwt, bearer, signature	substring list
auth	query-only on purpose — as a substring it eats author, authors, authored_by
email	masked by value, domain kept, wherever it appears

Every name is covered. The last two rows are the places the two implementations
genuinely differ, and in both php-library is the better of them: the client's
current auth substring over-redacts ?author=, and its email rule redacts a
whole field only when the name says email, where masking by value catches an
address in ?contact= or ?to= as well and keeps the domain, which is
diagnostic.

Why now

The client's console-client branch is not merged yet. Settling this before
it lands means the divergence never reaches master, and its UrlScrubber can
delegate to a plain new Logger with no addRemove() or addQueryNames()
extras at all.

Verification

Applied in the same pass to the console Scrubber, the WordPress Redactor
and both JS clients, and pinned in the shared corpus (scrub-body.json, whose
urls group drives the query-only half). Console full suite green, WordPress
22/22, node 23/23, the 645-test JS suite green, Logger 21/21, parity green
across five senders.

Upgrading

Additive. One more substring name and one more query name are redacted than
before; nothing that was redacted stops being.

v8.5.34 — the names westbahn redacted and this list did not

The names a client redacted and this list did not

A client's OTEL pipeline keeps its own denylists. Comparing them against this
library's showed drift in both directions — and nothing caught it, because
check-sender-parity.php compares the path rule only. The field- and
query-name lists had diverged unseen.

name	client denylist	php-library	promoted?
jwt, bearer	substring	—	✅ → substring list
otp, pin, hash	exact	—	✅ → query-only
apikey	exact	already covered by api[_-]?key	—
auth	substring	query-only	stays query-only
cookie	—	substring	(client lacks it)
sess	substring	—	❌ deliberately not
email	substring	masked by value, everywhere	—

Why the query-only half is query-only

Because that is what the client had them as: its DENYLIST_EXACT matches a
whole name, where its substring list holds password/token/secret. As
field names they over-redact, and this codebase has scar tissue on exactly
that:

• hash eats content_hash and filehash — the very content hashes the
  asset carve-out exists to keep. A bundle named <group>_<md5>.<mtime>.js
  once came back as [redacted] and 273 browser errors could not say which
  file threw.
• pin eats shipping, mapping, spinner, pinned and zipping —
  the same mistake pass once made with passengers, compass and bypass.

As a query parameter they are nearly always credentials, so that is where they
now bite.

Why sess is not promoted

The console stores session_id as a first-class column and correlates by it —
it is in the error detail, the issue facets and the grid. Redacting that name
would blind its own session view. The client can keep it locally; it should
not become everyone's default.

Pinned, and applied everywhere

The shared corpus (scrub-body.json in the console repo) grew a urls group
for the query-only half, and the same promotion landed in the console's
Scrubber, the WordPress Redactor and both JS clients in one pass. A list
change obligates every sender — which is what the corpus is for.

Verified: console 39/39 plus its full suite, WordPress 22/22, node 23/23 plus
the 645-test JS suite, php-library Logger 21/21, parity green across five
senders.

Also in this release

Rollup::routeName() gained an optional trailing $module (v8.5.33), and
a client now delegates its four rollup primitives — isStream,
routeName, bucketFor, hostName, plus DURATION_BOUNDS and
DURATION_BUCKETS — rather than carrying character-identical copies. Its own
codeception suite passes unchanged, 8 tests / 50 assertions.

Upgrading

Additive. Two more substring names and three more query-parameter names are
redacted than before; nothing that was redacted stops being.

v8.5.33 — a request body is parsed before it is scrubbed

⚠️ Security: the raw request body was scrubbed as free text, and a body is a document

Sender logs the raw request body so the console can replay the request that failed — a
JSON API call's $_POST is empty, so without it a replay is a bare method and URL. That body
went through the same flat <name> = <value> search as an exception message.

One mistake, six holes, all of them shipped:

shape	what happened
{"password": "correct horse battery staple"}	not redacted at all — the value class excluded whitespace, so the closing backreference could never reach the quote. It did not redact partially; it did not match
opts[api_key]=live_…	walked past — [ and ] were not in the name class, and that is the shape every form posts a nested field in
<password>x</password>	no XML body was touched anywhere. The separators were =>, : and =; XML writes the name and the value either side of a >
XML-RPC system.multicall	names its credentials not at all — the password is the second positional <string>, so no name-based rule could ever find it
{"query":"… login(password: \"x\") …"}	the escaped quote was read as the value, leaking the real one and writing a malformed document into storage
user=marcin in a body	the username rule never ran on text, so the same field meant different things depending on which channel carried it

Passwords contain spaces. Settings forms post bracketed names. The two most brute-forced
endpoints on the internet speak form-encoding and XML-RPC. Every one of those was a miss.

What changed

Ovos\Service\Console\Body parses rather than pattern-matches. JSON and form bodies are
decoded, handed to the key walk — Logger::remove(), which never had any of these holes
because it walks keys — their string values run through the text scrubber, and the result is
re-encoded.

Parsing also decodes, which is the half no pattern could reach:
note=password%3A%20hunter2 carries no separator for a text search to see, and a whole JSON
blob stuffed into one form field is one opaque token. Both are now read.

XML-RPC is reduced to its shape: every <string> becomes its length while the method names
stay. system.multicall × 200 × wp.getUsersBlogs says more about what is happening than the
passwords did.

Logger::removeText() keeps the same four passes for the places a body is not involved —
messages, backtraces, header values — where the identical holes also bit.

New: console.request_body

off | structure | full, default structure. There was no switch before: the body was
read on every non-GET report, 16 KB of it, at every log level, and nothing could turn it off.
structure parses, drops the fields named like credentials, masks e-mails and re-encodes at
8 KB; full is the previous behaviour.

An endpoint whose purpose is to receive a password — wp-login.php, xmlrpc.php,
/wp-json/jwt-auth/, /oauth/token and the usual siblings — sends no body whatever the mode
says. The console applies the same rule again on write, which is what reaches a deployment
running an older copy of this library.

Also

• Logger::maskEmails() is public and isSecretName() joins it — the question "is this
  name a credential?" asked on its own, which a header allowlist needs. A sender was otherwise
  obliged to keep its own copy of the secret-name list just to ask it.
• Rollup::routeName() takes an optional trailing $module — the ZF1 CMS sites route on a
  module/controller/action triple, so their copy carried a third segment and with it a second
  copy of the validation regex and the __other fallback. Optional and last, so every existing
  caller is untouched.

Together those two let the CMS writers delegate their whole redaction half — 323 lines whose
redact() was Logger::remove() verbatim, kept equal to this library by hand and policed by
a parity script. What that buys is not the lines: it is that "do the copies still agree?" stops
being a question anyone can answer wrongly.

Verification

The rules are pinned by a shared corpus in the console repo
(project/application/tests/fixtures/scrub-body.json, 17 cases) driven through every sender.
This library: 21 checks green, with its own suites at Logger 21/21 and Sender 27/27.

Upgrading

No API is removed and no call site changes. An application that wants the previous behaviour
sets console.request_body: full; one that would rather send nothing sets off.

v8.5.32

⚠️ Fix: two installs sharing an FPM pool shared one security-event budget

The rolling cap on type=security reports counted into
ovos:console:security:<minute> whatever the install. APCu belongs to the whole pool
and a pool can serve several — prod and test on one box, a handful of tenant vhosts —
so they all spent the same SECURITY_MAX_PER_MINUTE allowance.

The failure mode is quiet on both sides: a credential-stuffing wave against one install
reports its own 60 events a minute as designed, and the install next to it, doing
nothing unusual, has its auth failures silently dropped for as long as the wave lasts —
precisely when they are worth having.

Sender::securityKey() now puts the install's configured cache prefix
(cache.prefix, typically !ENV CACHE[PREFIX]) in front of SECURITY_PREFIX, joined
by the same Cache\Prefixer the cache stores use, with cachePrefix() — added in
v8.5.30 for the rollup counters — reading it off the app config. It is public static,
so the key can be asserted without touching APCu.

This was the last hardcoded APCu key in the sender; a scan of src/ found no other
(everything else keys off an injected prefix or a cache id). It is the same defect the
rollup counters had in v8.5.30, one key further along.

Inert where it does not apply: an install that configures no cache prefix keeps the
key it had. Where it does apply, the counter restarts once, inside its own 120 s window.

Tests: theSecurityCapCountsPerInstall — two installs key apart, one install still
shares a counter across its workers, the minute still rolls, and no prefix leaves the
key unchanged.

v8.5.30

⚠️ Fix: two installs sharing an FPM pool shared their rollup counters

APCu belongs to the whole pool, and a pool can serve more than one install — prod
and test on one box, a handful of tenant vhosts, a staging site beside the live one.
Every key this accumulator wrote was ovos:console:rollups:… whatever the install,
so those installs shared all of it, in three ways of increasing severity:

1. Counters summed across installs. …:<minute>:r:/user/index was one bucket for
   all of them, so one project's console got another's traffic as its own denominator.
2. One flush watermark. Whichever install crossed the minute boundary first
   advanced flushed, and the other then treated its own unshipped minutes as done.
3. The same (instance, seq). Both shipped under one identity, and the console
   dedups retries by exactly that pair — so it dropped one of the two as a duplicate.
   An additive counter never recovers from a fragment that was never added.

The namespace now comes from config, not from anything derived: cache.prefix
(typically !ENV CACHE[PREFIX]), the same value Cache\Perishable hands the APCu
store, joined onto PREFIX by the same Cache\Prefixer. Sender reads it
(Sender::cachePrefix()) and passes it in; the key constants became suffixes that
key() puts behind the prefix, and the APCUIterator pattern carries it too, so a
flush cannot even see another install's keys.

The identity needed the same treatment for a separate reason: separate keys give each
install its own seq counter, but the identity value is p<pid>-<base36 time> —
and one pool means one worker serving both installs, so the pid and the second can be
identical. A base36 crc32 of the key prefix goes in as a third segment, well inside
the console's [A-Za-z0-9._-]{1,64} identity shape (21 characters for a typical
prefix).

Inert where it does not apply: an install that configures no cache prefix keeps
exactly the keys it had. After the upgrade a pool mints a fresh instance, which is
the case the design already covers — an APCu restart looks the same to the console.

Verified against a real pool, which the unit tests leave alone by design: two installs
counting the same minute and the same route ship 3 and 7 requests under their own keys,
with distinct identities and no keys left behind. The suite adds
keysCarryTheInstallNamespace to Tests\Service\Console\Rollup.

The same fix landed in ovos/cms first; this is the port back to the original.

v8.5.29

Versioned cache: a compacted rule set, followed incrementally and shared through APCu

Every store instance used to load the whole rules stream with one XRANGE, and under
PHP-FPM every request is a fresh instance: at 1 147 rules in production that was
~110 KB and a parse per request, linear in the invalidation rate.

Rules now keeps the newest rule per tag (the all rules whole), and a refresh
fetches only what was appended since the last id it holds. SharedRules keeps the set
in APCu, keyed by the rules key and the connection, so a request adopts it instead of
loading the stream. One worker is elected with apcu_add to refresh a stale set while
the others keep it for the read; on an emptied APCu one worker loads and the others
wait briefly.

A process still sees its own invalidation at once, a wipe drops the shared set whatever
the wiping instance's option, a slower refresh never rolls the entry back, and exact
reads (rules_cache_ms 0) elect nobody.

New option rules_shared_cache, on where APCu is enabled. The Lua is unchanged, so
no FUNCTION reload is needed.

Tests: fourteen new cases through a probe that records every rules fetch, plus a
fresh-instance (FPM) benchmark scenario with a NoSharedRules twin on the versioned
benchmarks.

Console: the batch carries the deployment's tags

console.tags — a yml list, or one comma string such as !ENV CONSOLE[TAGS] — is
stamped on every event of the batch as the top-level tags list the console indexes
(a tenant, a region, a team; its TAGS column, one filter per tag). Trimmed, empties
dropped, at most ten; the console lowercases and validates each. Per-event tags keep
riding a capture's extra bag as tags, which the console unions with these.

Over OTLP the same list goes out as the resource attribute tags — OTEL has no tags
concept of its own, so this is the console's own attribute, beside service.version
and deployment.environment.name.

Docs

README.CACHE described the per-read walk over every unseen rule (O(1 + R)) that the
compacted rule set removed, and still spoke of a short-lived per-instance cache. Reads
are now O(T) against the newest rule per tag, the set is shared and refreshed
incrementally, the measured table is marked as predating the change, and the two new
FPM benchmark rows are named with their re-run noted as pending rather than filled in
with invented numbers.

v8.5.28

Tests: the lost-stream cache test stops racing the server clock

src is byte-identical to v8.5.27 — no runtime behaviour changes. It is tagged
because tests ship in the dist (no /tests export-ignore) and consumers run them,
so a flaky one is a red build for anyone pinned to v8.5.27.

lostRulesDoNotResurrectAnItemAfterAnotherRule slept so the server clock would move
between the rules stream's death and its rebirth, and still failed — ovos/console's CI
hit it on every push once its vendor followed a7fe142. A sleep is a bet on the runner,
not a fact about the data: stream ids are milliseconds, and 2 ms of sleeping buys
nothing if the scheduler hands the process back inside the same one.

The test now controls the stamp rather than the clock. It opens the rules stream by
hand at 1-0 — an id from 1970, in the shape cache_versioned_invalidate writes,
first and all — and the item stamps on that. Whatever id the reborn stream opens
with comes from the server clock and is decades newer, so rulesLostSince() sees a
rebuild every time, on any machine, at any speed. The assertion is the one it always
made; only the arithmetic is no longer a race.

rulesLostSince() itself is unchanged: it remains an ordering that cannot see a
rebuild inside its own millisecond, and the note in the source still states that
limitation and the identity-based fix it would take. No test depends on losing that
race any more.

v8.5.27

⚠️ Fix: the versioned cache could serve what it was told to forget

RedisVersioned's rules stream is the only record that an invalidation ever
happened, and it carried a TTL — which made it an eviction candidate like any
item. Evict it and XRANGE answers an empty array, indistinguishable from a
group that was never invalidated, so every item invalidated but not yet read
came back with its old value. Silently, for the whole group.

Reproduced under volatile-lru, the policy we run: 32 MB, one tagged item,
invalidated and not read again, and the stream was gone after 7 000 unrelated
writes while the item survived to be served stale. It is the same shape as the
RedisTagAwareAdapter tag Sets that bit us on allkeys-lru years ago, and the
reason we moved to volatile-lru — which protects Symfony's design and did not
protect this one.

The stream now carries no TTL, as Symfony's tag Sets do not, and PERSIST
strips the one an earlier version left behind, so an existing group repairs
itself on its next invalidation. Under volatile-* or noeviction the stream
is no longer in the eviction pool at all.

Under allkeys-* it can still be lost, and the read path answers for that
rather than trusting it. Every rule now records whether it opened the stream
(XADD NOMKSTREAM refuses to create one, so a nil reply says there was none),
and an item stamped older than an opening rule has seen rules the stream no
longer holds: a miss rather than a hit. The group recomputes once instead of
serving what it was told to forget.

Upgrading: run a volatile-* or noeviction eviction policy. No call site
changes. Two limits are worth knowing: an item written before its group's
first-ever invalidation carries no watermark to check and is served if the
stream is later lost, and the check is an ordering, so it cannot see a stream
lost and reborn inside the same millisecond its items were stamped in. Both
need the stream to go, which under the policies above means a DEL, a
FLUSHDB or a slot lost with a cluster node, not eviction.

Console sender: the untracked pass

Service\Console\Untracked finds the git or svn working copy at or above
BASE_DIR, asks it read-only (git --no-optional-locks ls-files --others --exclude-standard -z, never git status; svn status without -u, an
unversioned directory opened to its files) and builds the report ovos/console
takes at POST /api/v1/ingest/files with platform php.

An executable-shaped or server-config file is listed by path — urgent under a
web-reachable directory (console.files.web, default public), high elsewhere
— and everything else is counted per directory with an extension histogram and
never named. Every failure to ask is null, never a guess: no working copy,
proc_open closed, a non-zero exit, a run past the budget (stdout to a temp
file and a proc_get_status poll, because a Windows pipe blocks regardless of
the blocking flag).

Sender::untrackedReport() builds it (CLI only), Sender::reportFiles() posts
it and answers the console's code, and Sender::reportUntracked() does both for
a cron line. An empty answer still posts, which is how a finding goes GONE on
the console.

Console sender: the working copy also names what changed and what is gone

Beside the untracked list, git --no-optional-locks -c core.fileMode=false diff --no-renames --name-status -z HEAD — both answers are required, since a report
built from one would mark the other half's findings GONE. svn's M/A/R rows
are modified, !/D missing, a property-only change nothing.

A modified tracked file is always listed by path: its name is repository
content, not customer data. It carries the untracked tiers plus one of its own —
a browser script (.js, .html, .svg) under the web directory is high, the
card skimmer's shape. A missing tracked file is info, and the root area carries
foreign/modified/missing counters in the checksum pass's words.
core.fileMode=false, so a chmod is not a modification; the class docblock says
when a deploy should git update-index --refresh.