What Changed Since 2016

The recipe pages on this wiki were written in 2015–2016, when DAD was alpha software that let you type Ruby into the web UI. Several of the things those pages depend on no longer exist, and a few claims in the old Home page were later found to be wrong. This page maps the old material onto the current code so the recipes stay readable without being mistaken for working instructions.

Removed: arbitrary Ruby in Jobs

Old design: Job#script held Ruby, and scheduler.rb ran eval(job.script) as whatever user the scheduler ran as (root under the old provision.sh). Creating a Job was therefore remote code execution by construction.

Current design: Job#script is a name looked up in JobHandler::HANDLERS, a closed registry of eight operations. Job validation rejects an unregistered name, and JobHandler.perform raises UnknownHandler rather than falling back to eval. See Jobs.

Migration 20261014000001_retire_legacy_job_scripts cleared script and next_run on every pre-existing Job row, because that Ruby cannot be translated into a handler name and the scheduler would otherwise raise a Critical alert on every cycle forever.

To carry a recipe forward, implement it as a handler in JobHandler::HANDLERS and deploy. That is deliberate: adding capability is a code change and a review, not something an operator can do from a browser.

Removed: parse_event Ruby in display filters

Old design (Displays as it stood in 2015): a Display's script defined a Ruby parse_event(event_text) method, and Event#parsed eval'd it in the web process.

Current design: a Display's display_script is a small declarative spec (fields, regex, label, join) that DisplayRule parses and never evaluates. See Displays.

The old scripts cannot be translated one-for-one — the spec selects and labels fields, it does not build arbitrary strings. The regex-heavy examples in the old page are the ones that port best, via the regex directive.

Removed: Math monkey patching

The old Statistical Alerting page opens by leaning on mean/variance/standard deviation "monkey patched onto the Math class". That reopened Ruby's core Math module process-wide, shared with every gem that touches it, and it carried a real bug. The helpers now live in Stats (app/models/stats.rb), with the hourly reductions in BucketStats. Empty input returns 0.0 rather than NaN, because NaN used to reach the dashboard charts and render gauges with NaN axes.

Added: integrity constraints the application never had

A 2026 pass found three things no layer had ever enforced. Each was reproduced with a failing test against the then-current master before being fixed.

A blank Frequency field could stop every job. Job#frequency was an unvalidated integer, and scheduler.rb computed next_run on a line sitting outside its begin/rescue. A nil frequency raised past the rescue, out of the loop and out of the process; Restart=always restarted it, the crashing row's next_run had not advanced so it was re-selected immediately, and no Job ran at all. The loop body is now JobScheduler (testable), frequency is validated and defaults to 3600, and a row with no usable interval is disarmed rather than crashing the process. See Jobs.

Two accounts could share a username. Nothing enforced uniqueness, and do_logon resolves credentials with find_by, so the lowest id won and the second account could never sign in — silently, with no error to the operator. A blank username was also accepted and reachable by submitting the logon form with the field left empty. users.username now has a unique index plus model validation. The migration repairs existing duplicates by suffixing the row id and renames blanks to disabled-account-<id>; it deletes no account, because an account is an operator's work.

The per-request session lookup scanned the whole table. gatekeeper resolves cookies[:sessionID] with find_by(session_hash:) on every authenticated request, and sessions.session_hash had no index at all. It now has a unique one. See Access Control.

Migration 20261108000001_add_integrity_constraints carries all three. It runs disable_ddl_transaction! and builds the indexes CONCURRENTLY on Postgres, so it does not lock the tables the web tier writes to.

Corrected: the single-quote mangling note

The 2016 Home page warned that event text had single quotes doubled because the inserts were not bound queries. That is gone. All SQL is parameterized (exec_query(sql, name, binds), routed through SqlPlaceholders.query, which rewrites the ? placeholders Rails' API uses into the $N form PostgreSQL wants). Event text is stored at full fidelity.

Corrected: what secret_key_base actually protects

An older review claimed that recovering the committed 2015 production secret_key_base allowed impersonating any user. It does not. Login state is cookies[:sessionID], a SecureRandom.hex(32) token looked up in the sessions table and written as a plain cookie value; nothing in the app uses cookies.signed/cookies.encrypted/MessageVerifier/MessageEncryptor. The key signs the Rails cookie_store session, which carries only the CSRF token and flash. So leaking it defeats CSRF protection but does not impersonate anyone, and rotating it logs nobody out. See Access Control.

That item is now closed. The value remains recoverable from the 2015 commit, but no live deployment is known to still hold it, which makes rotation moot; and rewriting shared history in a public repository with forks does not un-leak it, so that remains declined. bin/rotate_secret_key_base still exists as the general rotation path, because a signing key with no supported way to change it is its own problem regardless.

Removed: an index on events_words that nothing could use

events_words carried two indexes: the composite (word_id, generated, event_id) search actually reads, and a standalone index on generated added in 2015. The second one was unreachable from every query in the application on both engines, while every ingest insert paid to extend it: 28 bytes per row — 34 MB at 1.27M rows, roughly 2.8 GB at the 100M-row scale the project assumes — and 5.1% of insert time.

Nothing reads the join table by time alone: search always restricts on word_id, retention drops whole partitions rather than DELETEing by timestamp, and the word admin screens never touch it. Confirmed rather than reasoned: EXPLAIN never selected a plan through it, and on Postgres the partition child's composite index recorded idx_scan=2 while it recorded idx_scan=0.

Two numbers that had been quoted for it did not survive re-measurement. It was recorded as making search 2.3–185× faster: true when that was written, before the search rewrite, and false after it — search is now identical with or without the index (5.94 ms vs 5.95 ms). The "~6%" insert cost re-measures at 5.1%. The general lesson is that a performance claim measured against one query shape stops being true the moment that shape is rewritten, and nobody goes back to re-check it.

Dropping it is not the usual DROP INDEX CONCURRENTLY. Postgres refuses that form for a partitioned index:

ERROR:  cannot drop partitioned index "index_events_words_on_generated" concurrently

A plain DROP INDEX cascades to every partition and leaves no orphaned child indexes, but takes ACCESS EXCLUSIVE on the parent, so the migration sets a 5 second lock_timeout. On a busy collector that means the migration fails fast and is re-run later, rather than stalling ingest behind whatever writer happens to hold the lock. See Development.

Added: ingest limits that only production hit

A later pass looked at the ingestion write path and found two defects that the test suite could not reach. Both were reproduced with failing tests against the then-current master first.

A production-sized batch was rejected by the database. Each flush built one multi-row INSERT per table with no cap on rows. events_words takes three columns and gets roughly sixteen index rows per event, so the production default batch of 2000 events needs on the order of 96,000 bind parameters, against a per-statement ceiling of 65,535. Past the ceiling the adapter raises too many SQL variables — a hard failure, not a slowdown. The suite could not see it because DEFAULT_BULK_INSERT_SIZE is 1 in development and test, so in-process tests flush one event at a time. Batches are now chunked at the adapter's ceiling and committed in a single transaction, so a batch is still all-or-nothing and batch size is no longer bounded by the database. See Architecture.

The importer died on its first word-cache prune. prune_words reads cache_lifetime before it writes it, and nothing ever assigned it: resetCaches initialises every other piece of ingest state but not that one, and the commit that extracted EventIngestor out of Event moved the reader and the writer while dropping the @@cachelifetime = 320 assignment. Pruning is triggered once more than 100,000 words are cached, so this surfaced mid-file on a long ingest as TypeError: can't convert NilClass into an exact number. The three tests that touched prune_words all set the value by hand, which is why the suite stayed green. The default is restored and resetCaches now owns it.

The lesson both share: ingest state lives in class-level variables, and resetCaches is the only place it is defined. Anything not initialised there is broken in exactly the ways above — invisible until a run is long or large enough.

Changed: how words get into the index

Not a defect — a throughput finding, recorded here because it changes what the ingest path does.

Ingest used to resolve every token that missed the in-memory word cache straight against the database, one at a time, as each line was parsed: a lookup, and for a new word an insert in its own transaction. On 20,000 realistic syslog lines that was 186,089 statements and 46,515 transactions, and the words table accounted for about 55% of total ingest time. Raising the batch size could not help: once batches were chunked to the database's parameter limit (above), throughput stopped improving past roughly 2000 events per batch, so the remaining cost was elsewhere.

Unknown tokens are now held until the batch is flushed and resolved in bulk — one indexed lookup for words that already exist, one bulk insert for the rest. On the same corpus that is 360 statements and 67 transactions, and end to end 6.4× faster with an empty words table and 2.36× faster once it is populated. The steady-state number is the one that matters for a running collector; the 6.4× is what a backfill sees.

What did not change: the events_words rows written are identical — same row count, same word count, no orphans, no null word_ids — and the search plan is unchanged. This is a faster way to build the same index, not a different index. Nothing about query behaviour, truncation, or the 255-character limit moved. See Architecture.

Worth knowing if you are reading ingest logs after a crash: a word seen for the first time now reaches the words table when its batch is flushed rather than when its line was parsed. It is still covered by the same at-least-once guarantee; it just lands slightly later.

Added: a Docker Compose stack, and the suite now runs on PostgreSQL

The repo ships Dockerfile, docker-compose.yml, docker-compose.db-port.yml, .dockerignore and bin/docker-test. One image; one container per long-lived process — db (PostgreSQL, pinned to a major), migrate (one-shot), web, importer, scheduler, syslog — matching the three systemd units rather than supervising everything in one container. Production deployment is the three units under systemd/ plus Apache/Passenger.

bin/docker-test runs minitest and cucumber against the Compose database and then makes the same schema-shape and INVALID-index assertions the CI test job makes. It is now the preferred way to run the suite. At the time it landed, the SQLite path still skipped 33 tests the production engine needed exercised; that gap is now closed outright, see "Removed: SQLite as the development and test engine" below.

Two things worth knowing, both of which used to be invisible:

• The image could not build. The old Dockerfile.dev had a bare gem install bundler -v 2.6.7 --no-document line, which is not a Dockerfile instruction, so docker build failed on it. It also shipped no JavaScript runtime, so every view render raised "Could not find a JavaScript runtime" — ExecJS is reached through the sprockets pipeline in the layout. No CI job built the image, so neither was ever noticed. Both fixed, and a docker CI job now builds the image and drives the running stack end to end — including a UDP probe that asserts a syslog packet reaches events, which is the only automated test of the collector.
• The collector and the importer agree on paths by coincidence. The collector hardcodes /var/DAD/Logs/...; import_syslog.rb uses paths relative to its working directory. They line up only because every systemd unit sets WorkingDirectory=/var/DAD, and nothing validates it. The image therefore uses WORKDIR /var/DAD. See Development.

Removed: SQLite as the development and test engine

PostgreSQL is now the only engine, in development, test and production. The sqlite3 gem is gone from the Gemfile, config/database.yml uses the postgresql adapter everywhere, and CI's separate SQLite and Postgres jobs collapsed into one test job against a pinned PostgreSQL service major.

This is the completion of a line of work rather than a new preference. The engine split is what produced the worst bug in the project's history: raw SQL written in the form SQLite accepts, rejected outright by Postgres, production unable to insert a single event while every CI job stayed green. Running the suite on the real engine made that visible; removing the other engine makes it impossible. Concretely:

• The suite reports 0 skips. It used to be 32 skips on SQLite and 3 on Postgres, with three more tests defined only on Postgres. Adapter-conditional skips are exactly how a green run stopped meaning anything.
• Vagrantfile and provision.sh are deleted. They were the other half of the same split — provision.sh installed the sqlite3 package and was the only thing that generated the secrets the documentation described. The systemd units remain the deployment path and now document their own secret setup. The CI provisioning job became systemd.
• test/support/scratch_database.rb and test/support/query_plan.rb centralise what four test files had each open-coded. See Development.

Two things that read like dead SQLite support and are not: Event.approximate_count's fallback to an exact COUNT(*) — the pg_stat_user_tables sum over events_p* comes back NULL on any Postgres install whose partitioned shape was never built, which is a real state reachable by running plain db:migrate — and EventPartitioning's supported? guard, which keeps a Job row against a non-partitioned database from alerting once a cycle forever. Neither is a second engine; both handle an unpartitioned PostgreSQL install. The approximate_count fallback had no test and now has one (test/models/event_count_fallback_test.rb); removing either as "dead SQLite support" would turn a loud failure into a silently wrong number.

SqlPlaceholders.query also stays. It is not multi-engine support; it is the one place that turns the ?-and-binds form Rails documents into the $N form PostgreSQL wants.

Changed: the syslog collector is Rust, not Perl

syslog.pl is deleted. syslog_collector/ is a zero-dependency Rust crate (std only) that speaks the same UDP/514 protocol and writes the same staging records. The Docker image builds it in a separate stage, so the runtime image no longer installs perl at all. On a host, build it with cargo build --release and install it into /usr/local/bin, where the dad-syslog unit points.

The rewrite was a chance to measure the collector rather than read it, and three measurements turned out to be data loss, now fixed:

• Perl block-buffered print LOG with no signal handler. 40 messages sent, 0 bytes durable after SIGTERM and after SIGKILL. The old docs called the 0-byte staging file "buffering, not data loss" — that was wrong; nothing ever closed the handle. The Rust collector flushes per record, which measures free.
• Perl left the in-flight file in LogStaging on every restart, and nothing reads LogStaging — so up to 60s of logs were orphaned permanently, including on a planned systemctl restart. The collector now recovers those at startup.
• recv($buf, 1540) silently truncated larger datagrams. The cap is now 65 507, the maximum UDP payload.

Everything else is byte-identical: run against the same 22-packet corpus, the file handed to the importer was the same 2116 bytes with the same SHA-256. The record format (<source-ip> <message>) and traffic-driven rotation are unchanged, because Event#event_fields and the CI probe depend on them. See Development.

Added: a browser-level test suite, and the frontend cleaned up around it

Six changes, landed in order, each with its own guard. None of them changed what the app does; several changed what a reader can see, and two fixed real defects.

A suite that renders pages. Nothing in the old suite rendered app/views/layouts/application.html.erb — not once. The nav bar is generated at runtime from $_models and filtered by each model's hidden?, so a model that fails to load at boot, or a rights change, removes a tab from every page while every test stays green. Same for the dashboard's #events, #stats and #*_chart divs: the dashboard JS writes into them by id, so renaming one blanks the landing page.

test/system/ now covers the layout, nav bar, alert queue, auth flow, search form, jobs screens, form errors, responsive markup and id uniqueness — 40 runs / 247 assertions. It runs on rack-test, not a real browser: ApplicationController#ssl forces http → https outside development, and there is no chromium-driver matching the installed Chromium, so Selenium fails at session creation. The consequence is documented in Development and is worth internalising before writing a test: rack-test runs no JavaScript, so assert on the static shell the server rendered.

The structural reason this gap could not be closed by accident: railties sets DEFAULT_TEST_EXCLUDE to test/{system,dummy,fixtures}/**/*_test.rb, so rake test — the command CI and bin/docker-test run — skips test/system. It needs its own rake test:system task, which both now run.

One stylesheet instead of two. application.css and a stock scaffolds.css.scss both went into the same bundle. The second had never been edited since the app was generated — which is exactly why it still mattered: every generated _form.html.erb uses div.field, div.actions, #error_explanation; every show page uses #notice; .field_with_errors is emitted by Rails itself and appears in no view. Deleting it would have restyled every form. Merged, after checking mechanically over the compiled bundle that neither file referenced a selector the other defined — zero overlap, so the cascade cannot change. Verified by diffing the compiled bundle rule by rule against a saved baseline.

A viewport, and two layouts that could not fit a phone. The layout emitted no viewport meta tag, so a phone rendered at a ~980px virtual width and scaled the result down — every width-based media query then keyed off a width the device does not have. lang="en" was missing too. Two things genuinely did not fit:

layout	required width	problem
nav bar	19 tabs × 120px = 2280px	#navbar ul set overflow: hidden, so tabs past the edge were not wrapped and not reachable — clipped. An admin saw 10 of them.
alert queue	200 + 200 + 700 + 40 = 1140px	display:table with fixed display:table-column widths; no phone is that wide.

Below 768px the tabs now wrap and shrink and the queue stops being a fixed-width table. Deliberately no hamburger menu — that would need JavaScript the app does not have.

One error-summary partial. All 14 _form.html.erb files carried the same 11-line error_explanation block. Normalising the model name out left one distinct shape across all 14 — and the loop variable had already drifted: 8 used |msg|, 6 used |message|. Replaced by app/views/shared/_error_explanation.html.erb; 154 lines removed, 14 added. The heading noun ("prohibited this user from being saved") was a hardcoded string in each copy and is now record.model_name.human.downcase.

Fixed: 209 duplicate id attributes across 17 pages. Duplicate ids are invalid HTML, and browsers resolve getElementById and #id CSS to the first match, so every later element with that id is silently unaddressable. Found by scanning rendered HTML, not source:

source	duplication
nav bar <li id="active"> / <li id="inactive">	10–11 per page, every page
_alerts.html.erb #alert-time, #alert-system, #alert-desc, #alert-ack, #criticality-N	51 on /alerts
23 views re-emitting the layout's <p id="notice">	1 per affected page
#turbokiller (5 views), #comment (per-comment partial)	1–2 per page

Repeated per-row ids became classes (.criticality-5, .alert-time, …), nav active state became li.active, and the layout now owns the single #notice. Duplicate ids in rendered output went 209 → 0 across 17 pages, with div counts, links, forms and normalised cell text unchanged. test/system/unique_ids_test.rb fails on any duplicate, so it cannot regress.

Fixed: complete: $.ajax() fired a stray XHR for the current page, in three places.

jQuery.ajax({
        type:'GET',
        url:'/events.js',
        dataType: 'script',
        complete: $.ajax()      // <-- calls $.ajax() with NO options
        })

complete: is a property, so $.ajax() is invoked while the options object is being built, not registered. Two consequences, both real:

1. With no options, jQuery falls back to ajaxSettings.url — which the built bundle confirms is url:He.href, i.e. document.location.href. Each of these fetched the page you were already on. On the dashboard that is a full HTML render of /events, queries and all.
2. jQuery's Callbacks.add only pushes values where isFunction is true. A jqXHR is an object, so it was dropped: the complete: handler never ran and never has.

Sites were events.js.coffee, systems.js.coffee, and — the one that mattered — the 60-second dashboard poll at the bottom of app/views/events/index.js.erb, which fired unconditionally. Reachability was checked rather than assumed: systems.js.coffee was in fact inert, because its guard is #system-gauges and the only such element is commented out in systems/show.html.erb.

The line is removed at all three sites rather than replaced — there was never a working callback to restore, and inventing one would change behaviour rather than fix it. The two .coffee files became plain .js so the app's own behaviour reads without a CoffeeScript step, but coffee-rails itself stays: //= require turbolinks compiles turbolinks-2.5.4's own turbolinks.js.coffee, and Gemfile.lock shows turbolinks (2.5.4) → coffee-rails, so removing the gem fails the production precompile.

Also $("services_chart") → $("#services_chart") in systems/show.js.erb. As written it was a type selector for a <services_chart> element that never exists, so the "Loading…" line was dead. The adjacent draw_graph("services_chart", …) is correct — it takes an id string, not a selector.

The CoffeeScript→JS conversion was checked rather than asserted: the real sprockets-compiled output and the new JS were run under node against a stubbed jQuery that records every $.ajax call, selector touched and handler registered, across five DOM scenarios. In every one the intended behaviour is identical — same hide #spinner, same bind:click, same each *[id*=searchload] n=1, same intended AJAX GET /events.js dataType=script. The only difference is the stray call: 1 → 0.

Two guards now hold this: dashboard_render_test.rb renders /events.js and rejects any no-argument $.ajax() and any complete:, and CI greps the built bundle for [A-Za-z_$]+\.ajax\(\s*\) — verified to match both occurrences in the old bundle and zero in the new one.

Windows events are a new source (2026-11)

Pages here describe DAD as syslog-only. It no longer is: Windows Event Log events collected by OpenWEC can be ingested. They are converted into the same staging record format, so everything on Architecture and Search applies to them unchanged — but two things a reader might assume do not hold:

• A Windows event's Service is a Windows provider name (Microsoft-Windows-Security-Auditing), not a daemon name. Filters written against syslog daemon names will not match it.
• Fields land as key=value pairs inside hunks (eventid=4688, data_targetusername=jsmith), not as the space-separated positional fields a syslog line has. Event#event_fields still splits on whitespace, so = reads as a separator between two indexed words.

See Windows Events.

Changed: DAD polls Windows hosts itself (2026-11)

The first cut of Windows collection required OpenWEC installed on the Windows side and a fourth DAD daemon to forward what OpenWEC wrote. That put the polling cadence outside DAD's control and made the operator run and secure a second collector.

DAD now polls the hosts itself: it runs wevtutil remotely over WinRM, converts what it reads, and queues it. The OpenWEC path is kept, because a subscription receives events pushed to it and a pull model cannot recover records for a host DAD cannot reach.

What a reader of the older pages would not expect:

• The poller is a Job, not a daemon. It runs inside dad-scheduler, so enabling direct polling adds no unit and no container. See Jobs.
• The polling interval is per host and adaptive, not a single global cadence. Each host has a priority band (polling_priorities, editable rows) and within its band its interval tracks the rate it is producing. The Job's frequency only sets how often the due-list is swept. See Windows Events.
• A target row stores a domain credential, encrypted at rest. That is why writing a target, polling one on demand, and editing a priority band require Admin even though reading them is a Viewer action — see Access Control — and why production refuses to boot without DAD_AR_ENCRYPTION_KEY and DAD_AR_ENCRYPTION_SALT. See Installation.
• Duplicates are expected, within a 60s overlap. wevtutil has no cursor, so the delta boundary is a widened time window. This extends the at-least-once guarantee the ingest path already had rather than introducing a new one.

Nothing below the conversion step changed: the same WindowsEvents.convert, the same staging line, the same LogsToProcess queue, the same importer.

Added: the Windows host list can come from Active Directory (2026-11)

Direct polling made WindowsTarget the list of machines DAD reads, but the list itself was still hand-maintained. DAD can now query Active Directory for computer objects and import the ones that are not already targets. See Active Directory Import.

What is easy to misread from the outside:

• The import is two steps, and the preview is stored in the database. Not a convenience. A WindowsTarget row makes this server connect out to address carrying a domain credential, so an apply step that took host names from the submitted form would be an SSRF primitive with a domain login attached, reached through a form whose only visible affordance is unchecking boxes. The candidate set is written server-side at preview time and apply intersects with it — it can only ever subtract. A host the directory never returned cannot be imported, whatever is posted.
• Admin is required to read these screens, unlike Windows targets, where reading the list is a Viewer action. The sensitive act is the enumeration: a query returns the whole machine inventory of a domain. See Access Control.
• A connection holds two credentials, a directory bind credential and the WinRM credential applied to everything it imports. The second exists because an imported host without one is a target that can never poll.
• Machines with no recognisable Windows OS are listed but left unchecked, and counted. A filter that quietly omits them is how an operator ends up believing a fleet is fully covered.
• Duplicate detection matches the FQDN and its first DNS label, case-insensitively. AD returns web01.corp.example.com; an operator typing a host by hand writes web01. Two targets for one machine double-poll it and interleave two offset states.

Nothing in the ingest path changed. An imported host is an ordinary WindowsTarget and polls through the same cycle as one added by hand.

Changed: the Compose database is bind-mounted, and the engine moved to PostgreSQL 18 (2026-11)

Two things about the Compose stack were wrong in ways that produced no error anywhere. See Development and Installation.

The database was in a named volume, and named volumes are quietly disposable.

• docker compose down -v deletes named volumes. Reproduced: a probe table was simply gone afterwards. The -v is muscle memory from cleaning up throwaway stacks, and it was taking the development database with it.
• A named volume is namespaced by the Compose project name, which defaults to the checkout's directory name. Rename or move the checkout and up creates a second, empty database; the old one stays behind under the old name, unreachable, and the dashboard just looks empty. Reproduced.

The stack now bind-mounts ./data/postgres. Verified the probe table survives down -v plus a full container recreate, and that the data follows the checkout across a project rename. data/ is git-ignored.

The postgres:17 pin had no reason behind it. Nothing in DAD uses a feature or avoids a regression tied to a specific major, and the common assumption that PostgreSQL has an LTS tier is false: every major is supported for exactly five years, so the newest major is the long-support choice (18 runs to November 2030; 17 stops in November 2029). Moved to 18 and checked for breakage rather than assuming it:

• The 18+ images changed the data layout. Mount /var/lib/postgresql, not /var/lib/postgresql/data — each major now lives in its own subdirectory (./data/postgres/18/docker on the host). Mounting the old path makes the entrypoint refuse to start.
• lc_collate and lc_ctype no longer exist as GUCs in 18; locale is per-database, so read pg_database.datcollate. Nothing in this repo reads them.
• Collation behaviour is unchanged, which matters because the varchar_pattern_ops index and the [term, term+U+10FFFF) range depend on it. Verified on 18: C orders A,B,a,b and en_US.utf8 orders a,A,b,B, same as 17.
• The pg_dump --table <partition> recovers zero rows into a fresh database claim had only ever been checked on 17. Re-checked on 18: still true.

Upgrading an existing cluster is not done by editing the pin — the on-disk format is not forward-compatible. Installation has the dump-and-restore procedure, which was executed end to end here (17 cluster → pg_dumpall → fresh 18 → rows intact) rather than written from memory.

The image also stopped shipping two toolchains no running process uses: the Rust toolchain and build-essential/libpq-dev now live in build stages, taking every service image from 1.22 GB to 440 MB.

Changed: each container is its own system, not one host and six services (2026-11)

The Docker stack reported one host. Every container's syslog log driver sends from the bridge gateway, so all five services arrived carrying 172.18.0.1, and System.find_or_add — which keys identity on systems.address — handed them all the same row. Measured before the change: one System, (172.18.0.1), holding events from dad-web, dad-db, dad-importer, dad-scheduler and dad-migrate. The per-host views showed a single machine.

The container name, which is the only host identity the line carries, sat in the application slot, because Docker's default frame omits the RFC 3164 hostname slot:

172.18.0.1 Nov 13 15:14:49 dad-web[149784]: Started GET /users/logon

lib/syslog_source.rb now resolves the application and host slots structurally, keying on the pid slot. See the "Which host an event came from" section of Architecture.

Two details that were assumed wrong until measured:

• The pid is dockerd's, not the container's. All five services reported 149784 — the same number — so it carries no per-process information and is never used as an identity.
• The sanitizer is gsub!(/([^a-zA-Z0-9 \-_:@\*\/.])/," "), where \- is an escaped literal hyphen, so [ and ] are replaced and sshd[4242]: becomes the three tokens sshd 4242 :. Reconstructing that same class with tr parses \-_ as the range -.._, which includes [ and ] and gives the opposite answer. Verified against the real expression.

The same rule fixed a second bug that has nothing to do with Docker. RFC 3164 as rsyslog forwards it by default does carry a hostname slot, and the positional read filed web01 as the Service — and the Service sanitizer strips digits, so the row said web — while dropping sshd entirely. The suite's own canonical line shape, <ip> <ts> <host> <svc>[pid]:, was mis-parsed the same way.

The rename is gated on the source address being one of this container's own default-route gateways and the importer actually being containerized. Without the second gate, a systemd deployment would rename any syslog arriving from the LAN router — routers do send syslog, and their address is their gateway. A remote host's address is never rewritten. DAD_SYSLOG_SELF_HOST=none disables the rewrite.

The Windows path was checked rather than assumed: it emits <host> <ISO-8601> <provider> … with no pid slot, so it takes the positional fallback and still records win10.windomain.local / microsoft-windows-security-auditing. See Windows Events.

Events already stored are not relabelled. Identity is decided as a line is read, so the pre-existing IP-addressed System keeps its history. After upgrading, expect one stale System holding everything ingested up to the restart and per-container Systems from that point on. Observed directly: the old row stopped receiving events at the minute the fixed stack came up, and sorting Systems by max(events.generated) puts it last despite holding the largest count — which is how to tell history from a live collapse.

Still true

• Word-position tracking was dropped in 2016 and is still not back: search is a word-prefix index, not a substring or phrase index. See Search.
• Events are stored at full fidelity, so display filters and alert logic must tolerate mixed case and punctuation.
• The dashboard, alerts, and statistics model described in the recipe pages is recognisably the same one, with the API surface (Event.search, Alert.genericAlert, Alert.hostUnreachable, System#hourly_stats, Event.diskUtilization) still present.
• Event.diskUtilization is present but its semantics changed: it now measures the partition holding the archives rather than /, and returns nil rather than 0 when the filesystem cannot be measured. Recipe code that compares it with > will raise on nil. See DAD Disk Space Warnings.

One caveat about Alert.hostUnreachable

The Hosts Up recipe calls Alert.hostUnreachable from a Job. The method still exists and works, but nothing in the current application calls it — the dashboard reachability check that used to fire it is not wired up, and System#monitor is a form field with no consumer. So "is this host still reporting" is currently something you have to implement, not something DAD does for you. System.with_events_within_hours is the primitive to build on.