Document that each container is its own system, not one host and six services
The Docker stack reported one host: every container's log driver sends from the
bridge gateway, so all five services arrived carrying the same address and
System.find_or_add -- which keys identity on systems.address -- gave them one row,
while each container name was filed as a Service.
Architecture gains "Which host an event came from": the frame shapes, the pid-slot
rule SyslogSource uses to tell an application from a hostname, and why the pid in
the Docker frame is dockerd's rather than the container's.
Also recorded: the same rule fixes a non-Docker mis-parse. RFC 3164 as rsyslog
forwards it does carry a hostname slot, and the positional read filed web01 as the
Service -- the Service sanitizer strips digits, so the row said web -- and dropped
sshd entirely.
The rename is gated on the address being one of this container's own gateways AND
the importer being containerized; without the second gate a systemd deployment
would rename syslog from a LAN router. Noted in Installation's env table that the
escape hatch is therefore inert under systemd, so nobody goes looking for a knob
they do not need.
Development's frame-format bullet corrected: the tag lands where the application is
read from, not where the Service is read from. The docker CI job description now
mentions the step asserting per-container Systems, which passed on the runner with
3 hosts and none collapsed onto the gateway.
Co-authored-by: openhands <openhands@all-hands.dev>
Match the wiki to PostgreSQL 18 and the bind-mounted database
Three pages said `docker compose down -v # discard the database too`. That is
now false, and it was the description of a defect rather than of intended
behaviour: the database lived in a Compose named volume, which `down -v`
deletes, and which is namespaced by the project name so renaming the checkout
started a second empty database. Both reproduced; the stack now bind-mounts
./data/postgres.
The engine moved 17 -> 18. The pin had no reason behind it, and PostgreSQL has
no LTS tier (every major gets five years), so the newest major is the
longer-support choice.
Also recorded the two 18-specific traps, both hit while verifying this: the
bind mount target is /var/lib/postgresql rather than .../data, and
lc_collate/lc_ctype no longer exist as GUCs. Collation behaviour itself is
unchanged, so the varchar_pattern_ops reasoning still holds.
Version claims that only restated the pin are now written without a number
where the behaviour is version-independent, so this page stops going stale on
the next bump. Claims that record an actual measurement ("measured on 17.11")
are left as measured.
New: an "Upgrading the PostgreSQL major version" section on Installation --
the dump-and-restore procedure was executed end to end (17 cluster ->
pg_dumpall -> fresh 18 -> rows intact) before being written down.
This wiki edit was created by an AI agent (OpenHands) on behalf of the user.
Document the shared Preference row and its fallback
Held locally until PR #45 merges: this describes behaviour that does not
exist on master yet.
Co-authored-by: openhands <openhands@all-hands.dev>
Fix nine false or stale claims, verified against the tree
A page-by-page audit of all 23 pages against repo master found seven claims that
are simply false and two that are stale. Each was re-checked against the tree
before being changed rather than taken on trust.
False:
- /events/recent as the dashboard feed endpoint. No such route: events is index +
collection { get :search }, and there is no recent.js.erb. The feed refreshes via
GET /events.js; EventsController#recent is unreachable dead code. (Interface.md.)
- "Every management index screen is paginated at 25 rows". systems is deliberately
unpaged, as are services, words (fixed 20), alerts, events, polling_priorities.
- The list of screens pagination replaced: that commit touched twelve controllers,
systems was not one of them, and hunks -- which it did touch -- was omitted.
windows_targets and active_directory_connections inherited it later.
- A Search row as a string "plus its time window". searches carries string,
user_id, description, short_description and nothing else; no timeframe column has
ever existed, so it is chosen again at run time.
- "the systemd units set /var/DAD/offline_storage, mode 0700". No unit sets the
variable; the path comes from the default plus WorkingDirectory. And nothing in
the app sets 0700 -- EventPartitionBackup uses mkdir_p, so the directory gets the
umask default. This one is load-bearing: it told operators a directory of raw log
lines was permission-protected when nothing protects it.
- db:seed "seeds the two maintenance jobs" -- it seeds four.
- JobHandler::HANDLERS as "seven operations" -- eight since poll_windows_events.
Stale:
- "Below 720px" in three pages; the media query is max-width: 768px.
- Statistical-Alerting.md crediting a Math monkey patch; the helpers are
Stats.mean/variance/standard_deviation, and the patch was removed.
Also documents the new self-logging stack: every service but syslog ships its own
stdout/stderr to the collector, the two measured traps that decide whether anything
is stored at all (the log driver resolves syslog-address in the daemon's namespace,
and the frame format decides the Service), the resulting single-System attribution,
and the CI step that asserts Service names rather than merely a row.
Checked and left alone: the Viewer rights row. An earlier audit flagged it for
omitting preferences, but preferences is Admin-gated in CONTROLLER_RIGHTS, so the
row is correct as written.
Co-authored-by: openhands <openhands@all-hands.dev>
Document importing Windows hosts from Active Directory
Adds Active Directory Import, and updates the pages that the feature changes.
The new page leads with the part that is not obvious from the requirement: the
preview is persisted server-side and apply may only subtract from it, because a
WindowsTarget row makes DAD connect out to a named host carrying a domain
credential. It also records the three query behaviours that are load-bearing
(paging against AD's silent 1000-object cap, unclassifiable operating systems,
domain controllers), the two-credential model, and what is and is not covered by
CI.
Home: the feature is the most recent pass, and the chronology now reads in order
-- the Windows-source paragraph no longer claims to be "the most recent".
Access Control: new section on why two controllers are Admin-only at the
controller level, where reads are gated, unlike windows_targets. Corrects
hidden? to a class method.
Development: the two remote integrations and the seams they sit behind; suite
counts corrected to 581 + 40 runs / 1969 assertions.
Interface: the connection screens and the two-page import, including which rows
arrive pre-checked and which do not.
Installation: DAD_AR_ENCRYPTION_* and WINDOWS_SSL_VERIFY now name the second
model that reads them; one variable covers the LDAPS and WinRM trust decision
rather than two.
Windows Events: cross-links, and the LDAP gap added to "limits worth knowing".
What Changed Since 2016: new dated section, and the system-suite assertion count
corrected to 247.
Every claim here was checked against the code. Two were wrong as first written
and corrected: the LDAPS/StartTLS note is on the show page, not the index, and
the import result lands on the Windows targets screen.
Co-authored-by: openhands <openhands@all-hands.dev>
Update the wiki for direct WinRM polling of Windows hosts
DAD now polls Windows hosts itself with wevtutil over WinRM instead of
requiring OpenWEC on the Windows side plus a fourth daemon to forward
what it wrote. The OpenWEC path stays documented as the alternative,
because a subscription receives events pushed to it and a pull model
cannot recover records for a host DAD cannot reach.
Windows-Events.md is restructured around the two paths sharing one
pipeline rather than around OpenWEC specifically. New material: the
priority bands and the adaptive interval (clamp(500/eps, band_min,
band_max), EMA-smoothed rate, 25% deadband), per-host disable, the
credential model and why the encryption keys must not be rotated, the
60s delta overlap and the duplicate trade, and the failure behaviour
that keeps one bad host or channel from costing the others.
Also corrected along the way, independent of this feature:
- Access-Control.md never documented ACTION_RIGHTS at all, only
CONTROLLER_RIGHTS. That is the mechanism that makes reading a target a
Viewer action while writing or polling one is Admin, so it is now
spelled out with the reason for each override.
- Interface.md had no Windows targets section, and its paginated-list
count was twelve.
- Jobs.md listed seven handlers; there are eight. Added a note that the
Job frequency is not the polling interval, since the two numbers being
different is easy to misread as a bug.
- Development.md test counts were 488/40 runs, 1679 assertions.
- Installation.md was missing DAD_AR_ENCRYPTION_KEY,
DAD_AR_ENCRYPTION_SALT and WINDOWS_SSL_VERIFY, and had no warning that
rotating the credential keys makes stored credentials unreadable --
unlike SECRET_KEY_BASE, which is safe to rotate.
- Architecture.md and Home.md described Windows collection as OpenWEC
only.
- What-Changed-Since-2016.md gets an entry, since the older pages
describe a design that no longer holds.
Stated as limits rather than glossed over: the WinRM transport is not
exercised in CI (no Windows host; the transport sits behind a seam and
the parsing/delta/queueing logic is tested against recorded wevtutil
XML), and the pull path reads rather than subscribes, so records written
while a host is unreachable are only recovered if they are still in the
channel next poll.
This wiki update was prepared by an AI agent (OpenHands) on behalf of the
repository owner.
Co-authored-by: openhands <openhands@all-hands.dev>
Document Windows event ingest via OpenWEC
Adds [[Windows Events]]: the OpenWEC setup, the four output formats and
their field mapping, the environment variables, and a troubleshooting
table that names which symptom means which broken link.
The two things an operator can get wrong in a way that corrupts data
rather than erroring get their own headings -- do not point OpenWEC at
Logs/LogStaging, and why one record is not one line.
Home: new paragraph and a row in the start table. Architecture: the
unit table gains dad-windows, and the "nothing else should touch Logs/"
line now says *why* -- the collector's startup recovery moves every file
it finds in LogStaging, so a second writer there gets its in-progress
file renamed away mid-append. Installation: OpenWEC as an optional
requirement, the windows compose profile, and the five WINDOWS_*
variables. Development: the docker job's new Windows leg, and suite
counts corrected to 488 runs / 1679 assertions. What Changed Since 2016:
DAD is no longer syslog-only, with the two assumptions that stop holding.
Co-authored-by: openhands <openhands@all-hands.dev>
Update the wiki for PostgreSQL-only dev/test and the deleted provision.sh
SQLite is no longer the development and test engine, and Vagrantfile and
provision.sh are gone, so most pages described a setup that no longer exists.
- Installation: drop the Vagrant quick-start, make the manual path create a
Postgres database per environment and use db:migrate_from_scratch, and move
the secret-file guidance off provision.sh onto the systemd units.
- Development: replace "Two databases, and why CI has both" with "PostgreSQL
everywhere", documenting why db:migrate_from_scratch is required, why
maintain_test_schema is now unconditionally false (its old DATABASE_URL
condition inverted once Postgres became the default), and what
test/support/scratch_database.rb and query_plan.rb are for. Suite counts
updated to 436 runs / 1237 assertions / 0 skips.
- Search: expand the collation finding to the actual mechanism -- the
[term, term + U+10FFFF) range and why no sound upper bound exists under
en_US.UTF-8 -- with an explicit "do not reintroduce" note.
- Architecture, Retention, Jobs, Access-Control, Home: correct the
engine claims and provision.sh references.
- What Changed Since 2016: new "Removed: SQLite as the development and test
engine" section, including the two guards that look like dead SQLite
support but handle an unpartitioned Postgres install.
Co-authored-by: openhands <openhands@all-hands.dev>
Document the Rust syslog collector; correct the buffering claim
The collector is now syslog_collector/, a zero-dependency Rust crate; syslog.pl
is deleted (DAD PR #32).
Architecture, Home, Development, Installation and What-Changed updated: the
pipeline diagram, the unit/service tables, the CI job table (six -> seven, with
the new `collector` job), and the `DAD_SYSLOG_*` environment table, which now
documents the port/staging/processing/rotate knobs the Rust collector reads.
Corrected a claim that turned out to be wrong. Architecture said the 0-byte
staging file meant "the data is in the buffer, not lost", and Development said
"neither is data loss". Measured: 40 messages sent to the Perl collector left 0
bytes durable after both SIGTERM and SIGKILL, because nothing closed the handle.
It was data loss. The Rust collector flushes per record, so the staging file is
readable immediately.
Also documented three other deliberate departures from Perl -- orphan recovery at
startup, the 65507-byte datagram cap replacing silent truncation at 1540, and the
record format that Event#event_fields depends on -- plus the caveat that the
Perl/Rust equivalence check is not automated.
What-Changed-Since-2016 gains a "Changed: the syslog collector is Rust, not
Perl" section.
Co-authored-by: openhands <openhands@all-hands.dev>
Document the Docker stack, the path coupling, and the rotation trigger
Development: bin/docker-test as the preferred test command, a "Running the
whole stack in Docker" section with the four things that cost an afternoon,
and the DatabaseCleaner remote-URL allowance cucumber needs against a
non-localhost host.
Installation: a Docker way between Vagrant and the manual install, and Docker
listed in Requirements.
Architecture: the pipeline diagram implied both halves resolve Logs/ paths the
same way. They do not. syslog.pl hardcodes /var/DAD/Logs/...; import_syslog.rb
uses paths relative to its working directory. They agree only because every
systemd unit sets WorkingDirectory=/var/DAD, and nothing validates it. Run
either from anywhere else and the collector accepts UDP packets and writes into
a tree the importer never reads, silently.
Also corrected in both diagrams: rotation is not "every 60s". RotateLog is
called from logit, so it fires on a 60s boundary *and only when a packet
arrives*. With no traffic the staging file is never renamed and the importer
has nothing to claim -- a quiet system looks like a broken importer. Verified
against syslog.pl.
What Changed Since 2016: the Docker stack, and the fact that the Dockerfile
that shipped could not build (a bare gem install line is not a Dockerfile
instruction) and had no JS runtime, which broke every view render through
ExecJS.
Home: status paragraph and a Docker row in the where-to-start table.
Co-authored-by: openhands <openhands@all-hands.dev>
Add interface screenshots and correct disk/search documentation
Adds an Interface page with screenshots of the running UI (login, dashboard,
search results, jobs, users/roles/rights, preferences, alerts, displays,
exclusions, stored searches, systems) taken against a seeded demonstration
dataset, and threads the relevant ones into the pages they describe.
Corrects the record on disk utilization: there is no dashboard disk gauge. The
author commented it out in bec738c ("Hiding disk gauge", 2015-02-28); the live
surfaces are the disk_utilization job and preferences.archiveFreeSpacePercent.
Documents the open search-form bug (RECOMMENDATIONS 2.3): the Rails check_box
always submits timesearch_enabled[enabled]=0, and EventsController#search
branches on the presence of that key, so every form search takes the absolute
date branch with a now->now window and reports 0 events.
Co-authored-by: openhands <openhands@all-hands.dev>
Bring the wiki in line with the current project
The wiki last changed in April 2016 and described an alpha that let you type
Ruby into the Jobs form and the display editor. Both of those designs are gone,
so the pages were not just stale but actively misleading.
New pages: Architecture, Installation, Search, Displays (rewritten), Jobs,
Retention and Archives, Access Control, Exclusions, Development, and What
Changed Since 2016, which maps the 2015 material onto the current code.
The eight alerting recipes are kept as a recipe archive with a banner and an
API-status table on Example Jobs: the analysis shape still ports, but they call
a Job design that no longer exists, and Math.mean/variance/standard_deviation
were removed in favour of Stats.
Home is now a landing page with a status summary and a task-based index.
Co-authored-by: openhands <openhands@all-hands.dev>