Repository navigation
Installation
- Ruby 3.3.8 (
.ruby-version) - Rails 8.1 (installed via Bundler; Bundler 2.6.7 per
Gemfile.lock) -
PostgreSQL 17 for every environment — production, development and test.
SQLite is not supported: it cannot partition
events/events_words, so a SQLite suite never exercises the shape production queries. See Development. - Apache + Phusion Passenger for the web UI, if you deploy by hand. The units
under
systemd/are the process supervisor. - OpenWEC, only if you want Windows Event Log events. It runs on Windows and is not part of this stack; DAD just reads the file it writes. See Windows Events.
-
Rust (
rustc+cargo) to build the syslog collector. It is a zero-dependency crate, so the distro packages are enough; build it into/usr/local/binwithcargo build --release. The Docker image needs nothing from you — it builds the collector in its own stage. - Docker + Compose, if you want the whole stack or the test suite against PostgreSQL without installing any of the above. See Development.
For development and testing — the app in its own containers, PostgreSQL in a separate one:
docker compose up --buildBrowse to http://localhost:3000 and sign in as admin / dad_dev_password.
A one-shot migrate service builds the PostgreSQL schema from db/migrate on an
empty database and seeds the roles, rights and admin user; on later starts it runs
plain db:migrate and leaves your data alone.
docker compose logs -f importer # the ingest daemon
docker compose exec db psql -U dad dad_development
docker compose down -v # discard the database tooThe stack also watches itself: every service except syslog ships its own stdout
and stderr to the collector, so container logs arrive as ordinary events under a
Service named after the container. Search for dad-web or dad-db in the events
view rather than reaching for docker compose logs.
This is the development and test path. Production deploys through the units
under systemd/ plus Apache/Passenger, described below, because that is what
serves the app over TLS. The container split mirrors those units one-for-one
— db, migrate, web, importer, scheduler, syslog — so the topology is
the same one production runs.
A seventh service, windows, sits behind a Compose profile because it needs an
OpenWEC installation this stack does not provide. It will not start under a plain
docker compose up:
docker compose --profile windows up -dGotchas and the reasoning behind each choice are on Development.
You need a PostgreSQL cluster reachable by the app in every environment.
bundle install
# development
createuser dad && createdb -O dad dad_development
export DATABASE_URL=postgres://dad:dad@127.0.0.1:5432/dad_development
bundle exec rake db:migrate_from_scratch
# test
createdb -O dad dad_test
export DATABASE_URL=postgres://dad:dad@127.0.0.1:5432/dad_test
bundle exec rake db:migrate_from_scratch
bundle exec rake test
# production
export SECRET_KEY_BASE=$(openssl rand -hex 64)
export DAD_DB_USER=... DAD_DB_PASSWORD=... DAD_DB_NAME=events_prod DAD_DB_HOST=127.0.0.1
export DAD_ADMIN_PASSWORD=$(openssl rand -hex 12)
bundle exec rake db:migrate_from_scratch RAILS_ENV=production
bundle exec rake db:seed RAILS_ENV=production
bundle exec rake assets:precompile RAILS_ENV=productionUse db:migrate_from_scratch, not db:migrate, on an empty database: the
committed db/schema.rb is a portable dump that cannot express PARTITION BY,
and db:migrate loads that dump and skips db/migrate entirely, so the
partitioned shape is never built. See Development.
db:seed is idempotent — it creates the five rights, the two roles, and the admin
user, and re-running it changes nothing. It deliberately does not reset an
admin password that has been changed since. It also seeds the four maintenance jobs
described on Jobs: maintain event partitions, prune expired sessions, tune the
dashboard feed, and poll Windows event logs.
Seeding creates the shared user_id = 0 Preference row as well. No migration
creates it, so before seeding the Preferences screen lists no row to edit at all.
The dashboard no longer depends on that row existing — with it absent, the live
feed falls back to 50 events rather than failing — but seeding it keeps the
Preferences screen usable out of the box. See Interface.
Set either DATABASE_URL or the individual DAD_DB_* variables. If
DATABASE_URL is set, Rails uses it and the DAD_DB_* variables are ignored for
the connection.
| variable | required | used by | notes |
|---|---|---|---|
SECRET_KEY_BASE |
yes (production) | Rails | Signs the Rails cookie_store session, which carries only the CSRF token and flash — not login state, which is a sessions table row. Without it Rails refuses to boot. See Access Control. |
DATABASE_URL |
one of these two | Rails / config/database.yml
|
e.g. postgres://rails:pw@127.0.0.1/events_prod. Takes precedence when set. |
DAD_DB_USER / DAD_DB_PASSWORD / DAD_DB_NAME / DAD_DB_HOST
|
↑ | config/database.yml |
Used only when DATABASE_URL is unset. DAD_DB_NAME defaults to events_prod, DAD_DB_HOST to 127.0.0.1. |
DAD_ADMIN_USER |
no | db/seeds.rb |
Defaults to admin. |
DAD_ADMIN_PASSWORD |
yes to seed | db/seeds.rb |
Seeding raises rather than creating a well-known default account. |
DAD_EVENT_RETENTION_DAYS |
no | EventPartitioning |
Postgres only. Days of event history retention keeps. Default 365. 0 or negative keeps every partition. A floor, not an exact age. |
DAD_OFFLINE_STORAGE_DIR |
no | EventPartitionBackup |
Where a month's CSV archive is written before its partition is dropped. Default offline_storage under the app root, which resolves to /var/DAD/offline_storage under systemd via WorkingDirectory=/var/DAD -- no unit sets this variable. |
DAD_AR_ENCRYPTION_KEY |
yes (production) |
WindowsTarget, ActiveDirectoryConnection
|
Active Record encryption key for stored WinRM and LDAP bind passwords. 32 bytes, hex. Production refuses to boot without it rather than falling back to plaintext. See Windows Events and Active Directory Import. |
DAD_AR_ENCRYPTION_SALT |
yes (production) |
WindowsTarget, ActiveDirectoryConnection
|
Key-derivation salt for the same. Rails 8.1 requires it whenever encryption is configured. |
WINDOWS_SSL_VERIFY |
no |
WindowsPoll, AdQuery
|
none trusts an internal CA's WinRM or LDAPS certificate instead of verifying against the system trust store. Set it deliberately; the default verifies. One variable covers both, rather than a second knob for the same decision. |
DAD_SYSLOG_DEBUG |
no | syslog_collector |
Echoes every received message to the journal. Set in a drop-in when you need to confirm the collector is receiving anything. |
DAD_SYSLOG_PORT |
no | syslog_collector |
UDP port to bind. Default 514, which is privileged and needs CAP_NET_BIND_SERVICE (the unit grants it). Raising it is how the test suite runs the collector without root. Under Compose it does double duty: it sets the collector's published port and the address every container's log driver sends to, so a host already holding 514 moves both at once. |
DAD_SYSLOG_STAGING / DAD_SYSLOG_PROCESSING
|
no | syslog_collector |
Where it writes and where it rotates into. Defaults are the production paths, /var/DAD/Logs/LogStaging and /var/DAD/Logs/LogsToProcess. Only override these in a test environment: import_syslog.rb reads Logs/LogsToProcess relative to its own working directory, so pointing the collector elsewhere silently disconnects the pipeline. |
DAD_SYSLOG_ROTATE_SECS |
no | syslog_collector |
Rotation interval. Default 60. Shortened only by the integration tests; there is no reason to change it in production. |
WINDOWS_INPUT |
no | import_windows_events.rb |
The file OpenWEC's Files driver writes. Default Logs/Windows/openwec.log. Must not be under Logs/LogStaging — that directory belongs to syslog_collector, which moves every file it finds there into LogsToProcess at startup. See Windows Events. |
WINDOWS_STAGING |
no | import_windows_events.rb |
Where converted lines are buffered before publishing. Default Logs/WindowsStaging. Must differ from both WINDOWS_QUEUE and LogStaging, for the same reason. |
WINDOWS_QUEUE |
no | import_windows_events.rb |
The queue dad-importer polls. Default Logs/LogsToProcess. Override only in tests: the importer reads that path relative to its own working directory. |
WINDOWS_MAX_LINES |
no | import_windows_events.rb |
Lines per published queue file. Default 5000. |
WINDOWS_POLL_INTERVAL |
no | import_windows_events.rb |
Seconds between input scans. Default 10. |
Keep the passwords and signing key in two root-owned files, mode 0600, never
committed:
-
/etc/DAD/environment— shell form (export KEY=value) -
/etc/DAD/environment.d/dad.conf— systemd form (KEY=value), read by the three units'EnvironmentFile
They are separate because systemd has no shell. Pointing EnvironmentFile at
the export-style file makes systemd parse the literal string
export SECRET_KEY_BASE as a variable name, so every value arrives empty and
Rails boots with a nil secret_key_base. If you add a variable, add it to both.
Generate each value once and keep it. Re-drawing the signing key on every deploy invalidates nothing about login (sessions are database rows) but does invalidate every CSRF token in flight, and re-drawing the database password mid-deploy stops the app connecting at all.
Use the separate script rather than editing the files by hand:
sudo bin/rotate_secret_key_baseIt writes a new key into both env files (keeping every other value), restarts
apache2 and the three dad-* services so Passenger stops holding the old value,
and leaves the previous env file at /etc/DAD/environment.pre-rotation.<UTC>.
Rotating does not log anyone out — sessions are rows in the sessions table,
not payloads signed with the key. See Access Control.
DAD_AR_ENCRYPTION_KEY and DAD_AR_ENCRYPTION_SALT encrypt the credentials
stored on Windows targets and on Active Directory connections: the WinRM
passwords, the directory bind password, and the polling credential an import
applies. Generate both once, into both env files:
bundle exec rails runner 'puts SecureRandom.hex(32); puts SecureRandom.hex(32)'Unlike SECRET_KEY_BASE, rotating these makes every stored Windows credential
unreadable — there is no re-derivation path, so each host's password, and each
directory's two passwords, have to be re-entered in the UI afterwards. They are
deliberately not in bin/rotate_secret_key_base.
Production refuses to boot without them rather than falling back to storing credentials in the clear. Development and test carry committed values, which are not secrets. See Windows Events.
Two things happen on migrate that an operator should know about:
- Migration
20261016000001convertseventsandevents_wordsto month-partitioned tables (Postgres only). - Migration
20261014000001clearsscriptandnext_runon every Job row, because scripts written for the oldevaldesign cannot run. Re-create any job you still want by choosing a registered handler. See Jobs.
If a CONCURRENTLY index build was ever interrupted, Postgres may be left with an
INVALID index that index_exists? still reports as healthy, which wedges
db:migrate. Check with:
select c.relname from pg_class c join pg_index i on i.indexrelid = c.oid where not i.indisvalid;