mdbkit 0.5.5 — fix ftdc summary crash

Fixed

mdbkit ftdc summary crashed on real diagnostic.data. Arguments to the value formatter were swapped during the 0.5.0 renderer rewrite, raising AttributeError: 'int' object has no attribute 'lower' as soon as it tried to format a metric. Every FTDC summary against a real file was affected. Reported by Jainesh Prabhu.

Added

End-to-end rendering tests for every command. The decoder was thoroughly tested and the formatting was not tested at all, which is exactly why this shipped. Each command now runs through the CLI to printed output, with assertions on the actual values rather than just the absence of an exception.

150 tests, up from 139.

Not affiliated with MongoDB, Inc.

mdbkit 0.5.4 — publish tooling works on macOS:

Maintainer tooling only. No changes to the tool itself.

Fixed

• publish.sh and publish.bat now bootstrap their own build tooling. Homebrew and system Pythons are externally managed (PEP 668), so build and twine cannot be installed into them. The scripts now create a local .venv, install the tools there once, and use it — no global installs, no direnv, no shell aliases.
• publish.sh now checks it is running under bash, since sh publish.sh on macOS runs it in POSIX mode.
• .venv/ added to .gitignore.

Includes the documentation fixes from 0.5.3: Linux install instructions before macOS, per-platform upgrade commands, and a fix for pipx upgrade silently reporting a stale version.

Not affiliated with MongoDB, Inc.

mdbkit 0.5.0 — why did it die, and what is the server actually doing?

Thanks to a dear friend (S.H.R) for the testing and the feature ideas behind most of this release.

Upgrade

pip install --upgrade mdbkit

Added

mdbkit oslog — what the operating system saw

mdbkit oslog /var/log/syslog
mdbkit triage mongod.log --oslog /var/log/syslog

A mongod log cannot record its own OOM kill — the process is gone before it can write anything. This reads the system log for the events that end database processes: OOM kills, file-descriptor limits, segmentation faults, filesystem and I/O errors, read-only remounts, conntrack exhaustion and systemd service exits. Used with triage, an unexplained restart is matched to the kernel line that caused it.

On journald systems mdbkit prints the journalctl command to capture instead of running it, keeping the guarantee that analysis commands never start a process.

mdbkit serverstatus — digest a serverStatus dump

mdbkit export-script serverstatus > export_serverstatus.js
mongosh --quiet --host HOST --eval "$(cat export_serverstatus.js)" > status.json
mdbkit serverstatus status.json

Checks concurrency tickets, WiredTiger cache against the 80% and 95% eviction thresholds, dirty cache, application-thread eviction, connection headroom, queued readers and writers, assertions, flow control, replication role and memory. Reads tickets from either wiredTiger.concurrentTransactions (pre-7.0) or queues.execution (7.0+).

Two dumps give true rates. Nearly everything in serverStatus is cumulative since startup, so one dump only yields lifetime averages:

mdbkit serverstatus before.json --after after.json

Same server, same counter: one dump reported 2,127 queries/sec; two dumps a minute apart reported 12,367/sec.

Scheduled monitoring

--exit-code returns 2 on CRIT, 1 on WARN, 0 otherwise, and --only CRIT,WARN trims the noise. The README has a worked cron wrapper. mdbkit still has no network code — it produces the verdict and your script sends it.

Disk performance from FTDC

Per-device utilisation, operation counts and average wait, derived from systemMetrics.disks.* already present in diagnostic.data. No iostat, no subprocess.

Fixed

• A log copied from another host is no longer correlated with local metrics. triage auto-discovered this machine's diagnostic.data and presented it alongside a different server's log — confident, wrong output, and a full-history decode that looked like a hang. It now compares hostnames and explains why it skipped, with a hard cap on the decode window.
• Multiple mongod instances are no longer guessed between. dbPath discovery took whichever process /proc listed first. It now lists every instance with its port and dbPath and asks for --dbpath.
• repl.lagSeconds was not a lag. It was mapped to members.0.optimeDate, an absolute epoch-milliseconds value, producing numbers like 1785398555000. Replaced with real lag derived from the spread across member optimes.
• Cumulative counters are no longer shown as min/avg/max. Counters report change across the window plus per-second rate; min/avg/max is kept for gauges where it means something.

139 tests, up from 108.

Not affiliated with MongoDB, Inc.

mdbkit 0.4.0 — did the index actually help?

Upgrade

pip install --upgrade mdbkit

Added

mdbkit compare — the follow-up to advise

mdbkit compare before.log --after after.log

You ran advise, created an index, and waited a day. This diffs query shapes between two logs and tells you whether it worked:

slow-query time DOWN 32%  (6.0m -> 4.1m across compared shapes)
shapes: 1 improved, 0 regressed, 0 new, 0 gone, 4 unchanged

IMPROVED
  shop.orders {createdAt:gt, status:eq} sort:{createdAt:-1}
    mean 1.7s -> 33ms (-98%)   scan 2976:1 -> 1:1  [COLLSCAN -> index, in-memory sort gone]

Shapes seen fewer than --min-count times are ignored, so a quiet log does not read as a regression.

Rotated logs, as one stream

Every command that reads a log now accepts multiple files or a shell glob, in filename order:

mdbkit queries "mongod.log*"
mdbkit triage /var/log/mongodb/mongod.log.*.gz
mdbkit loginfo mongod.log.1 mongod.log

mdbkit queries --shape N

Expands one row of the table into full detail: every plan observed, which client applications ran it, examined/returned counts, and the scan ratio.

Changed

• README restructured. Trust and safety now come before install, with commands to verify the claims yourself. The quickstart is self-contained, and the lab section has worked examples for --standalone, logs, parallel labs, and a complete before/after experiment.
• Testing playbook rebuilt around mdbkit lab — the scenarios that graduate the remaining beta detectors no longer need mlaunch or a separate cluster.
• publish.bat / publish.sh included for maintainers.

106 tests, up from 95.

Not affiliated with MongoDB, Inc.

mdbkit 0.3.0 — try it in 30 seconds, with or without a cluster

The friction release. Evaluating a MongoDB log analyser used to require a MongoDB with interesting problems in it. Now it requires neither.

Upgrade

pip install --upgrade mdbkit      # or: pipx upgrade mdbkit

Added

mdbkit demo — a realistic log, no MongoDB required

pip install mdbkit
mdbkit demo --with-extras -o demo.log
mdbkit triage demo.log --window 0 --no-sysprobe

Generates a structured logv2 stream containing a genuine incident: an index build, a connection storm from a single client, a replica set election, plan-executor errors, a slow WiredTiger checkpoint, and a burst of unindexed queries afterwards.

• --scenario incident | healthy | mixed
• --seed N — deterministic: the same seed produces a byte-identical log, so a demo behaves the same every time
• --with-extras also writes indexes.json, schema.json and explain.json, so advise and explain work immediately

mdbkit lab — a disposable local MongoDB

mdbkit lab start                    # 3-node replica set on 28110-28112
mdbkit lab seed                     # sample data + a mixed workload
mdbkit queries $(mdbkit lab logs | head -1)
mdbkit lab destroy --yes

seed runs indexed point lookups alongside deliberately unindexed queries, so the log immediately contains something worth analysing. Requires mongod on your PATH; Linux and macOS.

This is the only command in mdbkit that starts external processes, and it is tightly bounded: it runs only mongod/mongosh, binds to 127.0.0.1, defaults to port 28110 (far from 27017 so a lab can never be mistaken for a real deployment), and refuses to use or delete any directory it did not create. Every analysis command remains offline, read-only and subprocess-free — see SECURITY.md, which now states the scope explicitly rather than implying a blanket guarantee.

mdbkit connections now shows who authenticated

Per-IP rows gained first-seen, last-seen and appName, and there is a new authenticated-users table: successful and failed logins per principal, the auth database, when each account last authenticated, and which IPs it came from. Failed authentications are called out separately.

This answers the question that starts most access incidents — did that account connect, from where, and when last? — without connecting to anything.

Cluster health in mdbkit triage

Triage now leads with a cluster health finding derived entirely from the log: this node's replica set role, every peer's last known state, heartbeat failures, and whether the node is still serving connections. It escalates to CRIT on a logged shutdown or an unreachable member, and to WARN on heartbeat errors or a recent election.

It reports what the node last said about itself and its peers — the honest limit of an offline tool, and enough to answer "is this node serving, and what does it think of the others?"

diagnostic.data is found automatically

diagnostic.data lives inside the dbPath, which mdbkit already discovers, so triage now picks up FTDC metrics with no extra flags. --ftdc remains for pointing at a directory copied from another host.

Also

• .gitattributes normalises line endings for Windows contributors.

88 tests, up from 67 — including tests that the lab refuses to delete a directory it did not create.

Not affiliated with MongoDB, Inc.

mdbkit 0.2.1 — FTDC in seconds, not minutes

A performance and reliability release, driven by testing on real staging and production clusters. If you tried the FTDC features in 0.2.0 on a large diagnostic.data, this is an important upgrade.

Upgrade

pip install --upgrade mdbkit      # or: pipx upgrade mdbkit

Fixed

FTDC decoding: 25+ minutes → under 2 seconds

On a ~250 MB diagnostic.data, mdbkit ftdc and mdbkit triage --ftdc could run for 25–30 minutes at 100% CPU. That made them unusable during an actual incident — the opposite of the point. The decoder was rebuilding all ~2,200 metrics in every chunk when the summary needs about 20, and it was holding millions of samples in memory.

Three changes fix it:

• Selective decoding — only the metrics actually needed are materialised; the rest of the stream is skipped without the per-value arithmetic.
• Streaming statistics — min/avg/max/rate are computed incrementally instead of accumulating full history in memory.
• Time-window pre-filter — each chunk's timestamp is read before decompression, so out-of-window data costs almost nothing.

ftdc commands now default to the last 4 hours and print progress. Use --last 90m|4h|2d, --from/--to, or --all for the full history.

	before	after (250 MB)
triage --ftdc	25–30 min	~0.2s
ftdc summary (default window)	25–30 min	~2s

Crash in filter --as-explain

Rebuilding a query from certain batched update/delete log lines raised TypeError: 'int' object is not subscriptable. All command reconstruction is now type-guarded, and a single malformed log line can never abort a run over a large file — it is skipped instead.

Clearer FTDC message in triage

When the log and diagnostic.data cover different time ranges, triage now explains that directly instead of reporting "no metric chunks decoded."

Documentation

• The --report feature now has its own README section with sample Markdown output and a note on the self-contained HTML.
• FTDC windowing (--last / --all) and the performance behaviour are documented.

67 tests (up from 60). Performance validated against a 250 MB synthetic diagnostic.data matching real-world metric counts.

Not affiliated with MongoDB, Inc.

mdbkit 0.2.0 — FTDC decoding, incident triage with real metrics, and shareable reports

The incident release. mdbkit can now read MongoDB's own always-on metrics recorder, rebuild the queries behind slow operations, and export a report you can attach to a ticket — all offline, still with zero dependencies.

Upgrade

pip install --upgrade mdbkit      # or: pipx upgrade mdbkit

Added

FTDC decoding — mdbkit ftdc

Every mongod continuously writes diagnostic.data: CPU, memory, WiredTiger cache, connections, queues and operation counters, recorded whether or not you run any monitoring. It is compressed BSON that almost nobody reads. mdbkit now decodes it offline.

mdbkit ftdc summary /var/lib/mongodb/diagnostic.data
mdbkit ftdc timeline diagnostic.data --metric conns.current --step 300
mdbkit ftdc export diagnostic.data > metrics.csv

No agent, no database connection, no network. The directory can be copied off the host — it holds metrics only, never document contents.

Triage now includes system metrics — --ftdc

mdbkit triage mongod.log --ftdc /var/lib/mongodb/diagnostic.data

Log findings and real metrics in one view: cache pressure, connection saturation, read/write queue depth, mongod resident memory, CPU user/iowait, and throughput — alongside elections, connection storms, hot collections and index builds.

Query reconstruction — filter --as-explain

Getting an explain plan used to mean reading a raw log line and hand-writing the query. Now:

mdbkit filter mongod.log --ns shop.orders --slow 500 --last 3 --as-explain

rebuilds the runnable mongosh command — filter, projection, sort, limit and hint included. Add --explain-script for a ready-to-run .js file whose output feeds straight into mdbkit explain. Rebuilt commands contain real values from your log, so treat them as sensitive.

Shareable reports — --report

mdbkit triage mongod.log --report incident.html
mdbkit queries mongod.log --report slow-queries.md

Self-contained HTML (inline CSS, no JavaScript, no external assets — opens on an air-gapped machine) or Markdown for tickets and post-incident reviews.

Fixed

• explain accepts legacy shell output. NumberLong(...), ISODate(...), ObjectId(...) from the old mongo shell or Compass are unwrapped automatically instead of rejected.
• Misleading caveat removed. Analyzing a single explain always meant "shape seen only 1 time" — true by construction and useless. It now points you at the frequency check that actually matters.
• Unit-aware formatting. KB/MB/ms metrics are no longer formatted as if they were bytes.
• Batched writes logged at COMMAND level carry no per-operation predicate; --as-explain skips them rather than emitting an empty filter.

Documentation

• Complete reference for ftdc, --as-explain and --report.
• "Triage" is now explained in plain language for readers who don't know the term.
• A full worked explain example, from capturing the plan to analyzing it.
• Copy-paste placeholders use underscores (your_db_host) so a double-click selects the whole word.
• Severity filter examples now cover errors, fatals and warnings.

60 tests (up from 38). The FTDC codec is validated against the worked example in MongoDB's own format specification and round-tripped through an independent encoder.

Not affiliated with MongoDB, Inc.

mdbkit 0.1.1 — Python 3.8 support, noise filtering, and a full CLI reference

A quality release driven by real-world testing on staging and production clusters, plus QA against public MongoDB replica-set logs.

Upgrade

pip install --upgrade mdbkit      # or: pipx upgrade mdbkit

Fixed

• Python 3.8 support. mdbkit previously required 3.9+, which made it uninstallable on Ubuntu 20.04, Debian, and Amazon Linux 2 — the exact hosts most self-managed MongoDB deployments run on. No workaround needed now.
• Internal namespaces no longer drown the output. On real replica-set logs, local.oplog.rs tailing, config.system.indexBuilds, and other server housekeeping dominated every report — one test log showed 3,618 internal operations burying 256 real ones, and reported a fabricated "40% COLLSCAN" alarm that was entirely oplog tailing. admin, config, local, and .system. namespaces are now excluded by default; use --include-system to see them.
• Timestamps are consistent. Peak times were printed in UTC while the window header used the log's local timezone. All times now use the log's own timezone.
• Timezone-aware filtering. --from/--to accept offsets (+04:00), Z, naive timestamps, and date-only values, and no longer crash when comparing a naive bound against offset-carrying log timestamps.
• Fewer false alarms. Absolute thresholds stop trivial findings being flagged as warnings (a collection using 62% of 0.1s total is not an incident), and system index builds at startup are reported as INFO rather than WARN.

Added

• mdbkit triage defaults to the last 60 minutes of log time — triage is for incidents happening now. --window N to widen, --window 0 for the whole file.
• New triage detectors: slow-query volume with the peak minute, COLLSCAN share of slow operations, index-build activity, connection storms with peak timing and top source IPs, the hot collection's top three query shapes inline, and the running mongod's PID/RSS/uptime.
• Automatic dbPath discovery — from the log's startup line, the running mongod process, /etc/mongod.conf, or common defaults — so the disk check works even when the current log has no startup event. Stdlib only (/proc, statvfs); no shell-outs.
• Output limits: filter --limit N and --last N (most recent matches — usually what you want mid-incident); advise --limit N (default 10) and --ns NAMESPACE to focus on one collection.
• Clearer queries columns: cumMs (time summed across all occurrences, not one query), docsEx, and plan showing COLLSCAN / IXSCAN{fields} / IDHACK / +SORT.
• loginfo next-step hints — the error and warning counts now tell you the command to see them.
• Large-file warnings before analyzing multi-hundred-MB logs.

Documentation

• Complete CLI reference in the README: every flag for every command, what each queries column means, and all accepted timestamp formats.
• Install instructions rewritten for real-world hosts: pipx, PEP 668 / externally-managed environments, Python 3.8 boxes, and air-gapped installs.
• Index/schema export is now documented as Step 1 before advise, with production-style mongosh connection strings including authentication.
• Upgrade instructions added.

38 tests (up from 24), validated against real MongoDB 6.0 and 7.x logs.

Not affiliated with MongoDB, Inc.

mdbkit 0.1.0 — initial release

The first release of mdbkit — an offline toolkit for MongoDB 4.4+ structured logs. A spiritual successor to mtools' log tools for the JSON log format mtools never supported.

Install

pipx install mdbkit    # or: pip install mdbkit

What's in 0.1.0

• mdbkit loginfo — log summary: server versions, restarts, connections, warnings/errors
• mdbkit queries — slow queries grouped by query shape (literals stripped), with COLLSCAN and in-memory-sort flags and docsExamined:returned ratios
• mdbkit connections — connection churn by source IP, appName, and driver
• mdbkit filter — chainable log filtering; output stays valid logv2 JSON
• mdbkit advise — deterministic candidate-index recommendations (ESR rule) with evidence, confidence levels, caveats, and validation steps — rules, not AI
• mdbkit explain — feed it a saved .explain("executionStats") and get plain-English verdicts
• mdbkit triage (beta) — one-command incident snapshot: elections/stepdowns, connection storms, hot collections, error clusters, plus local disk/memory/load probes
• mdbkit export-script — prints mongosh scripts you run yourself to export schema/indexes; mdbkit never connects to your database

Privacy by design

Zero runtime dependencies (pure Python stdlib). No network calls, no telemetry — safe to run on air-gapped production hosts. Your logs never leave your machine. Read-only, always: mdbkit never runs commands against your cluster.

Validated against real-world logs (28k+ lines, 100% parse rate) including real replica-set election traffic.

Supports MongoDB 4.4 / 5.0 / 6.0 / 7.0 / 8.0. Requires Python 3.9+.

Not affiliated with MongoDB, Inc.