Logging and Diagnostics

Logging & Diagnostics

What this page covers: how the library logs — the silent-by-default model with an unsuppressible ERROR floor, the StorageLogConfig presets and per-topic controls, the privacy flags, where lines go (StorageLogSinks.auto() → host sink → SLF4J → no-op), the everydatabase.log.level system property, and how to construct a storage with logging pre-configured.

📌 Note — the library is silent by default. Routine operations emit nothing under the factory default (WARN); failures always emit. Everything in between is opt-in, per topic, editable at runtime.

---

The 30-second version

import br.com.finalcraft.everydatabase.log.*;
import br.com.finalcraft.everydatabase.modules.sql.SqlStorage;

// Construct a storage that already watches index work and migrations, with writes muted.
// Every backend has a (config, logConfig) constructor:
StorageLogConfig logCfg = StorageLogConfig.defaults()        // WARN: routine silent, failures visible
        .level(StorageLogTopic.INDEX,     StorageLogLevel.INFO)
        .level(StorageLogTopic.MIGRATION, StorageLogLevel.INFO)
        .mute(StorageLogTopic.WRITE);

SqlStorage sql = new SqlStorage(sqlConfig, logCfg);
sql.init().join();

// The config is LIVE — edit it at runtime and every repository reacts immediately:
sql.getStorageLogConfig()
   .level(StorageLogTopic.WRITE, StorageLogLevel.DEBUG)      // temporarily debug saves
   .includeKeys(true);                                       // opt-in: show entity keys

No sink wiring needed: out of the box, lines route to SLF4J if it's on the classpath, otherwise nowhere.

---

Silent by default, with an ERROR floor

There are six levels, least to most verbose:

OFF(0) < ERROR(1) < WARN(2) < INFO(3) < DEBUG(4) < TRACE(5)

The factory default threshold is WARN, so INFO/DEBUG/TRACE events stay silent and only warnings and failures surface. The key invariant: ERROR always emits — even on a topic set to OFF. No configuration can silence a failure. That's the "floor": you can dial routine chatter up or down freely, but errors are guaranteed visible.

Level	When it fires
ERROR	I/O errors, codec failures, any exception that propagates to the caller. Never suppressible.
WARN	Degraded-but-survived: a corrupt row skipped, an optimistic-lock conflict, an odd backfill result.
INFO	Milestones: init/close, index created/dropped, migration applied, batch write done.
DEBUG	Per-op detail: individual save/delete, a query with its result count, tx begin/commit, progress ticks.
TRACE	Maximum: per-entity find/exists, low-level internals. Low-traffic diagnostics only.

---

StorageLogConfig — presets and controls

Start from a preset, then refine. The config is live and thread-safe: one per Storage, shared by reference with all of that storage's repositories, so edits take effect immediately without restart.

Presets

Preset	Global threshold	Use
StorageLogConfig.defaults()	WARN	Production default; routine silent, failures visible. Honors the system property (below).
StorageLogConfig.silent()	OFF	Only the ERROR floor remains.
StorageLogConfig.verbose()	DEBUG	Most ops visible (saves, deletes, queries, progress).
StorageLogConfig.trace()	TRACE	Maximum verbosity. Low-traffic only.

Per-topic level overrides

A topic without an explicit override falls back to the global default. Override per topic to mix silence and detail:

StorageLogConfig cfg = StorageLogConfig.defaults()                // WARN globally
        .level(StorageLogTopic.INDEX,     StorageLogLevel.INFO)    // see index work
        .level(StorageLogTopic.MIGRATION, StorageLogLevel.INFO)    // see migrations
        .mute(StorageLogTopic.READ)                                // silence reads (ERROR floor stays)
        .mute(StorageLogTopic.QUERY);                              // silence queries

• level(topic, level) — set a specific threshold for one topic.
• mute(topic) — shorthand for setting the topic to ERROR (only the floor remains).
• reset(topic) — drop the override, falling back to the global default.
• defaultLevel(level) — change the global default itself.

📌 Note — mute(topic) does not mean "no output ever." It mutes to ERROR, so failures on that topic still emit. There is no way to silence an error; that's the floor by design.

The topics

Topic	Covers
LIFECYCLE	init, close, pool opened/closed; also health() results.
SCHEMA	Creating tables/collections/directories, adding/dropping _idx_* columns.
INDEX	Creating/dropping indexes, reconciling declared vs. persisted, backfilling new index columns (with progress).
MIGRATION	Pending check, per-migration apply/skip, completion summary.
WRITE	save (single upsert) and saveAll (batch).
DELETE	delete (single or batch).
READ	find, findMany, exists, count, all.
QUERY	findBy and query with conditions — separate from READ so query diagnostics can be toggled alone.
TRANSACTION	inTransaction begin / commit / rollback.
TRANSFER	StorageTransfer begin, per-collection progress, completion.
HEALTH	health() checks — a splittable subset of LIFECYCLE for frequent polls.

💡 Tip — READ and QUERY are deliberately separate topics. On a read-heavy service you can keep READ muted but turn QUERY to DEBUG to watch which indexed queries run and how many rows they return, without drowning in find chatter.

Privacy flags (all default false)

Log lines carry counts, durations, collection names, and index/migration metadata — never entity content — unless you opt in. These are local-debugging switches, not for production with sensitive data:

cfg.includeKeys(true)          // entity key strings (capped at maxKeysListed, default 10)
   .includeValues(true)        // truncated toString() of a saved entity — single-entity SAVE only
   .includeQueryValues(true);  // literal query filter values (paths/operators are shown either way)

• includeKeys(true) adds entity identity (key strings only), capped by maxKeysListed(int).
• includeValues(true) adds entity content (a truncated toString()), capped by maxValueLength(int), and only on single-entity saves — batch summaries never carry per-entity values.
• includeQueryValues(true) adds literal filter values to QUERY lines (field paths and operators are always shown; the values are the opt-in part).

⚠️ Gotcha — includeValues(true) logs entity field content. Enable it for local diagnostics only; never in production where entities may hold sensitive data. includeKeys is the lighter switch (identity only) when you just need to know which entities were touched.

Progress reporting

Long-running operations (index backfill, migrations) can emit throttled progress ticks. Tune with one call: progress(enabled, stepPct, throttleMs, minTotal) — a tick fires when both the percent-step and the time-throttle thresholds are met (or on completion), and only for operations at or above minTotal rows. Defaults: enabled, every 10%, at most one per second, minimum 500 rows.

---

Constructing a storage with logging

Two paths, both shown above:

1. Pre-configured at construction — every backend has a (config, logConfig) constructor (InMemoryStorage takes just the StorageLogConfig):

   SqlStorage        sql    = new SqlStorage(sqlConfig, logCfg);
   PostgreSqlStorage pg     = new PostgreSqlStorage(sqlConfig, logCfg);
   H2SqlStorage      h2     = new H2SqlStorage(sqlConfig, logCfg);
   MongoStorage      mongo  = new MongoStorage(mongoConfig, logCfg);
   LocalFileStorage  files  = new LocalFileStorage(localFileConfig, logCfg);
   InMemoryStorage   mem    = new InMemoryStorage(logCfg);

2. Edited at runtime — the Storages typed factories (createSQL, createMongo, …) build a storage with StorageLogConfig.defaults(); reach the live config afterwards:

   SqlStorage sql = Storages.createSQL(sqlConfig);
   sql.getStorageLogConfig().level(StorageLogTopic.WRITE, StorageLogLevel.DEBUG);   // takes effect at once
   // or swap the whole config:
   sql.setStorageLogConfig(StorageLogConfig.verbose());

getStorageLogConfig() returns the same object the repositories read, so in-place edits are picked up immediately. setStorageLogConfig(...) replaces it wholesale (the dispatcher re-reads on every emit).

---

Where the lines go — StorageLogSinks

A StorageLogSink is the destination. The default config uses StorageLogSinks.auto(), which resolves its target per event, in this order:

1. A host sink installed via StorageLogSinks.installDefault(...), if any.
2. SLF4J, if org.slf4j.LoggerFactory is on the classpath (detected reflectively — no NoClassDefFoundError when absent). Loggers are named everydatabase.<topic>.
3. No-op (silent) otherwise.

So the library never requires a logging framework: with SLF4J present it just works; without it, it's quiet. A host application installs its own bridge once, globally:

// e.g. a Bukkit plugin routing storage logs to its own logger:
StorageLogSinks.installDefault(event -> {
    if (event.error() != null)
        plugin.getLogger().log(java.util.logging.Level.WARNING, event.format(), event.error());
    else
        plugin.getLogger().info(event.format());
});

installDefault(...) replaces any previously installed host sink; uninstallDefault() removes it (falling back to SLF4J/no-op); getInstalledDefault() returns the current one.

Other ready-made sinks for tests or custom routing: noop(), stdout() (one line per event to System.out, with a stack trace for errors), consumer(Consumer<String>) (formatted line), structured(Consumer<StorageLogEvent>) (the raw event — use this to access event.error()), and tee(a, b) (forward to two sinks). Set one per storage with cfg.sink(mySink).

⚠️ Gotcha — installDefault(...) is global and static, not per-storage. It's the host-wide bridge. For a single storage's destination, use cfg.sink(...) instead.

📌 Note — slf4j-api is an optional dependency: it's compileOnly, probed reflectively at runtime, and no-ops when absent — so the library never drags a logging framework into your build. See Distribution Flavors.

---

event.format() — the rendered line

Each StorageLogEvent renders to a single human-readable line via format() (also its toString()). The event also exposes structured fields if you want to route programmatically: backend(), op(), level(), topic(), collection(), affected(), total(), percent(), durationMs(), keys(), value(), detail(), and error().

Sample formatted lines:

[storage:sql]   INDEX_RECONCILE player_data created=2 dropped=1 backfilled=1200 in 340ms
[storage:sql]   SAVE_BATCH      player_data entities=5000 in 1200ms (4166/s)
[storage:mongo] INDEX_CREATE    player_data field=location.world order=ASC
[storage:sql]   QUERY           player_data conditions=[level RANGE, world EQ] results=37 in 8ms
[storage:sql]   SAVE            player_data FAILED - SQL save failed

💡 Tip — a throwing sink never breaks a storage operation; sink exceptions are swallowed. You can bridge to a flaky destination without risking the data path.

---

Quick verbosity for tests / CI

StorageLogConfig.defaults() honors the system property everydatabase.log.level — set it to raise verbosity without touching application code (invalid values are ignored, keeping the WARN default):

-Deverydatabase.log.level=info     # lifecycle, index, migration, batch summaries
-Deverydatabase.log.level=debug    # + saves, deletes, queries, progress ticks
-Deverydatabase.log.level=trace    # maximum verbosity

📌 Note — this only affects storages created from a defaults()-derived config (the factory path). A hand-built silent()/verbose()/trace() config, or any explicit defaultLevel(...), ignores the property — you asked for a specific level, you get it. In production, leave the property unset.

---

See also

• Architecture Overview — where logging sits relative to Storage and Repository.
• Moving Data Between Backends — the TRANSFER topic and watching a transfer's progress.
• Indexing & Queries — what the INDEX and QUERY topics report.
• Schema Migrations — what the MIGRATION topic reports.
• Distribution Flavors — SLF4J is optional and compileOnly; how the flavors handle it.
• Concurrency & Threading — events fire on the async executor; sinks must be thread-safe.
• Gotchas & Pitfalls — the ERROR floor, mute ≠ silent, installDefault is global.