Skip to content

0.2

Latest
Latest

Choose a tag to compare

View all tags
@fabriziomello fabriziomello released this 17 Aug 20:14
0.2
0a8b782

pg_stat_log v0.2

Performance-focused release of pg_stat_log. The entry storage was rewritten as an index-based chaining hash table, making the logging hot path O(1) expected even when the table is completely full.

Highlights

  • O(1) lookups at any load factor. The entries array was previously an open-addressing hash table sized exactly to max_entries, so it could reach 100% load. Once full, a message with an untracked signature found no empty slot to terminate the probe and scanned all max_entries slots before being dropped — an O(N) walk under the exclusive LWLock, exactly during the diverse log storms the extension is meant to observe. Entries are now stored in a separate-chaining hash table linked by array indices, so lookups, inserts, and drops walk a single bucket chain whose average length is the load factor — O(1) expected even at saturation.

Changes

  • Replace the per-slot used flag with an int32 next chain link (slot size unchanged) and append a bucket-heads array after the entries in the same flat stats block.
  • Chain links are array indices, not pointers, so the block remains snapshot-safe (memcpy) and persistence-safe (verbatim fwrite/fread), as required by the fixed-amount Custom Cumulative Stats API.
  • Memory cost: one int32 per entry for the bucket-heads array (~1.15x the raw entry data; still a hard cap fixed at startup).
  • Behavior is unchanged: already-tracked signatures keep counting after saturation, and only genuinely new signatures are dropped (counted in n_dropped).

Documentation

  • New README section "Design notes: why a custom hash table" explaining why PostgreSQL's standard hash tables (dynahash, simplehash, dshash) cannot be used under the fixed-amount custom stats constraints, with a matching summary in the PgStatLog struct comment.

Tests

  • New TAP check verifying that an already-tracked signature continues to increment when the table is full.

Upgrade notes

  • No SQL-level changes — the extension script is unchanged, so no ALTER EXTENSION ... UPDATE is needed. Install the new shared library and restart the server.
  • The shared stats block layout changed, so counters persisted by 0.1 are discarded on the first startup with 0.2 (statistics start fresh, as after a crash recovery).

Requirements

  • PostgreSQL 18 or later
  • Must be loaded via shared_preload_libraries
Assets 2
Loading