Skip to content

Add a delta-maintained summary table tag-cloud strategy - #86

Merged
igor-alexandrov merged 3 commits into
masterfrom
claude/infallible-booth-41e511
Aug 18, 2026
Merged

Add a delta-maintained summary table tag-cloud strategy#86
igor-alexandrov merged 3 commits into
masterfrom
claude/infallible-booth-41e511

Conversation

@igor-alexandrov

Copy link
Copy Markdown
Collaborator

Summary

Adds a third tag-cloud generator strategy, metka:strategies:table, alongside view and materialized_view: a real summary table (tag_name primary key, taggings_count bigint NOT NULL) kept exact by statement-level triggers that apply per-tag deltas from transition tables.

Motivation

Benchmarks (10k posts, 5 tags each, 100-tag vocabulary, PostgreSQL 18) showed the materialized_view strategy's statement-level REFRESH MATERIALIZED VIEW CONCURRENTLY costs ~7 ms per tagged write statement at that size — creates 12x and tag-list updates 47x slower than an unmaintained table, with cost growing with table size since every refresh is a full recompute. Delta upserts from transition tables instead keep writes within benchmark noise of an unmaintained table (creates 1,484 vs 1,633 i/s; updates 7,879 vs 8,248 i/s), stay transactional and exact, and read faster than the matview.

Implementation

  • Generator modeled on materialized_view (same options, with --table-name in place of --view-name); migration seeds the table via INSERT .. SELECT UNNEST .. GROUP BY from existing rows.
  • One function + one FOR EACH STATEMENT trigger per operation, since a transition table is only registered for its own trigger:
    • INSERT: aggregate new_rows, upsert with ON CONFLICT .. DO UPDATE adding the delta.
    • UPDATE: signed UNION ALL over new_rows/old_rows, HAVING SUM(d) <> 0 to skip untouched tags, upsert, then delete rows at <= 0.
    • DELETE: subtract aggregated old_rows counts, then delete rows at <= 0.
  • Verified live against PostgreSQL 18: UNNEST of a NULL array yields no rows and array concatenation treats a NULL operand as empty (array_cat is not strict), so no explicit NULL guards are needed; a mixed workload (multi-row statements, NULL/empty arrays, no-op updates, duplicate tags within a row) leaves the table exactly matching a live UNNEST .. GROUP BY aggregation — the model test asserts this at the end.
  • README documents the trade-off: same read shape as the views, near-zero write overhead, O(tags touched) maintenance; but it is a plain table, so writes that bypass the triggers (TRUNCATE, dump restore) require a manual reseed — same ownership caveat as raw column writes.

Testing

  • bundle exec rake test: 158 runs, 433 assertions, 0 failures (PostgreSQL 18.3).
  • RuboCop and mdl clean.

🤖 Generated with Claude Code

igor-alexandrov and others added 2 commits August 18, 2026 11:44
Statement-level triggers read the transition tables and upsert per-tag
deltas into a real summary table, so maintenance is O(tags touched) per
statement instead of the materialized view strategy's full recompute on
every refresh. Verified against PostgreSQL 18: UNNEST of a NULL array
yields no rows and array concatenation treats a NULL operand as empty,
so the trigger functions need no explicit NULL guards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The suite now seeds two extra Metka tables whose aggregates are maintained
by the materialized_view and table strategies (DDL matching the generators'
output) and reports them in the cloud and write suites. A final check
verifies both aggregates still match a live UNNEST..GROUP BY aggregation
after every suite has run. Results and both READMEs refreshed from the new
run: summary reads are ~45x faster than live aggregation for either
strategy, but the matview refresh makes creates 12x and tag replacements
47x slower than bare Metka, while the delta-maintained table stays within
benchmark noise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@igor-alexandrov
igor-alexandrov force-pushed the claude/infallible-booth-41e511 branch from 21f725c to d865409 Compare August 18, 2026 15:55
@igor-alexandrov

Copy link
Copy Markdown
Collaborator Author

Rebased on master to pick up the benchmark suite from #85, and extended it to cover the tag-cloud strategies: the cloud/write suites now include Metka variants maintained by the materialized_view and table strategies (DDL matching the generators' output), and every run ends with an integrity check of both aggregates against a live UNNEST .. GROUP BY aggregation.

Fresh numbers (10k posts, PostgreSQL 18.3), bare / materialized_view / table:

Operation bare materialized_view table
Tag cloud read 209 i/s 9,082 i/s 9,337 i/s
Create post with 5 tags 1,631 i/s 133 i/s 1,524 i/s
Replace tags 8,122 i/s 173 i/s 7,696 i/s
Bulk seed 10k posts 0.21 s 0.24 s 0.17 s
Storage 2.68 MB 2.78 MB 2.75 MB

Integrity check after all suites: 0 mismatching tags for both strategies.

The README now recommends the table strategy over on-the-fly clouds and
documents the switch for existing users: the generated migration doubles as
the migration path since it backfills from existing rows, and call sites
move from Model.tag_cloud to plucking the summary table. To make that safe
under live traffic, the generated migration now locks the source table in
SHARE ROW EXCLUSIVE mode before seeding — CREATE TRIGGER takes that lock
level anyway, but taking it up front closes the window where a write
committing between the seed's snapshot and trigger creation would be seen
by neither and permanently drift the counts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@igor-alexandrov
igor-alexandrov merged commit 353cef8 into master Aug 18, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant