Add a delta-maintained summary table tag-cloud strategy - #86
Merged
Conversation
Statement-level triggers read the transition tables and upsert per-tag deltas into a real summary table, so maintenance is O(tags touched) per statement instead of the materialized view strategy's full recompute on every refresh. Verified against PostgreSQL 18: UNNEST of a NULL array yields no rows and array concatenation treats a NULL operand as empty, so the trigger functions need no explicit NULL guards. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The suite now seeds two extra Metka tables whose aggregates are maintained by the materialized_view and table strategies (DDL matching the generators' output) and reports them in the cloud and write suites. A final check verifies both aggregates still match a live UNNEST..GROUP BY aggregation after every suite has run. Results and both READMEs refreshed from the new run: summary reads are ~45x faster than live aggregation for either strategy, but the matview refresh makes creates 12x and tag replacements 47x slower than bare Metka, while the delta-maintained table stays within benchmark noise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
igor-alexandrov
force-pushed
the
claude/infallible-booth-41e511
branch
from
August 18, 2026 15:55
21f725c to
d865409
Compare
Collaborator
Author
|
Rebased on master to pick up the benchmark suite from #85, and extended it to cover the tag-cloud strategies: the cloud/write suites now include Metka variants maintained by the Fresh numbers (10k posts, PostgreSQL 18.3), bare / materialized_view / table:
Integrity check after all suites: 0 mismatching tags for both strategies. |
The README now recommends the table strategy over on-the-fly clouds and documents the switch for existing users: the generated migration doubles as the migration path since it backfills from existing rows, and call sites move from Model.tag_cloud to plucking the summary table. To make that safe under live traffic, the generated migration now locks the source table in SHARE ROW EXCLUSIVE mode before seeding — CREATE TRIGGER takes that lock level anyway, but taking it up front closes the window where a write committing between the seed's snapshot and trigger creation would be seen by neither and permanently drift the counts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 18, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a third tag-cloud generator strategy,
metka:strategies:table, alongsideviewandmaterialized_view: a real summary table (tag_nameprimary key,taggings_count bigint NOT NULL) kept exact by statement-level triggers that apply per-tag deltas from transition tables.Motivation
Benchmarks (10k posts, 5 tags each, 100-tag vocabulary, PostgreSQL 18) showed the
materialized_viewstrategy's statement-levelREFRESH MATERIALIZED VIEW CONCURRENTLYcosts ~7 ms per tagged write statement at that size — creates 12x and tag-list updates 47x slower than an unmaintained table, with cost growing with table size since every refresh is a full recompute. Delta upserts from transition tables instead keep writes within benchmark noise of an unmaintained table (creates 1,484 vs 1,633 i/s; updates 7,879 vs 8,248 i/s), stay transactional and exact, and read faster than the matview.Implementation
materialized_view(same options, with--table-namein place of--view-name); migration seeds the table viaINSERT .. SELECT UNNEST .. GROUP BYfrom existing rows.FOR EACH STATEMENTtrigger per operation, since a transition table is only registered for its own trigger:new_rows, upsert withON CONFLICT .. DO UPDATEadding the delta.UNION ALLovernew_rows/old_rows,HAVING SUM(d) <> 0to skip untouched tags, upsert, then delete rows at<= 0.old_rowscounts, then delete rows at<= 0.UNNESTof a NULL array yields no rows and array concatenation treats a NULL operand as empty (array_catis not strict), so no explicit NULL guards are needed; a mixed workload (multi-row statements, NULL/empty arrays, no-op updates, duplicate tags within a row) leaves the table exactly matching a liveUNNEST .. GROUP BYaggregation — the model test asserts this at the end.TRUNCATE, dump restore) require a manual reseed — same ownership caveat as raw column writes.Testing
bundle exec rake test: 158 runs, 433 assertions, 0 failures (PostgreSQL 18.3).🤖 Generated with Claude Code