Skip to content

2.4.0-rc1

Pre-release
Pre-release

Choose a tag to compare

@robrigo robrigo released this 10 Sep 21:33
· 4 commits to main since this release
Immutable release. Only release title and notes can be modified.
fd05348

Repairs the market-stats refresh, which stopped advancing behind a backlog instead of draining it, and adds an opt-in fast path for asset counts.

Upgrading

  • Image ghcr.io/atomicassets/atomicassets-api:2.4.0. The 2.4 and latest tags move to it. A deployment pinned to the 2.3 tag stays on 2.3.3 and does not take this release.
  • The migration set moves to 2.0.10, and the filler applies it on boot. 2.0.10 recreates atomicmarket_stats_markets_updates with a dedup key, a claim token and absolute autovacuum thresholds, deduplicating and compacting whatever backlog it holds. It locks only that queue, so the API server is unaffected. The cost follows the rows that survive deduplication: 5 million of them rebuild in about 16 seconds and 20 million in about 71 seconds, and both copies exist until it commits, so budget roughly twice the queue's size on disk. UPGRADING.md carries the measurements and a sizing query.
  • Stop the running filler before starting one on 2.0.10. The rebuild holds the queue lock across its copy, and a filler still processing blocks holds row locks on the same queue, so an overlap fails the version on its lock timeout and the new process retries on its next boot.
  • The migration does not drain the backlog. The filler drains it in bounded batches once the reader is near the chain head, so expect the queue to fall over the first hours rather than at boot. The burn-down itself needs no operator step.
  • Five environment variables tune the drain, all optional: ATOMICMARKET_STATS_MARKET_DRAIN_INTERVAL_S (default 60), ATOMICMARKET_STATS_MARKET_BATCH_SIZE (default 1000), ATOMICMARKET_STATS_MARKET_DRAIN_BUDGET_MS (default 50000), ATOMICMARKET_STATS_MARKET_STATEMENT_TIMEOUT_S (default 300) and ATOMICMARKET_STATS_MARKET_WORK_MEM_MB (default 256). Raise the batch size to burn a large backlog down faster.
  • A rollback to an earlier image keeps working. That image calls the recompute with no argument, which resolves through the parameter default and drains one batch every two minutes. The queue and triggers need no schema rollback.

Features

  • /v1/assets/_count can sum the atomicassets_asset_counts totals the filler already maintains instead of counting asset rows, through a new boolean enable_fast_asset_counts in the args of the atomicassets and atomicmarket namespaces. It defaults to false. Eligible filters are collection, schema and template, collection and template whitelists and blacklists, authorized collection accounts, burned state, the transferable and burnable flags, and template name match and search across both immutable and mutable template data. Every other filter, including owner, ids, data filters, bounds and the market price joins, keeps counting rows, and a filter added later does so by construction. (#203)
  • The setting is opt-in because it makes those aggregate totals authoritative for a public count. The filler's trigger and its aggregation job keep them exact for ordinary writes and for fork rollback, but a restore, an import, a trigger disabled during maintenance or a bug can leave them adrift, and a total wrong that way is served with no error. Compare the two counts on your own data before enabling it; src/scripts/recount-asset-counts.ts reports and repairs drift. (#203)

Bug fixes

  • The market-stats refresh no longer fails with 57014, canceling statement due to statement timeout, against a backlog. It ran as one unbounded statement on the runtime pool, whose 30 second statement_timeout is the only one PgBouncer transaction pooling lets through, and each cancellation rolled back the statement's own queue claim, so every following tick re-read the same backlog and atomicmarket_stats_markets stopped advancing. The recompute now claims a bounded batch per call, on the long-running pool and under a per-batch timeout, and yields to the block reader between batches. (#210)
  • atomicmarket_stats_markets_updates deduplicates its rows. A listing written repeatedly added one queue row per write, so a hot sale could hold thousands, and the table had no autovacuum tuning to reclaim them. An auction still holds a second row for its end time, which is what makes the auction resolve once that time passes. (#210)

Security

  • qs moves to 6.16.0, clearing GHSA-4mjr-xmp4-gh2g, a denial of service through an attacker-controlled isBuffer, and GHSA-x5fp-wj9c-mxmx, an array-limit bypass through bracket-key comma parsing. It parses every query string express and body-parser hand to the read endpoints. GHSA-848j-6mx2-7j84 against elliptic has no patched version published and is still open. (#211)

Other changes

  • express-rate-limit moves to 8.6.2, which keys IPv4-in-IPv6 addresses by range rather than by notation. An IPv4-mapped address written without a dotted quad shared one bucket with its whole range, and an ordinary IPv6 address ending in a dotted quad was keyed into the bucket of the unrelated IPv4 client it appeared to name. Every notation of one address now produces one key. (#191)
  • p-queue moves to 9.3.3, a patch on the queue the block reader draws through. (#151)
  • The development toolchain takes its pending updates: eslint, mocha, c8, knip, globals, the SWC compiler and register hook, and actions/setup-node in CI. None of them ships in the image. (#116, #117, #121, #123, #150, #190, #192, #193)

Commits

  • 5c163f4 docs(changelog): trim the 2.3.3 entry to what an operator acts on (#207)
  • d4e53ab docs(upgrading): require the rc cleanup before 2.0.8 rewrites the view (#209)
  • 9cb2880 chore(deps): bump actions/setup-node from 6 to 7 (#116)
  • 815d02f chore(deps-dev): bump the eslint group across 1 directory with 2 updates (#117)
  • 97dce7e chore(deps-dev): bump @swc-node/register from 1.11.1 to 1.12.1 (#123)
  • d0d9233 chore(deps): bump p-queue from 9.3.1 to 9.3.3 (#151)
  • 4e1a36f chore(deps): bump express-rate-limit from 8.5.2 to 8.6.2 (#191)
  • 5002d0d chore(deps-dev): bump globals from 17.6.0 to 17.11.0 (#192)
  • 4ec8f35 chore(deps-dev): bump c8 from 11.0.0 to 12.0.0 (#121)
  • bd6fd1f chore(deps-dev): bump knip from 5.88.1 to 6.34.0 (#190)
  • d3886d6 chore(deps-dev): bump the mocha group across 1 directory with 3 updates (#150)
  • 2416d2b docs(scripts): describe the recount connection string by role (#212)
  • 5cf64b0 chore(deps): clear the qs denial-of-service advisories (#211)
  • 91a8bab chore(deps-dev): bump @swc/core from 1.15.43 to 1.16.2 (#193)
  • 077cd95 fix(atomicmarket): bound the stats-market recompute so a backlog drains (#210)
  • 27d36e4 feat(api): enhance asset counting with fast count support (#203)
  • fd05348 chore(release): 2.3.4 (#213)

Full changelog: 2.3.3...2.4.0-rc1