Skip to content

docs(issues): add implementation specs for EPIC #1525 overhaul persistence#1702

Merged
josecelano merged 3 commits intotorrust:developfrom
josecelano:1525-persistence-plan
Apr 22, 2026
Merged

docs(issues): add implementation specs for EPIC #1525 overhaul persistence#1702
josecelano merged 3 commits intotorrust:developfrom
josecelano:1525-persistence-plan

Conversation

@josecelano
Copy link
Copy Markdown
Member

@josecelano josecelano commented Apr 21, 2026

Summary

This PR adds the planning documentation for EPIC #1525 — Overhaul Persistence: an EPIC-level spec and eight sub-issue implementation specs that together define how the persistence layer will be completely overhauled.

The specs have been written, cross-reviewed for consistency, and linter-clean (linter all exits 0). They are intended to be merged first so that the implementation sub-issues can be worked on and reviewed against a stable spec document.

Note: This PR only merges the specs. Issue #1525 should remain open until all implementation sub-issues (#1525-01 through #1525-08) are completed and merged.


Specs included

File Scope
docs/issues/1525-overhaul-persistence.md EPIC – delivery strategy, sub-issue ordering, acceptance criteria
docs/issues/1525-01-persistence-test-coverage.md DB compatibility matrix runner (Bash, SQLite + MySQL + PostgreSQL)
docs/issues/1525-02-qbittorrent-e2e.md qBittorrent E2E Rust runner with DockerCompose wrapper
docs/issues/1525-03-persistence-benchmarking.md Persistence benchmark runner (--before-image/--after-image CLI)
docs/issues/1525-04-split-persistence-traits.md Split monolithic Database into 4 narrow traits + aggregate supertrait
docs/issues/1525-05-migrate-sqlite-and-mysql-to-sqlx.md Green-parallel migration of both drivers to sqlx
docs/issues/1525-06-introduce-schema-migrations.md Replace raw DDL with sqlx::migrate!(), legacy bootstrap for pre-existing DBs
docs/issues/1525-07-align-rust-and-db-types.md Widen NumberOfDownloads u32→u64, INTEGER→BIGINT migrations
docs/issues/1525-08-add-postgresql-driver.md Add full PostgreSQL driver, testcontainers tests, user-facing docs

Key design decisions documented

  • Trait split order: traits split in Overhaul persistence #1525-04 (sync), made async in Overhaul persistence #1525-05 — no big-bang rewrite.
  • Migration history alignment: all backends share identical migration filenames/timestamps for consistent sqlx history tracking.
  • Legacy bootstrap: pre-existing SQLite and MySQL databases get their history aligned via direct INSERT INTO _sqlx_migrations with precondition guards — not a public apply_fake() API.
  • DROP TABLE IF EXISTS: chosen consistently for all five table drops across all backends.
  • Concurrent benchmark workloads: std::thread::spawn + blocking reqwest (not rayon, which is wrong for I/O-bound work).
  • DockerCompose wrapper: shared at src/console/ci/compose.rs, reused by both the qBittorrent runner and the benchmark runner.
  • PostgreSQL database pre-creation: documented requirement — CREATE DATABASE torrust_tracker must exist before connection.

Checklist

  • linter all exits 0
  • cargo machete — no unused dependencies
  • Commit is GPG-signed
  • All 15 issues from cross-spec review addressed
  • project-words.txt updated with new technical terms (dbname, isready, VARCHAR)

… persistence

Add the EPIC planning doc and eight sub-issue specs covering the full
persistence overhaul:

- torrust#1525:    EPIC – Overhaul Persistence (delivery strategy + ordering)
- torrust#1525-01: Persistence test coverage (DB compatibility matrix)
- torrust#1525-02: qBittorrent E2E test runner
- torrust#1525-03: Persistence benchmarking framework
- torrust#1525-04: Split persistence traits (narrow interfaces)
- torrust#1525-05: Migrate SQLite and MySQL drivers to sqlx
- torrust#1525-07: Align Rust and DB types (u32→u64, INTEGER→BIGINT)
- torrust#1525-08: Add PostgreSQL driver

Also adds new technical terms to project-words.txt (dbname, isready,
VARCHAR) required for the cspell spell-checker to pass.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a set of planning/specification documents for EPIC #1525 (“Overhaul Persistence”) so follow-up implementation PRs can be reviewed against a stable, agreed design.

Changes:

  • Adds the EPIC plan plus sub-issue implementation specs (#1525-01 through #1525-08) under docs/issues/.
  • Updates project-words.txt with new technical terms used in the specs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
project-words.txt Adds cspell allowlist entries for new persistence/migration terminology.
docs/issues/1525-overhaul-persistence.md EPIC-level delivery strategy, ordering, and acceptance criteria for the persistence overhaul.
docs/issues/1525-01-persistence-test-coverage.md Spec for a DB compatibility matrix runner to baseline persistence behavior across DB versions.
docs/issues/1525-02-qbittorrent-e2e.md Spec for a qBittorrent-based end-to-end test harness (Docker Compose driven).
docs/issues/1525-03-persistence-benchmarking.md Spec for a before/after persistence benchmark runner and baseline artifacts.
docs/issues/1525-04-split-persistence-traits.md Spec for splitting the monolithic persistence trait into narrower context traits + aggregate supertrait.
docs/issues/1525-05-migrate-sqlite-and-mysql-to-sqlx.md Spec for migrating SQLite/MySQL to async sqlx using a “green parallel” approach.
docs/issues/1525-06-introduce-schema-migrations.md Spec for adopting sqlx::migrate!() and a legacy bootstrap path for pre-v4 DBs.
docs/issues/1525-07-align-rust-and-db-types.md Spec for widening download counters (u32→u64, INTEGER→BIGINT) via migrations and explicit encode/decode helpers.
docs/issues/1525-08-add-postgresql-driver.md Spec for adding PostgreSQL driver support, migrations, tests, and QA harness extensions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread project-words.txt
Comment thread docs/issues/1525-07-align-rust-and-db-types.md Outdated
Comment thread docs/issues/1525-06-introduce-schema-migrations.md Outdated
Comment thread docs/issues/1525-08-add-postgresql-driver.md Outdated
Comment thread docs/issues/1525-08-add-postgresql-driver.md Outdated
- project-words.txt: sort newly-appended words alphabetically
- 1525-06: fix double semicolon typo in MIGRATOR.run() snippet
- 1525-07: replace "reversible" with "tracked as a forward schema
  change" — sqlx has no down/rollback migrations
- 1525-08: fix migration-1 valid_until type (BIGINT → INTEGER to
  align with other backends; migration-4 widens to BIGINT)
- 1525-08: fix Error::migration_error argument order (e, DRIVER) →
  (DRIVER, e) to match the signature used throughout 1525-06
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.43%. Comparing base (a95af24) to head (0cc8528).
⚠️ Report is 4 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1702   +/-   ##
========================================
  Coverage    86.42%   86.43%           
========================================
  Files          288      288           
  Lines        22672    22672           
  Branches     22672    22672           
========================================
+ Hits         19594    19596    +2     
+ Misses        2843     2838    -5     
- Partials       235      238    +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Task 9 was missing two required steps:

1. share/container/entry_script_sh — the container bootstrap script
   hard-codes only sqlite3 and mysql; without a postgresql elif branch
   the container exits 1 when that driver is selected. Spec now
   includes the exact elif block and the updated error message.

2. compose.yaml — the demo compose file only had a mysql service;
   spec now adds a postgres service (postgres:16, with healthcheck,
   POSTGRES_DB env var, and a named volume) and updates the tracker's
   depends_on to include both mysql and postgres.

Also extends the overall Acceptance Criteria section with checkboxes
for entry_script_sh and compose.yaml.
@josecelano
Copy link
Copy Markdown
Member Author

ACK 0cc8528

@josecelano josecelano merged commit 4b8a21f into torrust:develop Apr 22, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants