feat(review): FSRS-6 scheduling state and review history (#238 phase 2a) - #261
Merged
Conversation
Records real spaced-repetition memory state per term — Stability, Difficulty, due date, plus an append-only review log — using FSRS-6, the algorithm Anki uses. Deliberately additive and invisible. The legacy Leitner scoring keeps running untouched, nothing reads the new state to pick review words yet, and reading-view colours are unchanged. That makes the whole phase reversible and lets both schedulers be compared on real data before anything is retired (phase 2b). Decisions behind this, resolving the three questions #238 was gated on: * WoStatus stays MANUAL, not derived from stability. Deriving colours from FSRS imports an Anki assumption that does not hold here: in Anki every card is reviewed, whereas in LWT review is optional and reading is the primary loop. Users who never open the review page would see words they deliberately marked known drift back on their own. This also makes "colours unchanged after upgrade" true by construction. * Four grades, no 2-button mode. Hard/Easy carry the signal that makes FSRS beat SM-2; the legacy binary answer maps to Again/Good. * Hand-port, not a vendored dependency. The official fsrs-rs-php needs a hand-compiled Rust extension (disqualifying for self-hosters — we just fixed Windows CI for a *bundled* extension in #259), and the only pure-PHP package on Packagist is a single-commit v0.1 declaring PHP 8.1-8.3 against our 8.2-8.5. Fsrs6Scheduler is a port of py-fsrs v6.3.1 and carries that project's MIT notice; the rest of LWT stays public domain. Pinning the version matters: py-fsrs's unreleased main widens the short-term stability clamp from (Good, Easy) to (Hard, Good, Easy), turning a same-day Hard from a ~44% stability cut into a no-op. The reference vectors caught exactly that bug in the first draft of the port. Existing terms seed lazily on their first graded review, mapping each status to the stability that reproduces its legacy interval (1/2/9/27/71 days for statuses 1-5) with lastReview = WoStatusChanged. No bulk backfill, so a 100k-term vocabulary costs nothing at upgrade and nobody's queue floods. 98/99 are never scheduled. RecordScheduledReview runs from SubmitAnswer as a shadow write: only after the legacy update succeeded, and swallowing storage errors so a scheduling failure can never break the review the user just submitted. The migration's FK columns are int(10) unsigned, not the mediumint(8) that db/schema/baseline.sql still declares — 20251221_120000 widens words.WoID, and a mismatch fails with errno 150. The test-DB runner tolerates failed statements silently, so this presented as tables that simply never appeared. Verified: 24 reference vectors across 7 sequences to 1e-9, property tests (difficulty saturation, lapses never raise stability, Hard<Good<Easy, retrievability 0.9 after one stability period), and integration tests against real MySQL for seeding, upsert-vs-append and due counting. psalm 0 errors, phpcs PSR12 clean, 9130 tests green.
Both ran long for a changelog — rationale and implementation detail belong in the design doc and PR, not here. Cut to what a user needs to know, with a pointer to the docs page for the rest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements phase 2a of #238, following the review in this comment.
LWT now records real spaced-repetition memory state per term — Stability, Difficulty, a due date, and an append-only review log — using FSRS-6, the algorithm Anki uses.
Deliberately additive and invisible
The legacy Leitner scoring keeps running untouched, nothing reads the new state to pick review words yet, and reading-view colours are unchanged. The whole phase is reversible, and both schedulers can be compared on real data before anything is retired in 2b.
The three gating decisions
1.
WoStatusstays manual, not derived from stability. Against the original proposal. Deriving colours from FSRS imports an Anki assumption that doesn't hold here — in Anki every card is reviewed; in LWT review is optional and reading is the primary loop. Users who never open the review page would watch words they deliberately marked known drift back on their own. It also makes "colours unchanged after upgrade" true by construction rather than something to verify.2. Four grades, no 2-button mode. Hard/Easy carry the signal that makes FSRS beat SM-2. The legacy binary answer maps to Again/Good via
Rating::fromBinary(), so existing callers keep working without a second mode to maintain.3. Hand-port, not a vendored dependency.
fsrs-rs-php(official)scottlaurent/fsrsFsrs6Scheduleris a port of py-fsrs v6.3.1 and carries that project's MIT notice; the rest of LWT stays public domain.What the reference vectors caught
Pinning the version matters. py-fsrs's unreleased
mainwidens the short-term stability clamp from(Good, Easy)to(Hard, Good, Easy)— turning a same-day Hard from a ~44% stability cut into a no-op. I ported frommainfirst; the vectors failed and caught it. That is the entire argument for generating them rather than trusting a read of the source.Seeding
Existing terms seed lazily on their first graded review, mapping each status to the stability that reproduces its legacy interval (1/2/9/27/71 days for statuses 1–5) with
lastReview = WoStatusChanged. No bulk backfill — a 100k-term vocabulary costs nothing at upgrade and nobody's queue floods. 98/99 are never scheduled.RecordScheduledReviewruns fromSubmitAnsweras a shadow write: only after the legacy update succeeded, and swallowing storage errors so a scheduling failure can never break the review the user just submitted.Migration gotcha worth knowing
The FK columns are
int(10) unsigned, not themediumint(8)thatdb/schema/baseline.sqlstill declares —20251221_120000widenswords.WoID, and a mismatch fails with errno 150. The test-DB runner tolerates failed statements silently, so this presented as tables that simply never appeared rather than as an error. Documented inline in the migration.Verification
What's explicitly not here
No 4-grade UI, no queue reordering, no interval fuzzing (it only exists to spread Anki's daily load), no parameter optimiser (needs accumulated history — which this PR starts collecting). All 2b.
Also worth noting for #228:
term_schedule.TsStateuses Anki'scards.typevalues, andreview_logmaps onto Anki'srevlog, so the.apkgexporter can populatecards.data+revlogin 2b with no format change.