LWT 3.4.0: Self-repairing database upgrades, shell-free books
Two database fixes make this release worth taking for anyone upgrading: migrations that fail are no longer written down as successful, and an upgrade no longer strips the database of its foreign keys. Both were found chasing a reported crash where importing an EPUB, editing a language or importing a dictionary all failed with Table 'books' doesn't exist.
Fixed
A migration that failed was recorded as applied, so the schema silently stayed broken (#247, #271). The runner logged each failing statement and wrote the migration down as applied anyway, so it was never retried and the gap only surfaced much later as a 500 in whatever feature needed the missing table.
That is how installs ended up without books and local_dictionaries. Both declare a foreign key on languages(LgID) typed INT(11) UNSIGNED, and where languages.LgID was still tinyint(3) — the widening in 20251221_120000_add_inter_table_foreign_keys.sql having never taken effect — InnoDB refused the whole CREATE TABLE with errno 150, even under FOREIGN_KEY_CHECKS = 0.
The runner now records the real outcome (status, attempts and error on _migrations) and retries a failure when a later upgrade may have fixed its prerequisite. Reference columns are realigned with the key they point at, on the widest member of each family, so a column is only ever widened and no value can be truncated. A repair migration recreates the tables and columns the failed migrations never created, so an affected database mends itself on first boot, with no manual SQL. Anything still failing is listed on the admin Server Data page instead of only in error_log.
Upgrading dropped every foreign key and only put some back (#272). Migrations run with the constraints cleared, because ALTER TABLE MODIFY is refused on a column one points at, but only pending migrations recreate theirs — and the migrations that created the rest were applied long ago. A 3.3.0 database upgraded to this release went from 14 constraints to none: nothing looked broken, but cascade deletes and orphan protection were gone. The set is now captured before the drop and restored afterwards.
DELETE /api/v1/books/{id} and PUT /api/v1/books/{id}/progress returned 405 despite being fully implemented, because the endpoint registry is keyed by path and a URL carrying an ID matches no key exactly. Both routes now work.
Added
Books are shell-free. /books and /book/{id} render entirely from /api/v1. The list keeps its behaviour: the language filter and pagination update the address bar, so a bookmarked ?lg_id=&page= still opens where the reader left off.
Security — XSS hardening, phase 8
The "build markup by string concatenation" pattern is gone from the frontend, closing DOM XSS holes where Google Translate output, multi-word markers, language names and LibreTranslate connection errors reached the DOM as markup. Regression is now held by an invariant and hostile-input tests rather than by vigilance.
Changed
EPUB import no longer depends on an external Composer package (#263). LWT ships its own EPUB reader, replacing kiwilan/php-ebook and four transitive dependencies, taking composer.lock from 20 packages to 15. XML is parsed with entity substitution off and network access disabled, so a hostile EPUB cannot mount an XXE attack.
Removed
Seven orphaned view templates, the last of the frame-era result plumbing, two superseded legacy term routes and the legacy multi-word edit page.
Upgrading is the usual docker compose pull && docker compose up -d --force-recreate; the database migrates itself on first boot. The full changelog is in CHANGELOG.md.