Ubuntu 26.04 base (ffmpeg 8, ImageMagick 7), Valkey 9, Postgres 18 (v2.7.0) - #68
Merged
Conversation
…2.7.0) Fixes #67. Base image moves to ubuntu:26.04: FFmpeg 6.1.1 -> 8.0.1 and ImageMagick 6.9 -> 7.1.2. The app pins Python 3.12 via deadsnakes in a dedicated venv (26.04 ships 3.14, which psycopg2-binary has no wheels for). The image checker invokes magick with a convert fallback, the temporal-outlier probe uses pts_time (pkt_pts_time was removed in newer FFmpeg and silently disabled that signal), and kernel header packages are purged from the final image, clearing the unfixable linux-libc-dev CVE tail. Also fixed: the benign ImageMagick warning gates checked for "@warning/png.c" but ImageMagick (6 and 7) emits "@ warning/png.c" with a space, so the benign-classification paths were unreachable; same token fixed in tools/fix_imagemagick_profile_warnings.py. docker-compose defaults change to postgres:18-alpine (BREAKING for existing volumes; the 18+ images also moved the volume mount from /var/lib/postgresql/data to /var/lib/postgresql) and valkey/valkey:9-alpine (drop-in; service name and URLs unchanged). A tested 15->18 dump/restore migration guide is in docs/DOCKER_SETUP.md. CI gains an image-integration job that builds the Docker image and runs the real-media parser tests inside it, since the ffmpeg/ImageMagick stderr parsers fail soft and only break observably against the shipped binaries. Verified: real-media suite passes inside the new image (10/10, including corruption detection on ffmpeg 8 / IM7); fresh compose stack on postgres 18 + valkey 9 scanned 5 samples end-to-end and flagged exactly the 2 planted corrupt files; 15->18 dump/restore dry-run row counts match; host suite 475 passed.
Media viewer fix: closing the player modal only hid it, so a playing
video or audio file kept playing (and buffering) in the background.
closeModal() now pauses any media in the modal and detaches its sources;
the viewer close button and outside-click both route through it.
Verified in-browser: play, close, playback stops and sources detach for
both video and audio.
Docs audit: every doc was checked against the codebase by parallel
auditors and corrected. Highlights:
- docs/api/README.md and docs/examples/* documented endpoints that never
existed (/api/scan-all, /api/stats/summary, /api/scan/parallel-v2*,
/api/export/csv, /api/cleanup) and omitted authentication; all examples
now call real endpoints with Bearer tokens.
- DOCKER_SETUP.md's example compose would not have booted (celery_app
module, /health healthcheck, missing SECRET_KEY, allkeys-lru eviction);
it now mirrors the real docker-compose.yml.
- CONFIGURATION.md/INSTALLATION.md described a Celery beat and a
four-queue layout that do not exist; replaced with the real APScheduler
jobs and the single pixelprobe queue. CELERY_WORKERS (read nowhere)
corrected to CELERY_CONCURRENCY everywhere.
- PERFORMANCE_TUNING.md rewritten: its entire env-var surface
(MAX_SCAN_WORKERS, BATCH_COMMIT_SIZE, ...) never existed in this code.
- Pool sizes (5+10, not 20+40), session lifetime (24h, not 30 days),
task lists, adaptive chunk tiers, stuck-scan thresholds, module/model
inventories all corrected against source.
- openapi.yaml: removed duplicate /configurations path key and the
nonexistent DELETE /configurations/{id}; schedule scan_type enum now
includes file_changes and orphan; export formats csv/json/pdf only.
- Removed a hardcoded deployment hostname from FIX_INCOMPLETE_SCANS.md.
Full-file pass over every doc (README, docs/, docs/api, docs/developer, docs/examples, docs/maintenance) removing AI-writing patterns while leaving audited technical facts untouched: significance inflation and promotional adjectives cut, "What it does / Why needed" bullet walls rewritten as prose, boilerplate best-practices lists reduced to their concrete items, duplicate link indexes and say-nothing intros removed, Unicode arrows in prose replaced with ASCII. Also removed two dead links into the gitignored docs/development/ directory and fixed list numbering in TOOLS_AND_SCRIPTS.md. Net 96 lines shorter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #67.
Base image: Ubuntu 24.04 to 26.04
magickwhen present, falling back toconverton ImageMagick 6 systems (Ubuntu's IM7 packages keepconvertas an alternatives symlink, but IM7 upstream deprecates it).pts_time-pkt_pts_timewas removed in newer FFmpeg, which silently disabled that corruption signal (the parser fails soft).Bonus fix found during verification: the gates that classify benign ImageMagick PNG/profile warnings checked for
@warning/png.c, but ImageMagick 6 and 7 both emit@ warning/png.cwith a space - the benign paths were unreachable since they were written. Fixed here and in tools/fix_imagemagick_profile_warnings.py.Valkey 9 (drop-in Redis replacement)
docker-compose broker image becomes
valkey/valkey:9-alpine. The service name, all redis:// URLs, and the wire protocol are unchanged; the code uses plain RESP plus one EVAL script, all supported. The Valkey image ships redis-* compatibility symlinks.PostgreSQL 18 (BREAKING compose default)
postgres:18-alpine. A 15-erapostgres_datavolume will not start on the 18 image - a tested dump/restore migration guide is in docs/DOCKER_SETUP.md, and README carries the warning. Users can pinpostgres:15-alpineto defer; the app supports 15 through 18./var/lib/postgresql/datato/var/lib/postgresql(major-version subdirectories, enablingpg_upgrade --link) and refuse to start with a mount at the old path. The compose file and guide cover this.CI
New
image-integrationjob builds the Docker image and runs the real-media parser tests inside it. The ffmpeg/ImageMagick stderr parsers fail soft (a format change yields zero detections, not an error), so only testing against the shipped binaries catches base-image regressions.Verification
@ warning/png.ctoken verified against real ffmpeg 8 / IM7 output;-regard-warningsexit-code differences between IM6 and IM7 traced through the classification paths (net effect: fewer false positives, warning-class corruption still caught).Media viewer: closing the player now stops playback
Closing the viewer modal only hid it, so a playing video or audio file kept playing in the background. closeModal() now pauses any media in the modal and detaches its sources; both close paths (X button, outside click) route through it. Verified in-browser: playback stops and sources detach for video and audio.
Documentation accuracy audit
Every doc was audited against the codebase and corrected (60+ findings). The worst offenders documented endpoints that never existed (/api/scan-all, /api/stats/summary, /api/scan/parallel-v2*, /api/export/csv, /api/cleanup) with no authentication; DOCKER_SETUP.md's example compose could not have booted (nonexistent celery_app module, authenticated /health as healthcheck, missing SECRET_KEY); CONFIGURATION.md described a Celery beat and four queues that do not exist; PERFORMANCE_TUNING.md tuned env vars that appear nowhere in the code. All example clients now authenticate and call real endpoints. openapi.yaml lost a duplicate path key and a nonexistent DELETE route. A hardcoded deployment hostname was removed from FIX_INCOMPLETE_SCANS.md. The PostgreSQL 15-to-18 migration guide is linked from the README docs index and the Requirements upgrade warning.