Skip to content

Ubuntu 26.04 base (ffmpeg 8, ImageMagick 7), Valkey 9, Postgres 18 (v2.7.0) - #68

Merged
ttlequals0 merged 4 commits into
mainfrom
feature/ubuntu-2604-stack
Jul 16, 2026
Merged

Ubuntu 26.04 base (ffmpeg 8, ImageMagick 7), Valkey 9, Postgres 18 (v2.7.0)#68
ttlequals0 merged 4 commits into
mainfrom
feature/ubuntu-2604-stack

Conversation

@ttlequals0

@ttlequals0 ttlequals0 commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Fixes #67.

Base image: Ubuntu 24.04 to 26.04

  • FFmpeg 6.1.1 becomes 8.0.1, ImageMagick 6.9 becomes 7.1.2 (Q16, non-HDRI variant).
  • Ubuntu 26.04 ships Python 3.14; the app pins 3.12 via deadsnakes and runs from /opt/venv (psycopg2-binary has no 3.14 wheels, and 3.12 matches the tested CI matrix). Same pattern MinusPod used for its 26.04 upgrade.
  • The image checker now invokes magick when present, falling back to convert on ImageMagick 6 systems (Ubuntu's IM7 packages keep convert as an alternatives symlink, but IM7 upstream deprecates it).
  • The temporal-outlier ffprobe call uses pts_time - pkt_pts_time was removed in newer FFmpeg, which silently disabled that corruption signal (the parser fails soft).
  • ImageMagick policy edits target /etc/ImageMagick-7 with a build-time assertion so a silent sed no-op fails the build instead of shipping default resource limits. The 26.04 policy has no PDF/HEIC coder blocks, so the old un-blocking seds are gone.
  • linux-libc-dev and build headers are purged from the final image, which clears the long tail of unfixable kernel-header CVEs from scanners. The unused pebble binary in the 26.04 rootfs is removed for the same reason.

Bonus fix found during verification: the gates that classify benign ImageMagick PNG/profile warnings checked for @warning/png.c, but ImageMagick 6 and 7 both emit @ warning/png.c with a space - the benign paths were unreachable since they were written. Fixed here and in tools/fix_imagemagick_profile_warnings.py.

Valkey 9 (drop-in Redis replacement)

docker-compose broker image becomes valkey/valkey:9-alpine. The service name, all redis:// URLs, and the wire protocol are unchanged; the code uses plain RESP plus one EVAL script, all supported. The Valkey image ships redis-* compatibility symlinks.

PostgreSQL 18 (BREAKING compose default)

  • compose default becomes postgres:18-alpine. A 15-era postgres_data volume will not start on the 18 image - a tested dump/restore migration guide is in docs/DOCKER_SETUP.md, and README carries the warning. Users can pin postgres:15-alpine to defer; the app supports 15 through 18.
  • Found during verification: the postgres 18+ Docker images also moved the expected volume mount from /var/lib/postgresql/data to /var/lib/postgresql (major-version subdirectories, enabling pg_upgrade --link) and refuse to start with a mount at the old path. The compose file and guide cover this.

CI

New image-integration job builds the Docker image and runs the real-media parser tests inside it. The ffmpeg/ImageMagick stderr parsers fail soft (a format change yields zero detections, not an error), so only testing against the shipped binaries catches base-image regressions.

Verification

  • Real-media suite inside the new image: 10/10 pass, including corruption detection, MP3 broken frames, invalid AIFF, and corrupted images on ffmpeg 8 / ImageMagick 7.
  • freezedetect/blackdetect log formats and the @ warning/png.c token verified against real ffmpeg 8 / IM7 output; -regard-warnings exit-code differences between IM6 and IM7 traced through the classification paths (net effect: fewer false positives, warning-class corruption still caught).
  • Fresh compose stack (postgres:18 + valkey:9 + this image): all containers healthy, admin setup, full scan via API - 5 sample files, exactly the 2 planted corrupt files flagged (ffmpeg and imagemagick respectively), scheduler lock held by the celery worker through Valkey.
  • Migration guide dry-run: 5000 rows dumped from postgres:15, restored into 18.4, row counts match, collation refresh clean.
  • Host suite: 475 passed, 8 skipped. Frontend builds.

Media viewer: closing the player now stops playback

Closing the viewer modal only hid it, so a playing video or audio file kept playing in the background. closeModal() now pauses any media in the modal and detaches its sources; both close paths (X button, outside click) route through it. Verified in-browser: playback stops and sources detach for video and audio.

Documentation accuracy audit

Every doc was audited against the codebase and corrected (60+ findings). The worst offenders documented endpoints that never existed (/api/scan-all, /api/stats/summary, /api/scan/parallel-v2*, /api/export/csv, /api/cleanup) with no authentication; DOCKER_SETUP.md's example compose could not have booted (nonexistent celery_app module, authenticated /health as healthcheck, missing SECRET_KEY); CONFIGURATION.md described a Celery beat and four queues that do not exist; PERFORMANCE_TUNING.md tuned env vars that appear nowhere in the code. All example clients now authenticate and call real endpoints. openapi.yaml lost a duplicate path key and a nonexistent DELETE route. A hardcoded deployment hostname was removed from FIX_INCOMPLETE_SCANS.md. The PostgreSQL 15-to-18 migration guide is linked from the README docs index and the Requirements upgrade warning.

…2.7.0)

Fixes #67.

Base image moves to ubuntu:26.04: FFmpeg 6.1.1 -> 8.0.1 and ImageMagick
6.9 -> 7.1.2. The app pins Python 3.12 via deadsnakes in a dedicated venv
(26.04 ships 3.14, which psycopg2-binary has no wheels for). The image
checker invokes magick with a convert fallback, the temporal-outlier probe
uses pts_time (pkt_pts_time was removed in newer FFmpeg and silently
disabled that signal), and kernel header packages are purged from the
final image, clearing the unfixable linux-libc-dev CVE tail.

Also fixed: the benign ImageMagick warning gates checked for
"@warning/png.c" but ImageMagick (6 and 7) emits "@ warning/png.c" with a
space, so the benign-classification paths were unreachable; same token
fixed in tools/fix_imagemagick_profile_warnings.py.

docker-compose defaults change to postgres:18-alpine (BREAKING for
existing volumes; the 18+ images also moved the volume mount from
/var/lib/postgresql/data to /var/lib/postgresql) and valkey/valkey:9-alpine
(drop-in; service name and URLs unchanged). A tested 15->18 dump/restore
migration guide is in docs/DOCKER_SETUP.md.

CI gains an image-integration job that builds the Docker image and runs
the real-media parser tests inside it, since the ffmpeg/ImageMagick stderr
parsers fail soft and only break observably against the shipped binaries.

Verified: real-media suite passes inside the new image (10/10, including
corruption detection on ffmpeg 8 / IM7); fresh compose stack on
postgres 18 + valkey 9 scanned 5 samples end-to-end and flagged exactly
the 2 planted corrupt files; 15->18 dump/restore dry-run row counts match;
host suite 475 passed.
Media viewer fix: closing the player modal only hid it, so a playing
video or audio file kept playing (and buffering) in the background.
closeModal() now pauses any media in the modal and detaches its sources;
the viewer close button and outside-click both route through it.
Verified in-browser: play, close, playback stops and sources detach for
both video and audio.

Docs audit: every doc was checked against the codebase by parallel
auditors and corrected. Highlights:
- docs/api/README.md and docs/examples/* documented endpoints that never
  existed (/api/scan-all, /api/stats/summary, /api/scan/parallel-v2*,
  /api/export/csv, /api/cleanup) and omitted authentication; all examples
  now call real endpoints with Bearer tokens.
- DOCKER_SETUP.md's example compose would not have booted (celery_app
  module, /health healthcheck, missing SECRET_KEY, allkeys-lru eviction);
  it now mirrors the real docker-compose.yml.
- CONFIGURATION.md/INSTALLATION.md described a Celery beat and a
  four-queue layout that do not exist; replaced with the real APScheduler
  jobs and the single pixelprobe queue. CELERY_WORKERS (read nowhere)
  corrected to CELERY_CONCURRENCY everywhere.
- PERFORMANCE_TUNING.md rewritten: its entire env-var surface
  (MAX_SCAN_WORKERS, BATCH_COMMIT_SIZE, ...) never existed in this code.
- Pool sizes (5+10, not 20+40), session lifetime (24h, not 30 days),
  task lists, adaptive chunk tiers, stuck-scan thresholds, module/model
  inventories all corrected against source.
- openapi.yaml: removed duplicate /configurations path key and the
  nonexistent DELETE /configurations/{id}; schedule scan_type enum now
  includes file_changes and orphan; export formats csv/json/pdf only.
- Removed a hardcoded deployment hostname from FIX_INCOMPLETE_SCANS.md.
Full-file pass over every doc (README, docs/, docs/api, docs/developer,
docs/examples, docs/maintenance) removing AI-writing patterns while
leaving audited technical facts untouched: significance inflation and
promotional adjectives cut, "What it does / Why needed" bullet walls
rewritten as prose, boilerplate best-practices lists reduced to their
concrete items, duplicate link indexes and say-nothing intros removed,
Unicode arrows in prose replaced with ASCII. Also removed two dead links
into the gitignored docs/development/ directory and fixed list numbering
in TOOLS_AND_SCRIPTS.md. Net 96 lines shorter.
@ttlequals0
ttlequals0 merged commit 3751c19 into main Jul 16, 2026
12 checks passed
@ttlequals0
ttlequals0 deleted the feature/ubuntu-2604-stack branch July 16, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ffmpeg 8 / Valkey redis 9 / Postgres 18

1 participant