Skip to content

Single-file rescan UX + post-fork DB engine disposal (v2.6.41)#56

Merged
ttlequals0 merged 1 commit intomainfrom
fix/single-file-scan-ux-2.6.41
Apr 27, 2026
Merged

Single-file rescan UX + post-fork DB engine disposal (v2.6.41)#56
ttlequals0 merged 1 commit intomainfrom
fix/single-file-scan-ux-2.6.41

Conversation

@ttlequals0
Copy link
Copy Markdown
Owner

@ttlequals0 ttlequals0 commented Apr 26, 2026

Summary

  • Single-file rescan UI flips to "done" then resumes minutes later. The Flask /api/scan-file route created a ScanState row with its generated scan_id, but the Celery worker's scan_service.scan_single_file then created a second row with a different scan_id and tracked progress on that one. The UI's progress monitor briefly saw no active scan and reported the rescan complete before the worker started hashing. scan_single_file now accepts an optional scan_id and reuses the existing row; scan_media_task passes scan_id through for scan_type='single'.
  • Silent first-attempt failures on Celery worker scans caused by post-fork libpq socket sharing. Symptom in worker logs: psycopg2.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq on one ForkPoolWorker, surfacing as a bare NotImplementedError from sqlalchemy/engine/result.py:_indexes_for_keys in a sibling worker. Fixed by disposing the SQLAlchemy engine in the worker_process_init Celery signal so each forked child builds its own connection pool. The existing log-handler setup in that signal was merged into a single _setup_worker_process handler.
  • Scan progress flickers to "failed" during transient retries. scan_media_task's catch-all exception handler used to set phase='failed' and is_active=False on every error, including ones about to be retried. The handler now keeps the row active (phase='initializing' with a Retrying after error (attempt N/M) message) when more retries remain, only marking the scan failed once the retry budget is exhausted or the error is the known-fatal PGRES_TUPLES_OK corruption case.
  • Bump python-dotenv 1.0.0 -> 1.2.2 to resolve CVE-2026-28684 (MEDIUM, arbitrary file overwrite via symlink follow).
  • Refactors out of /simplify: extracted is_db_connection_corruption helper in pixelprobe/utils/celery_utils.py (was inline at two sites in tasks.py); replaced raw 'initializing' phase strings with SCAN_PHASES['INITIALIZING'] from pixelprobe/constants.py.

Version

pixelprobe/version.py: 2.6.40 -> 2.6.41

Test plan

  • All 320 tests pass (8 skipped, 0 failures); 9 min runtime
  • New regression test: test_scan_single_file_reuses_existing_scan_state covers the create-new vs reuse-existing branch in scan_single_file
  • Docker image ttlequals0/pixelprobe:2.6.41 built for linux/amd64 and pushed
  • Trivy: 0 HIGH, 0 CRITICAL, 0 MEDIUM at the application layer (Ubuntu 24.04 base MEDIUMs unchanged)
  • After merge: production deploy + verify UI shows continuous progress on a single-file rescan, and worker logs no longer contain PGRES_TUPLES_OK or NotImplementedError

….41)

Fix the UI flickering to "done" mid-rescan and the silent first-attempt
NotImplementedError on Celery worker scans.

- scan_service.scan_single_file accepts an optional scan_id and reuses the
  existing ScanState row when one matches. The Celery scan_media_task
  passes scan_id through for scan_type='single'. Eliminates the second
  ScanState that the UI's progress monitor lost track of.
- worker_process_init signal now disposes the SQLAlchemy engine in each
  forked Celery worker, so post-fork libpq sockets aren't shared with the
  parent process. Removes the recurring PGRES_TUPLES_OK / NotImplementedError
  pattern in the worker logs.
- scan_media_task catch-all preserves the ScanState row across retries
  (phase='initializing', is_active=True with a 'Retrying after error'
  progress message) instead of marking it failed/inactive on every attempt.
- Extract the corruption-error string check into is_db_connection_corruption
  in pixelprobe/utils/celery_utils.py and use it at both detection sites.
- Replace stringly-typed scan phase constants with SCAN_PHASES['INITIALIZING'].
- Bump python-dotenv 1.0.0 -> 1.2.2 for CVE-2026-28684 (MEDIUM, symlink
  arbitrary file overwrite).

New regression test: test_scan_single_file_reuses_existing_scan_state.
All 320 tests pass; trivy clean of HIGH/CRITICAL/MEDIUM at the app layer.
@ttlequals0 ttlequals0 merged commit 4271740 into main Apr 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant