Skip to content

v1.3.0 — decord → PyAV+TOC swap

Choose a tag to compare

View all tags
@praneethnamburi praneethnamburi released this 17 May 11:33
· 81 commits to master since this release
v1.3.0
2f2ba22

Video-reader backend swap: drop the decord runtime dependency in favour of an in-tree PyAV+TOC reader. Public API is preserved — all existing call sites (vr[i].asnumpy(), slicing, vr.get_batch([...]).asnumpy(), len(vr), iteration, vr.get_avg_fps(), vr.get_frame_timestamp([...]), Video(VideoReader) subclass) continue to work unchanged.

The swap is motivated by the 2026-05-16 frame-parity test: decord disagrees with the ffmpeg-CLI oracle on 8/28 frames of a production VFR telemed clip (PSNR 32–37 dB on the divergent frames); PyAV+TOC matches the oracle pixel-exact on all 28 frames across 11 test clips and every codec exercised (libx264 / libx264+B / h264_nvenc ±B / hevc_nvenc / paired CPU↔GPU). The swap is therefore not just a dependency cleanup — it is a correctness improvement on real lab videos.

Added

  • VideoReader and cpu re-exported at the datanavigator package root (from datanavigator import VideoReader, cpu). Decord-compatible API surface — VideoReader(uri, ctx, width, height, num_threads, fault_tol) constructor; vr[i] / slice / get_batch([...]) returning objects with .asnumpy(); len(), iteration, get_avg_fps(). cpu(0) is a no-op sentinel for call-site parity (PyAV is CPU-only).
  • datanavigator/_vendor/pims_pyav_reader.py — PyAVReaderIndexed class vendored from PIMS (BSD-3-Clause, upstream commit d599596, 2023-01-24). Full license reproduced in datanavigator/_vendor/LICENSE-PIMS. PIMS itself is not a runtime dependency.
  • TOC sidecar cache (schema v2). PyAV's TOC build cost is paid once per video and cached as <video>.dnav-toc (JSON content; the .json extension is intentionally omitted so *.json walkers in downstream tooling don't pick the sidecar up). Keyed on path + size + mtime + SHA-256 of the first/last 64 KiB. Cache miss prints datanavigator: building TOC for <name>... and saves a sidecar; cache hit is silent and sub-second. v2 also records per-frame pts / duration / time_base so get_frame_timestamp is a cache hit on second open. datanavigator.precompute_toc(paths, force=False) batch-warms the cache for a sequence of videos before an interactive session.
  • VideoReader.get_frame_timestamp(indices) — decord-compatible per-frame timestamp lookup. Returns an (N, 2) float64 ndarray of [start, end] times in seconds. Backed by per-frame frame.pts (display-order, strictly more accurate than packet.pts * time_base on B-frame streams).
  • Unit-test coverage at tests/test_video_reader.py for the shim and the sidecar flow.

Changed

  • Video-reading backend swapped from decord to PyAV+TOC. Public API preserved. Internal imports in utils.py, videos.py, and opticalflow.py route through the in-tree shim.
  • CI matrix re-adds macos-latest. continue-on-error removed from Windows and macOS runners — test failures now block on all three OSes.

Fixed

  • Video.gray and opticalflow.lucas_kanade grayscale conversion channel order. Both previously called cv.COLOR_BGR2GRAY on RGB-decoded frames, silently swapping R and B before the grayscale weighting. Now use cv.COLOR_RGB2GRAY. Observable behavior change for any caller depending on the buggy luminance.

Removed

  • decord runtime dependency (replaced by in-tree PyAV+TOC reader).
  • numpy<2 pin in pyproject.toml and requirements.yml (held by decord + tables<3.10; both gates are now gone).
  • Implicit tables<3.10 upper bound: dep is now tables>=3.10.

Infrastructure

  • pyproject.toml: decord → av; numpy<2 → numpy; tables → tables>=3.10.
  • requirements.yml: mirrored.
  • .github/workflows/ci.yml: macos-latest re-added; continue-on-error dropped on all runners; brew ffmpeg step added for macOS.

📦 PyPI 1.3.0 · 📋 CHANGELOG

Assets 2
Loading