fix: replace ZM_COLOUR system with AVPixelFormat for format dispatch

[connortechnology]

Summary

• Replace legacy ZM_COLOUR_* / ZM_SUBPIX_ORDER_* integer pair with AVPixelFormat as the single source of truth for pixel format dispatch across the codebase
• Add src/zm_pixformat.h with central format helpers (zm_pixformat_from_colours, zm_is_rgb32, zm_is_rgb24, zm_is_yuv420, zm_bytes_per_pixel, zm_db_colours_to_pixformat)
• Migrate all ~100 format dispatch comparisons in zm_image.cpp, camera subclasses, zm_monitor.cpp, zm_mpeg.cpp, zm_ffmpeg.cpp from colours/subpixelorder checks to imagePixFormat/AVPixelFormat checks
• Fix imagePixFormat sync bug in WriteBuffer, Assign, AssignDirect — these updated colours/subpixelorder without updating imagePixFormat, causing format misidentification (vertical lines and washed-out colors in live stream)
• Fix DeColourise bug where imagePixFormat was not updated to AV_PIX_FMT_GRAY8
• Add AVPixelFormat pixelFormat member + PixelFormat() accessor to Camera
• Add PixFormat() accessor to Image
• Deprecate GetFFMPEGPixelFormat(), delegate to zm_pixformat_from_colours()
• Deprecate ZM_COLOUR_* and ZM_SUBPIX_ORDER_* constants in zm_rgb.h
• Add deprecation notice on Monitor.Colours web UI dropdown
• Add 13 Catch2 test cases (105 assertions) for format mapping helpers

Test plan

• ctest — 105 pixformat assertions pass, no regressions
• Full build clean (all targets: zmc, zms, zmu, zma, zm_rtsp_server)
• V4L2 MJPEG camera with Colours=4 (RGB32): verify color output, no vertical lines
• Monitor with Colours=1 (GRAY8): verify still works as grayscale
• Event recording via h264 encoder: verify video output is correct

Fixes #4735

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR modernizes ZoneMinder’s pixel-format handling by making AVPixelFormat the authoritative format identifier (replacing the legacy ZM_COLOUR_* + ZM_SUBPIX_ORDER_* pair) to fix long-standing format-collision issues and reduce fragile dispatch logic across cameras, images, monitors, and MPEG/FFmpeg components.

Changes:

• Introduces src/zm_pixformat.h with centralized pixel-format mapping/predicates and migrates many call sites to dispatch on AVPixelFormat.
• Fixes several Image format-sync issues by keeping imagePixFormat updated in buffer assignment/write paths and when de/colourising.
• Adds Catch2 unit tests for pixformat mappings and adds a web UI deprecation notice for the legacy Monitor “Colours” dropdown.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
web/skins/classic/views/monitor.php	Adds deprecation notice for the legacy “Colours” UI setting.
tests/CMakeLists.txt	Registers new pixformat test file in the test suite.
tests/zm_pixformat.cpp	Adds unit coverage for format mapping helpers and collision regression.
src/zm_pixformat.h	Adds centralized AVPixelFormat mapping helpers and predicates.
src/zm_rgb.h	Marks legacy colour/subpixel constants as deprecated and documents DB compatibility.
src/zm_camera.h	Adds AVPixelFormat pixelFormat member + accessor; deprecates legacy accessors.
src/zm_camera.cpp	Initializes pixelFormat and uses it for stream codecpar format.
src/zm_ffmpeg.cpp	Deprecates GetFFMPEGPixelFormat() and delegates to pixformat helper.
src/zm_mpeg.cpp	Simplifies codec setup pixel-format selection via pixformat helper.
src/zm_image.h	Adds PixFormat() accessor; deprecates AVPixFormat() getter.
src/zm_image.cpp	Migrates dispatch to AVPixelFormat, fixes several sync issues, updates JPEG handling logic.
src/zm_monitor.cpp	Updates image handling/dispatch to use PixFormat() and pixformat helpers.
src/zm_local_camera.cpp	Updates palette/target format matching and auto-select logic to use AVPixelFormat helpers.
src/zm_remote_camera_rtsp.cpp	Updates camera constructor format dispatch to use pixelFormat helpers.
src/zm_ffmpeg_camera.cpp	Updates camera constructor format dispatch to use pixelFormat helpers.
src/zm_libvnc_camera.cpp	Updates VNC camera format dispatch to use pixelFormat helpers.
src/zm_libvlc_camera.cpp	Updates libVLC camera format dispatch and chroma selection via pixformat helpers.

Comments suppressed due to low confidence (2)

src/zm_image.cpp:1004

• HighlightEdges computes destination pointers using this->linesize (source stride). If the source grayscale image came from an AVFrame with padding (linesize > width), then phigh will advance too far and can write past high_buff. Use the destination image stride (high_image->LineSize()) or compute offsets from width/bytes-per-pixel instead of reusing the source linesize.

    for ( unsigned int y = lo_y; y <= hi_y; y++ ) {
      const uint8_t* p = buffer + (y * linesize) + lo_x;
      uint8_t* phigh = high_buff + (((y * linesize) + lo_x) * 3);
      for ( unsigned int x = lo_x; x <= hi_x; x++, p++, phigh += 3 ) {
        bool edge = false;
        if ( *p ) {
          edge = (x > 0 && !*(p-1)) || (x < (width-1) && !*(p+1)) || (y > 0 && !*(p-width)) || (y < (height-1) && !*(p+width));
#if 0
          if ( !edge && x > 0 && !*(p-1) ) edge = true;
          if ( !edge && x < (width-1) && !*(p+1) ) edge = true;
          if ( !edge && y > 0 && !*(p-width) ) edge = true;
          if ( !edge && y < (height-1) && !*(p+width) ) edge = true;
#endif
        }
        if ( edge ) {
          RED_PTR_RGBA(phigh) = RED_VAL_RGBA(colour);
          GREEN_PTR_RGBA(phigh) = GREEN_VAL_RGBA(colour);
          BLUE_PTR_RGBA(phigh) = BLUE_VAL_RGBA(colour);
        }
      }
    }
  } else if ( zm_is_rgb32(p_pixfmt) ) {
    for ( unsigned int y = lo_y; y <= hi_y; y++ ) {
      const uint8_t* p = buffer + (y * linesize) + lo_x;
      Rgb* phigh = (Rgb*)(high_buff + (((y * linesize) + lo_x) * 4));
      for ( unsigned int x = lo_x; x <= hi_x; x++, p++, phigh++ ) {

src/zm_image.cpp:1399

• The YUV420/J420 JPEG path assumes a packed 4:2:2 (YUYV) layout (e.g., offset uses width*2 and reads bytes as Y U Y V). But the predicate now triggers for AV_PIX_FMT_YUV420P/YUVJ420P, which are planar 4:2:0 formats with different layout/stride; this can read out of bounds and/or encode incorrect colors. Either switch this branch to the correct packed pixel format (e.g., AV_PIX_FMT_YUYV422) or rework it to read true planar YUV420P data (using proper plane pointers/strides).

    } else if (zm_is_yuv420(imagePixFormat)) {
      cinfo->in_color_space = JCS_YCbCr;
    } else {
      /* Assume RGB */
      /*
      #ifdef JCS_EXTENSIONS
      cinfo->out_color_space = JCS_EXT_RGB;
      #else
      cinfo->out_color_space = JCS_RGB;
      #endif
      */
      cinfo->in_color_space = JCS_RGB;
    }
  }  // end format dispatch

  jpeg_set_defaults(cinfo);
  jpeg_set_quality(cinfo, quality, FALSE);
  cinfo->dct_method = JDCT_FASTEST;

  jpeg_start_compress(cinfo, TRUE);
  if (config.add_jpeg_comments && !annotation_.empty()) {
    jpeg_write_marker(cinfo, JPEG_COM, reinterpret_cast<const JOCTET *>(annotation_.c_str()), annotation_.size());
  }
  // If we have a non-zero time (meaning a parameter was passed in), then form a simple exif segment with that time as DateTimeOriginal and SubsecTimeOriginal
  // No timestamp just leave off the exif section.
  if (timestamp.time_since_epoch() > Seconds(0)) {
#define EXIFTIMES_MS_OFFSET 0x36   // three decimal digits for milliseconds
#define EXIFTIMES_MS_LEN  0x03
#define EXIFTIMES_OFFSET  0x3E   // 19 characters format '2015:07:21 13:14:45' not including quotes
#define EXIFTIMES_LEN     0x13   // = 19
#define EXIF_CODE       0xE1

    // This is a lot of stuff to allocate on the stack.  Recommend char *timebuf[64];
    char timebuf[64], msbuf[64];

    tm timestamp_tm = {};
    time_t timestamp_t = std::chrono::system_clock::to_time_t(timestamp);
    strftime(timebuf, sizeof timebuf, "%Y:%m:%d %H:%M:%S", localtime_r(&timestamp_t, &timestamp_tm));
    Seconds ts_sec = std::chrono::duration_cast<Seconds>(timestamp.time_since_epoch());
    Microseconds ts_usec = std::chrono::duration_cast<Microseconds>(timestamp.time_since_epoch() - ts_sec);
    // we only use milliseconds because that's all defined in exif, but this is the whole microseconds because we have it
    snprintf(msbuf, sizeof msbuf, "%06d", static_cast<int32>(ts_usec.count()));

    unsigned char exiftimes[82] = {
      0x45, 0x78, 0x69, 0x66, 0x00, 0x00, 0x49, 0x49, 0x2A, 0x00, 0x08, 0x00, 0x00, 0x00, 0x01, 0x00,
      0x69, 0x87, 0x04, 0x00, 0x01, 0x00, 0x00, 0x00, 0x1A, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
      0x02, 0x00, 0x03, 0x90, 0x02, 0x00, 0x14, 0x00, 0x00, 0x00, 0x38, 0x00, 0x00, 0x00, 0x91, 0x92,
      0x02, 0x00, 0x04, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff,
      0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
      0xff, 0x00
    };
    memcpy(&exiftimes[EXIFTIMES_OFFSET], timebuf, EXIFTIMES_LEN);
    memcpy(&exiftimes[EXIFTIMES_MS_OFFSET], msbuf, EXIFTIMES_MS_LEN);
    jpeg_write_marker(cinfo, EXIF_CODE, (const JOCTET *) exiftimes, sizeof(exiftimes));
  }

  if (zm_is_yuv420(imagePixFormat)) {
    std::vector<uint8_t> tmprowbuf(width * 3);
    JSAMPROW row_pointer = &tmprowbuf[0];  /* pointer to a single row */
    while (cinfo->next_scanline < cinfo->image_height) {
      unsigned i, j;
      unsigned offset = cinfo->next_scanline * cinfo->image_width * 2; //offset to the correct row
      for (i = 0, j = 0; i < cinfo->image_width * 2; i += 4, j += 6) { //input strides by 4 bytes, output strides by 6 (2 pixels)
        tmprowbuf[j + 0] = buffer[offset + i + 0]; // Y (unique to this pixel)
        tmprowbuf[j + 1] = buffer[offset + i + 1]; // U (shared between pixels)
        tmprowbuf[j + 2] = buffer[offset + i + 3]; // V (shared between pixels)
        tmprowbuf[j + 3] = buffer[offset + i + 2]; // Y (unique to this pixel)
        tmprowbuf[j + 4] = buffer[offset + i + 1]; // U (shared between pixels)
        tmprowbuf[j + 5] = buffer[offset + i + 3]; // V (shared between pixels)
      }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

For V4L2_PIX_FMT_RGB24, this sets subpixelorder to BGR and then derives pixelFormat from that, which will treat an RGB24 capture buffer as BGR24 (red/blue swapped). This should use ZM_SUBPIX_ORDER_RGB (and resulting AV_PIX_FMT_RGB24) when the capture palette is RGB24.

Suggested change

	subpixelorder = ZM_SUBPIX_ORDER_BGR;
	subpixelorder = ZM_SUBPIX_ORDER_RGB;

[Copilot]

zm_colours_from_pixformat() can return false for unrecognised AVPixelFormat values, leaving colours/subpixelorder unchanged. That can make Image state inconsistent with imagePixFormat (and with the assigned frame). Please handle the false case explicitly (e.g., log and set imagePixFormat=AV_PIX_FMT_NONE plus a safe colours/subpixelorder), or restrict AssignDirect() to supported formats.

Suggested change

	zm_colours_from_pixformat(imagePixFormat, colours, subpixelorder);
	if ( !zm_colours_from_pixformat(imagePixFormat, colours, subpixelorder) ) {
	Error("AssignDirect called with unsupported pixel format: %d", frame->format);
	imagePixFormat = AV_PIX_FMT_NONE;
	colours = 0;
	subpixelorder = 0;
	}

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 8 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.