image/jpeg: `jpeg.DecodeConfig` fails on image with sampling-factor: 2x2,1x1,1x2

[bozaro]

What version of Go are you using (go version)?

$ go version
go version go1.19.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bozaro/.cache/go-build"
GOENV="/home/bozaro/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/bozaro/go/pkg/mod"
GONOPROXY=""
GONOSUMDB="github.com/joomcode/*"
GOOS="linux"
GOPATH="/home/bozaro/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.19"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.19/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3402794597=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I try execute jpeg.DecodeConfig for image image.jpeg and got error luma/chroma subsampling ratio at https://github.com/golang/go/blob/go1.19.1/src/image/jpeg/reader.go#L391.

identify -verbose image.jpeg Output

$ identify -verbose image.jpeg
Image:
  Filename: image.jpeg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Mime type: image/jpeg
  Class: DirectClass
  Geometry: 1440x624+0+0
  Units: Undefined
  Colorspace: sRGB
  Type: TrueColor
  Base type: Undefined
  Endianness: Undefined
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Pixels: 898560
    Red:
      min: 0  (0)
      max: 255 (1)
      mean: 173.819 (0.681644)
      standard deviation: 87.5221 (0.343224)
      kurtosis: -0.990504
      skewness: -0.80329
      entropy: 0.558242
    Green:
      min: 0  (0)
      max: 255 (1)
      mean: 168.212 (0.659656)
      standard deviation: 88.9555 (0.348845)
      kurtosis: -1.1937
      skewness: -0.683327
      entropy: 0.568201
    Blue:
      min: 0  (0)
      max: 255 (1)
      mean: 186.976 (0.73324)
      standard deviation: 87.2747 (0.342254)
      kurtosis: -0.759731
      skewness: -0.985872
      entropy: 0.561858
  Image statistics:
    Overall:
      min: 0  (0)
      max: 255 (1)
      mean: 176.336 (0.691513)
      standard deviation: 87.9174 (0.344774)
      kurtosis: -1.00832
      skewness: -0.815069
      entropy: 0.562767
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 1440x624+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 39
  Orientation: Undefined
  Profiles:
    Profile-icc: 452 bytes
  Properties:
    date:create: 2022-09-10T20:09:08+00:00
    date:modify: 2022-09-10T20:09:08+00:00
    icc:description: sRGB ImageOptim.com
    jpeg:colorspace: 2
    jpeg:sampling-factor: 2x2,1x1,1x2
    signature: ee84ebccc5b0bfcb885a9e654e7cbd033d82d0e8ba4d9d2d1f276f6ce742cfd8
  Artifacts:
    filename: image.jpeg
    verbose: true
  Tainted: False
  Filesize: 57127B
  Number pixels: 898560
  Pixels per second: 169.419MB
  User time: 0.000u
  Elapsed time: 0:01.005
  Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org
x

What did you expect to see?

I expects image.Config with actual image size and color model.

What did you see instead?

error: luma/chroma subsampling ratio

[seankhliao]

cc @nigeltao

[ZekeLu]

The following table "decodes" the SOF₀ frame header of the image¹. As you have pointed out, the sampling factors for each component are:

• Y: 2x2
• Cb: 1x1
• Cr: 1x2

The comment in the source code states that:

We also assume that the two chroma components have the same subsampling ratio².

Since v(Cb): 1 is not the same as v(Cr): 2, it returns the error luma/chroma subsampling ratio.

jpeg.DecodeConfig just returns image.Config of an image. I think it should not care about whether the image is supported by jpeg.Decode. I will send a CL to change the behavior. The CL will focus on (*decoder).processSOF, so it just makes jpeg.DecodeConfig work with more but not all valid images.

Offset	Size	Hex value	Value	Description
SOF0 Frame Header
0x274	2	FF C0		SOF0, start of frame marker
0x276	2	00 11	17	Lf: frame header length
0x278	1	08	8	P: sample precision
0x279	2	02 70	624	Y: number of lines
0x27B	2	05 A0	1440	X: Number of samples per line
0x27D	1	03	3	🔴 Nf: 3 components, YCbCr images
Y Component
0x27E	1	01	1	Ci: Component identifier
0x27F	1	22	2, 2	🔴 h: 2, v: 2
0x280	1	00		Tqi: quantization table destination selector
Cb Component
0x281	1	02	2	Ci: Component identifier
0x282	1	11	1, 1	🔴 h: 1, v: 1
0x283	1	01		Tqi: quantization table destination selector
Cr Component
0x284	1	03	3	Ci: Component identifier
0x285	1	12	1, 2	🔴 h: 1, v: 2
0x286	1	01		Tqi: quantization table destination selector

Footnotes

1. https://www.w3.org/Graphics/JPEG/itu-t81.pdf, B.2.2 Frame header syntax, Page 35. ↩

2. https://github.com/golang/go/blob/go1.19.1/src/image/jpeg/reader.go#L369-L393 ↩

[gopherbot]

Change https://go.dev/cl/430315 mentions this issue: image/jpeg: bypass support checking in processSOF for config only calling

[kovidgoyal]

Turns out adding support for decoding "non-standard" subsampling ratios is a trivial fix with a ~100 line patch and no compatibility changes. Simply decode into a 4:4:4 YCbCr image with upsampled data. Patch against my fork of the stdlib jpeg module: kovidgoyal/imaging@92b82f2

It's 20 lines in reader.go and 70 lines in scan.go. Before anyone asks, no I dont have the time/patience to send you a PR, feel free to use the code in the patch linked above if you want to fix this. It should be utterly trivial to adapt the patch to the stdlib jpeg package.