image/jpeg: unsupported JPEG feature: luma/chroma subsampling ratio

[benjiro29]

What version of Go are you using (go version)?

go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK='/root/Indribell/go.work'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2633292476=/tmp/go-build -gno-record-gcc-switches'

What did you do?

img, _, err = image.Decode(reader)
if err != nil {
	return Log.Error(err, `Decode`)
}

https://asuracomics.com/wp-content/uploads/2022/07/Untitled-1.jpg
https://asuracomics.com/wp-content/uploads/2021/12/heavenlydemoninstructor-1.jpg

What did you expect to see?

No error

What did you see instead?

unsupported JPEG feature: luma/chroma subsampling ratio)

While the images are displayed correctly in multiple programs, Go Image can simply not decode them without erroring out.

[ianlancetaylor]

CC @nigeltao

[dmitshur]

Issues #13614 and #2362 appear to be related.

[nigeltao]

Those asuracomics.com links don't work for me, but I suspect that this is a duplicate of #2362.

In hindsight, the image.YCbCrSubsampleRatio design, which only supports a fixed allow-list of luma/chroma subsampling ratios, was a mistake. But, due to Go 1 compatibility, fixing that is not going to be easy, per #2362 (comment)

[nigeltao]

Issue #13614 is unrelated, as it involves Go's JPEG encoder. This issue is about the decoder.

[benjiro29]

Those asuracomics.com links don't work for me, but I suspect that this is a duplicate of #2362.

Updated link that is working:

https://asuratoon.com/wp-content/uploads/2022/07/Untitled-1.jpg
https://asuratoon.com/wp-content/uploads/2021/12/heavenlydemoninstructor-1.jpg

Fyi, in regards to #2362 comment

and I'm not sure if it's worth the cost, given how infrequent such images

My ratio of images found with this issue was about ~20% on that website alone.

[KevinColemanInc]

@benjiro29 (or anyone else), is there a quick fix for this? My guess is to use a c-library to parse it, but i'd prefer a pure golang solution.

[benjiro29]

@benjiro29 (or anyone else), is there a quick fix for this? My guess is to use a c-library to parse it, but i'd prefer a pure golang solution.

There is zero solutions for this issue. I bypass it by simply not handling images that have this issue and keeping the original size (with consequences if its like a 8MB image that is now being displayed as a thumb image)...

As long as the attitude prevails of "we made a booboo by assuming nobody will run into this and now we fear fixing it because of backward compatibility" ... with that attitude, it will never get fixed. This issue is known for 13 years, ...

You can not rely on other image handling libraries, as all fall back to Go its build in Image in some way. Tried a bunch, all had the same issue.

Your only option is to waste a tone of time by including a external library and going to cgo route or include a external program that you call for dealing with those images. Both are horrible solutions.

So, do like the go dev's. ignore the issue by keeping the original image and let the dice roll.

[KevinColemanInc]

You can not rely on other image handling libraries, as all fall back to Go its build in Image in some way. Tried a bunch, all had the same issue.

Calling imagemagick fixes the issue, but this is 4-20x worse than pure-go resizing. Our supported qps drops from 25 to 15 qps (without scaling up more servers)

I can't ignore the issue, b/c our ML models can't support large images. and go-lang is the most central place for resizing the images before passing them to the ML models.

My tentative plan is to try to decode with golang. If that fails with the chroma error, fallback to imagemagick. Unfortunately our ci system doesn't support cgo.

If anyone cares to see my imagemagick code:

func img_resize(imageData []byte) ([]byte, error) {
	//maxSizeSetting := "jpeg:extent=" + strconv.Itoa(300) + "kb"
	maxSizeSetting := "jpeg:extent=300kb"
	minResizeSetting := "512x512>"   // resize image down to 512
	maxResizeSetting := "1024x1024<" // resize image up to 1024
	params := []string{
		"-", // input image is passed via stdin
		"-define", maxSizeSetting,
		"-resize", minResizeSetting,
		"-resize", maxResizeSetting,
		"-limit", "thread", "1",
		"-", // output is stdout
	}
	cmd := exec.Command("convert", params...)
	cmd.Env = append(cmd.Env, "MAGICK_THROTTLE=50")