image/jpeg: wrong decoding of a sRGB image

[shepik]

Q1. go version go1.6 linux/amd64
Q2.

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/root/gopath"
GORACE=""
GOROOT="/root/go"
GOTOOLDIR="/root/go/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

Q3. Decoding and re-encoding image produces wrong image.
Steps to reproduce:
a. wget -O test.jpg https://cloud.githubusercontent.com/assets/203454/13348844/ebd58db6-dc86-11e5-91c2-59daeddf50c7.jpg
b. go run file.go
c. open test.jpg and out.jpg in any image viewer

package main

import (
    "bytes"
    "os"
    "log"
    "image"
    "image/jpeg"
    "io/ioutil"
)


func main() {
    reader, err := os.Open("test.jpg")
    defer reader.Close()
    m, _, err := image.Decode(reader)
    if err != nil {
        log.Fatal(err)
    }

    buf := new(bytes.Buffer)
    jpeg.Encode(buf, m, &jpeg.Options{Quality: 85})
    ioutil.WriteFile("out.jpg", buf.Bytes(), 0644)
}

Q4. I expected to see image visually similar to original

Q5. I got image visually different from the original.

[minux]

/cc @nigeltao

[dwbuiten]

What did you use to make this JPEG? Next to nothing opens it properly; only libjpeg and MS's jpeg decoder seem to work, all other libs + photoshop seem to fail. FFmpeg outputs the same broken image.

[shepik]

I got it via vk.com api. Here it is in the web: http://vk.com/wall-6678551_670

I believe vk is not to blame either, because most of photos are fine. I've seen a couple of photos from multiple sources broken that way, so it's not an isolated problem.

[nigeltao]

This is a progressive JPEG image. There are two dimensions of progressivity: spectral selection (variables zs and ze in scan.go, ranging in [0, 63]) and successive approximation (variables ah and al in scan.go, ranging in [0, 8), from LSB to MSB, although ah=0 implicitly means ah=8).

For this particular image, there are three components, and the SOS markers contain this progression:

zs, ze, ah, al: 0 0 0 0 components: 0, 1, 2
zs, ze, ah, al: 1 63 0 0 components: 1
zs, ze, ah, al: 1 63 0 0 components: 2
zs, ze, ah, al: 1 63 0 2 components: 0
zs, ze, ah, al: 1 10 2 1 components: 0
zs, ze, ah, al: 11 63 2 1 components: 0
zs, ze, ah, al: 1 10 1 0 components: 0

The combination of all of these is complete (i.e. spectra 0 to 63 and bits 8 exclusive to 0) for components 1 and 2, but it is incomplete for component 0 (the luma component). In particular, there is no data for component 0, spectra 11 to 63 and bits 1 exclusive to 0.

The image/jpeg code, as of Go 1.6, waits until both dimensions are complete before performing the de-quantization, IDCT and copy to an *image.YCbCr. This is the "if zigEnd != blockSize-1 || al != 0 { ... continue }" code and associated commentary in scan.go.

Almost all progressive JPEG images end up complete in both dimensions for all components, but this particular image is incomplete for component 0, so the Go code never writes anything to the Y values of the resultant *image.YCbCr, which is why the broken output is so dark (but still looks recognizable in terms of red and blue hues).

My reading of the ITU T.81 JPEG specification (Annex G) doesn't explicitly say that this is a valid image, but it also doesn't rule it out.

In any case, the fix is, for progressive JPEG images, to always reconstruct the decoded blocks (by performing the de-quantization, IDCT and copy to an *image.YCbCr), regardless of whether or not they end up complete. Note that, in Go, the jpeg.Decode function does not return until the entire image is decoded, so we still only want to reconstruct each block once, not once per SOS (Start Of Scan) marker.

[gopherbot]

CL https://golang.org/cl/21062 mentions this issue.