Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

image/jpeg: "bad RST marker" error when decoding #40130

Closed
haoyang-zheng opened this issue Jul 9, 2020 · 7 comments · May be fixed by #41875
Closed

image/jpeg: "bad RST marker" error when decoding #40130

haoyang-zheng opened this issue Jul 9, 2020 · 7 comments · May be fixed by #41875
Labels
FixPending Issues that have a fix which has not yet been reviewed or submitted. NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.23

Comments

@haoyang-zheng
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.14.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/teikouyou/Library/Caches/go-build"
GOENV="/Users/teikouyou/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/teikouyou/dev/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/d7/wlm33f9550x_3bzzp_3b32h40000gn/T/go-build574141687=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

The following error occurred when I decoded a specific image.

invalid JPEG format: bad RST marker

Part of binary data of the image

ff ff ff ff ff ff ff ff ff ff ff ff d0 fd 32 4b

The correct marker here should be [ ff d0 ], but there is some ff bytes in front of it.

It seems that there is a consecutive 0xFF bytes in the SOS segment that causes problems in marker's matching.

This is a quote from jpeg Wikipedia.
https://en.wikipedia.org/wiki/JPEG#Syntax_and_structure

Note that consecutive 0xFF bytes are used as fill bytes for padding purposes, although this fill byte padding should only ever take place for markers immediately following entropy-coded scan data (see JPEG specification section B.1.1.2 and E.1.2 for details; specifically "In all cases where markers are appended after the compressed data, optional 0xFF fill bytes may precede the marker"

So I think this situation should be taken into account at image/jpeg/scan.go

go/src/image/jpeg/scan.go

Line 332 in 4f2a2d7

if d.tmp[0] != 0xff || d.tmp[1] != expectedRST {

What did you expect to see?

Image successfully decoded

What did you see instead?

invalid JPEG format: bad RST marker
@dmitshur dmitshur changed the title image jpeg error when decoding : bad RST marker image/jpeg: "bad RST marker" error when decoding Jul 10, 2020
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 10, 2020
@dmitshur dmitshur added this to the Backlog milestone Jul 10, 2020
@dmitshur
Copy link
Contributor

/cc @nigeltao @robpike per owners.

@shouldend shouldend mentioned this issue Oct 9, 2020
Open
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/260837 mentions this issue: image/jpeg: skip consecutive 0xff bytes

@lastzero lastzero mentioned this issue Oct 26, 2021
Closed
@moosepants
Copy link

@haoyang-zheng Do you have the example file for this issue to test against?

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/382754 mentions this issue: image/jpeg: improve handling of JPEG restart markers in non-ideal cases

@lastzero lastzero mentioned this issue Jun 25, 2022
Closed
@n0vad3v n0vad3v mentioned this issue Aug 12, 2022
Closed
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/571236 mentions this issue: image/jpeg: implement a more forgiving restart marker parser

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/571236 mentions this issue: image/jpeg: implement a more forgiving restart marker parser inspired by Lua libjpeg to fix bad RST marker issues

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/580755 mentions this issue: std/jpeg: ignore garbage bytes before a RST marker

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. FixPending Issues that have a fix which has not yet been reviewed or submitted. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 22, 2024
@dmitshur dmitshur modified the milestones: Backlog, Go1.23 Apr 22, 2024
@gunnsth gunnsth mentioned this issue Aug 6, 2024
Closed
@DaCHack DaCHack mentioned this issue Oct 5, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
FixPending Issues that have a fix which has not yet been reviewed or submitted. NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.23
Development

Successfully merging a pull request may close this issue.

image/jpeg: skip consecutive 0xff bytes shouldend/go
4 participants
@dmitshur @gopherbot @haoyang-zheng @moosepants