Bug: (invalid JPEG format: bad RST marker)

[mbelletti]

What does not work as expected?

Many photos from iphone XSMax perfectly visibles on my mac, and on other devices, when imported in photoprism, gave this error:

index: failed creating thumbnails for IMG_xyzw.jpeg (invalid JPEG format: bad RST marker)

I've Library-> Hidden full of them and not visible at all.

I attach a photo as sample

[lastzero]

see golang/go#40130 and golang/go#41875

[lastzero]

Looks like we need to patch Go for this 🍿

[mbelletti]

I see.
I've also a lot of:

`Missing 0xff00 Sequence'

It's the same bug?

Thank you

[lastzero]

Most likely unless only few files are affected. They might have other issues then or are really broken.

[lastzero]

https://twitter.com/lastzero/status/1452959588815413251

[lastzero]

Waiting for this issue to be fixed in Go. If it takes more than 2 months, we'll have to release our own Go distribution. Remind me!

[mumbledenoise]

Waiting for this issue to be fixed in Go. If it takes more than 2 months, we'll have to release our own Go distribution. Remind me!

It has been 2 months, here's your reminder! :)

[lastzero]

See:

• image/jpeg: skip consecutive 0xff bytes golang/go#41875 (comment)

[mumbledenoise]

See:

* [image/jpeg: skip consecutive 0xff bytes golang/go#41875 (comment)](https://github.com/golang/go/pull/41875#issuecomment-995181753)

The code change seems minimal, I'm shocked they haven't merged it yet. Perhaps you'll need to follow their process and submit your own PR to get this pushed through?

golang/go#41875 (comment)

[lastzero]

Tests are missing and you need to get familiar with the Google internal tools, which we currently don't have time for as we're drowning in work. Help welcome!

[moosepants]

FYI I took a look at the linked golang issue.
Created a test using the attached image which failed as described (bad RST). Thought it was job done...
The submitted code fix (skip consecutive 0xff bytes) did not resolve the issue 😒
It looks like the image has some minor corruption (i.e. extra bytes or missing bytes) and the golang jpeg processor is somewhat (surprisingly?) rudimentary in how it handles this.
I'm going to look into it a bit deeper (the libjpeg code has a good solution) but it may require more rework than I am prepared to put in (as a first time contributor).

As an aside, if there's an example image for the Missing 0xff00 Sequence issue I'd be interested to look at that whilst I have the golang src/build environment set up and my head is in the jpeg decoder. That may be solved by the linked fix 🤞

[moosepants]

@lastzero In case you didn't notice, I submitted a PR on golang that solves the issue for at least the attached test image (and possibly others).

https://go-review.googlesource.com/c/go/+/382754/

[lastzero]

@moosepants Thanks! Much appreciated as I am literally drowning in work 😵‍💫

[mbelletti]

I've found that all my images with Missing 0xff00 Sequence have some glitch.

As the image attached.

[lastzero]

@moosepants From what I can see, a review has been performed and some minor changes have been requested?
👉 https://go-review.googlesource.com/c/go/+/382754/

[moosepants]

Yes, I did see that - it's been on my todo for a while. It's winter here in Australia... a good time to work on code :) Will try to resolve the issues and move it forward. -- Seamus Allan +61 (0) 459 634 898

…

On 02/06/2022 1:33 PM Michael Mayer ***@***.***> wrote: @moosepants https://github.com/moosepants From what I can see, a review has been performed and some minor changes have been requested? 👉 https://go-review.googlesource.com/c/go/+/382754/ — Reply to this email directly, view it on GitHub #1673 (comment), or unsubscribe https://github.com/notifications/unsubscribe-auth/AQPYHTJ2TIJQ65EC4TBGBWLVNATPFANCNFSM5GXPXUSA. You are receiving this because you were mentioned.Message ID: ***@***.***>

[lastzero]

Failed on (x/tools) linux-amd64: https://storage.googleapis.com/go-build-log/0b8005c5/linux-amd64_a29c2471.log

Consult https://build.golang.org/ to see whether they are new failures. Keep in mind that TryBots currently test exactly your git commit, without rebasing. If your commit's git parent is old, the failure might've already been fixed.

[lastzero]

Since we now have a workaround, the priority of this issue has been changed from bug to enhancement.

[lastzero]

@mbelletti With the last commit, both images work for me! New preview build is available for testing soon. Let us know if you find any JPEGs that don't work yet so we can look into it.

[mbelletti]

@lastzero Thank you. I will do

[lastzero]

👉 https://demo.photoprism.app/library/albums/arqfxgfvn6fgunlr/view

[mbelletti]

I uploaded a jpg with a glitch that before gave errors (with Missing 0xff00 Sequence ), now is imported.
Thank you

https://demo.photoprism.app/library/albums/arqh3al2v8t3risn/view