New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault at import #36

Open
gmelchett opened this Issue Oct 16, 2018 · 10 comments

Comments

3 participants
@gmelchett
Copy link

gmelchett commented Oct 16, 2018

A fresh docker install (commit 6844c5f) crashed
when importing about 3000 photos.

Moving main jpg file "Camera/IMG_20170425_062158425.jpg" to "/Photos/Originals/2017/04/20170425_062158_2343698269B6.jpg"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x52fbd9]

goroutine 1 [running]:
github.com/RobCherry/vibrant.NewPaletteBuilder(0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/!rob!cherry/vibrant@v0.0.0-20160904011657-0680b8cf1c89/palette.go:202 +0x29
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).GetColors(0xc0003d7b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/colors.go:33 +0xcf
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexMediaFile(0xc000410da0, 0xc0003d7b80, 0x1, 0x1)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:92 +0x1e81
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexRelated(0xc000410da0, 0xc0003d7ae0, 0xc0003d7ae0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:231 +0x9b
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory.func1(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0x0, 0x0, 0x1a, 0xc0007a76c0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:104 +0x869
path/filepath.walk(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:358 +0x41c
path/filepath.walk(0xc0004203e0, 0x15, 0xecac40, 0xc0003cf930, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.walk(0xc000026047, 0xe, 0xecac40, 0xc0003cf380, 0xc00070ca20, 0x0, 0x30)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.Walk(0xc000026047, 0xe, 0xc00070ca20, 0x30, 0x28)
        /usr/local/go/src/path/filepath/path.go:404 +0x105
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory(0xc00070c9f0, 0xc000026047, 0xe)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:38 +0xb5
github.com/photoprism/photoprism/internal/commands.importAction(0xc00046e160, 0xeb9000, 0x33)
        /go/src/github.com/photoprism/photoprism/internal/commands/import.go:35 +0x307
github.com/urfave/cli.HandleAction(0xc87100, 0xdfb058, 0xc00046e160, 0xc0003fc100, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0xda5f8c, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0xdaca99, 0xe, 0x0, ...)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/command.go:210 +0x990
github.com/urfave/cli.(*App).Run(0xc0004544e0, 0xc00000c080, 0x2, 0x2, 0x0, 0x0)
        /go/pkg/mod/github.com/urfave/cli@v1.20.0/app.go:255 +0x687
main.main()
        /go/src/github.com/photoprism/photoprism/cmd/photoprism/photoprism.go:29 +0x2d7
@lastzero

This comment has been minimized.

Copy link
Member

lastzero commented Oct 16, 2018

Looks like an issue with color indexing, see #7

Guess when github.com/RobCherry/vibrant is gone, the issue will be gone...

@gmelchett

This comment has been minimized.

Copy link

gmelchett commented Oct 17, 2018

Thanks for having a look! I tried to import only the photo that caused the segfault in a new docker instance, and the crash occurred again.
I don't know if it is of any help, but here is a link to the photo that triggers this bug http://lithops.se/IMG_20170425_062158425.jpg (Original, ~10 MB large)

@u5surf

This comment has been minimized.

Copy link

u5surf commented Oct 18, 2018

@lastzero , @gmelchett
I tested @gmelchett 's image on vibrant library.
And I figured out the following issue of go image library that some special jpg cannot decode.
golang/go#10447

reproducing
wget http://lithops.se/IMG_20170425_062158425.jpg -O sample.jpg
and following code runs, then it can be output invalid JPEG format: short Huffman data

package main

import (
  "fmt"
  "image"
  _ "image/jpeg" // "net/http"
  "os"

  "github.com/RobCherry/vibrant"
)

func main() {
  file, _ := os.Open("sample.jpg")
  decodedImage, _, err := image.Decode(file)
  if err != nil {
    fmt.Println(err)
  }
  palette := vibrant.NewPaletteBuilder(decodedImage).Generate()
  fmt.Println(palette)
}

result:

$go run main.go
invalid JPEG format: short Huffman data
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10c8329]

goroutine 1 [running]:
github.com/RobCherry/vibrant.NewPaletteBuilder(0x0, 0x0, 0x1)
        /Users/yugo-horie/go/src/github.com/RobCherry/vibrant/palette.go:202 +0x29
main.main()
        /Users/yugo-horie/vibrant_test/main.go:26 +0xc7
exit status 2

But also I think it's better to be handling those errors properly.

@lastzero

This comment has been minimized.

Copy link
Member

lastzero commented Oct 18, 2018

Would https://github.com/disintegration/imaging also be affected? This is what we're using for resizing etc...

@u5surf

This comment has been minimized.

Copy link

u5surf commented Oct 18, 2018

I checked github.com/disintegration/imaging is affected because
image.Decode is called by imaging.Open(m.filename, imaging.AutoOrientation(true)) in your code.
i.e.

img, err := imaging.Open(m.filename, imaging.AutoOrientation(true))

but it maynot cause segfalt if you care failed safe the error that is catches next statements.

@lastzero

This comment has been minimized.

Copy link
Member

lastzero commented Oct 18, 2018

So ultimately that means we need to replace "image/jpeg" or we can not import certain images as the issue was closed by the golang team. Any opinions / alternatives?

@gmelchett

This comment has been minimized.

Copy link

gmelchett commented Oct 18, 2018

Looking at golang/go/issues/10447 it looks like the cause could be that the image is truncated, and indeed my photo is truncated. In Firefox (62), the bottom lines are white, while in Gimp (2.10) only the down right corner isn't correct.
The photos is taken by a Qualcomm based mobile phone with stock Android. (Moto X Force)

@lastzero

This comment has been minimized.

Copy link
Member

lastzero commented Oct 20, 2018

So ignoring it would be OK until we find a better solution?

@gmelchett

This comment has been minimized.

Copy link

gmelchett commented Oct 20, 2018

Yes, it would be OK. However, it would be nice it photoprism didn't crash upon such images. Ignoring them would be prefered, maybe copy/move these faulty images to a special folder with faulty images?

@lastzero

This comment has been minimized.

Copy link
Member

lastzero commented Oct 21, 2018

Of course it shouldn't crash later, but for now this is what we want. Fail fast. Otherwise, nobody would have complained and it might not have gotten any attention. Now we know there is a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment