Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import: Errors when files become unavailable while indexing #261

Closed
rus-ik opened this issue Feb 28, 2020 · 2 comments
Closed

Import: Errors when files become unavailable while indexing #261

rus-ik opened this issue Feb 28, 2020 · 2 comments
Assignees
Labels
bug Something isn't working released Available in the stable release

Comments

@rus-ik
Copy link

rus-ik commented Feb 28, 2020

Importing big collection from SMB mounted share, SMB service has been restarted, photoprism container crashed, here is tail of 'docker logs photoprism_photoprism_1':
(a question is 'Shouldn't be there something more reliable than just io.Copy?')

....
panic: runtime error: slice bounds out of range [:4294967287] with capacity 32768

goroutine 6492 [running]:
io.copyBuffer(0x7f24c1796520, 0xc035754060, 0x26cb4e0, 0xc00255f728, 0xc001d42000, 0x8000, 0x8000, 0x7, 0x23dd2e3, 0xa)
/usr/local/go/src/io/io.go:404 +0x378
io.Copy(...)
/usr/local/go/src/io/io.go:364
github.com/photoprism/photoprism/pkg/fs.Checksum(0xc027500e40, 0x39, 0x0, 0x0)
/go/src/github.com/photoprism/photoprism/pkg/fs/hash.go:46 +0x13b
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).Checksum(...)
/go/src/github.com/photoprism/photoprism/internal/photoprism/mediafile.go:252
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).CanonicalName(0xc0003d1440, 0x0, 0xc00eb50a80)
/go/src/github.com/photoprism/photoprism/internal/photoprism/mediafile.go:220 +0xef
github.com/photoprism/photoprism/internal/photoprism.(*Import).DestinationFilename(0xc0516bb5c0, 0xc0003d1440, 0xc032200fc0, 0x7, 0x0, 0x0, 0x0)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:190 +0x59
github.com/photoprism/photoprism/internal/photoprism.importWorker(0xc04f653e60)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import_worker.go:43 +0x168
github.com/photoprism/photoprism/internal/photoprism.(*Import).Start.func1(0xc04f653e60, 0xc053b4b6f0)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:74 +0x2b
created by github.com/photoprism/photoprism/internal/photoprism.(*Import).Start
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:73 +0x23f
panic: runtime error: slice bounds out of range [:4294967287] with capacity 32768

goroutine 6495 [running]:
io.copyBuffer(0x7f24c1796520, 0xc047ca7c60, 0x26cb4e0, 0xc029e7f510, 0xc001282000, 0x8000, 0x8000, 0x7, 0x23dd2e3, 0xa)
/usr/local/go/src/io/io.go:404 +0x378
io.Copy(...)
/usr/local/go/src/io/io.go:364
github.com/photoprism/photoprism/pkg/fs.Checksum(0xc020e2bd80, 0x39, 0x0, 0x0)
/go/src/github.com/photoprism/photoprism/pkg/fs/hash.go:46 +0x13b
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).Checksum(...)
/go/src/github.com/photoprism/photoprism/internal/photoprism/mediafile.go:252
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).CanonicalName(0xc032200b40, 0x0, 0xc0248aac80)
/go/src/github.com/photoprism/photoprism/internal/photoprism/mediafile.go:220 +0xef
github.com/photoprism/photoprism/internal/photoprism.(*Import).DestinationFilename(0xc0516bb5c0, 0xc032200b40, 0xc04b8fe240, 0x7, 0x0, 0x0, 0x0)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:190 +0x59
github.com/photoprism/photoprism/internal/photoprism.importWorker(0xc04f653e60)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import_worker.go:43 +0x168
github.com/photoprism/photoprism/internal/photoprism.(*Import).Start.func1(0xc04f653e60, 0xc053b4b6f0)
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:74 +0x2b
created by github.com/photoprism/photoprism/internal/photoprism.(*Import).Start
/go/src/github.com/photoprism/photoprism/internal/photoprism/import.go:73 +0x23f

@graciousgrey graciousgrey added the bug Something isn't working label Apr 21, 2020
@lastzero
Copy link
Member

No big surprise, sure PhotoPrism should survive eventually. Would you consider this a "must have" for a first release?

@lastzero lastzero added the low-priority Everything nice to have but not so important label Nov 15, 2020
@lastzero lastzero changed the title runtime error on copying file from mounted SMB share Indexer: Errors when files become unavailable while indexing Dec 5, 2020
@lastzero lastzero self-assigned this Jan 8, 2021
@lastzero
Copy link
Member

lastzero commented Jan 9, 2021

Nothing really bad happens when files are going while indexing (these errors are to be expected):

files-gone

Now taking a look at your stack trace... seems to happen while copying files from import to originals, so not really an indexing issue but a general file system copy / move error. Best we can probably do is display a nicer message and stop importing gracefully.

@lastzero lastzero changed the title Indexer: Errors when files become unavailable while indexing Import: Errors when files become unavailable while indexing Jan 9, 2021
@lastzero lastzero added please-test Ready for acceptance test and removed low-priority Everything nice to have but not so important labels Jan 9, 2021
@lastzero lastzero removed the please-test Ready for acceptance test label Jan 19, 2021
@lastzero lastzero added the released Available in the stable release label Oct 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working released Available in the stable release
Projects
Status: Release 🌈
Development

No branches or pull requests

3 participants