Add option to force full rescan

[nvllsvm]

What
Add an option to force a full rescan. In other words, rescan all files even when the file modified timestamp has not changed.

Why
It's possible for a file's contents to change without having it's modified timestamp updated. For example, my harmonize tool makes use of this in order to detect when a file needs to be reencoded.

Additional Information
My initial Gonic library scanned my Opus 192kbps collection. I later reencoded the collection as Opus 128kbps. The file paths and modified timestamps were unchanged.

After a library rescan, I noticed that album titles were missing. Note that the last row has a title. All albums except for the last row were added to my Gonic library prior to reencoding it.

sqlite> select * from albums where parent_id = 2578;
id          updated_at                           modified_at                         left_path   right_path                             right_path_u_dec  parent_id   cover       tag_artist_id  tag_title   tag_title_u_dec  tag_brainz_id  tag_year  
----------  -----------------------------------  ----------------------------------  ----------  -------------------------------------  ----------------  ----------  ----------  -------------  ----------  ---------------  -------------  ----------
2579        2019-12-02 00:56:01.796632971+00:00  2019-12-01 23:00:51.10143354+00:00  Zero Cult/  Zero Cult - Clouds Garden-2010 (FLAC)                    2578        cover.jpg   348                                                        0         
2580        2019-12-02 00:56:01.798885783+00:00  2019-12-01 23:00:40.711482557+00:0  Zero Cult/  Zero Cult - Closer Than Ever (2016) [                    2578        cover.jpg   348                                                        0         
2581        2019-12-02 00:56:01.803979479+00:00  2019-12-01 23:00:21.041575276+00:0  Zero Cult/  Zero Cult - Aleph (2014) FLAC-Web                        2578        cover.jpg   348                                                        0         
2582        2019-12-02 00:56:01.806533632+00:00  2019-12-01 23:01:03.921373022+00:0  Zero Cult/  Zero Cult - Excort (Unicorn Music, EP                    2578        cover.jpg   348                                                        0         
2583        2019-12-02 00:56:01.805637216+00:00  2019-12-01 23:01:14.571322713+00:0  Zero Cult/  Zero Cult - Ikebana                                      2578        cover.jpg   348                                                        0         
2584        2019-12-02 00:56:01.811363163+00:00  2019-12-01 23:01:31.05124481+00:00  Zero Cult/  Zero Cult - Shining (2012) FLAC-Web                      2578        cover.jpg   348                                                        0         
2585        2019-12-02 00:56:01.808944789+00:00  2019-12-01 23:00:30.161532297+00:0  Zero Cult/  Zero Cult - Art of Harmony                               2578        cover.jpg   348                                                        0         
2586        2019-12-02 00:56:01.810638947+00:00  2019-12-01 23:01:45.371177063+00:0  Zero Cult/  Zero Cult - Vacuum (Cosmicleaf Rec. 2                    2578        cover.jpg   348                                                        0         
2587        2019-12-02 00:56:01.792687937+00:00  2019-12-01 23:01:23.861278809+00:0  Zero Cult/  Zero Cult - Inside Out (2015) FLAC-We                    2578        cover.jpg   348                                                        0         
2588        2019-12-02 00:56:01.803065489+00:00  2019-12-01 23:01:03.23137628+00:00  Zero Cult/  Zero Cult - Dreams in Stereo                             2578        cover.jpg   348                                                        0         
2589        2019-12-02 00:56:01.800880099+00:00  2019-12-01 23:01:55.071131142+00:0  Zero Cult/  Zero Cult - Where Rivers Have No Name                    2578        cover.jpg   348                                                        0         
2590        2019-12-02 00:56:01.807273372+00:00  2019-12-01 23:01:27.601261124+00:0  Zero Cult/  Zero Cult - Tangoa (2018)                                2578        cover.jpg   348                                                        0         
2591        2019-12-02 00:56:01.793452163+00:00  2019-12-01 23:00:20.621577255+00:0  Zero Cult/  Zero Cult - Absolute Zero (2013)                         2578        cover.jpg   348                                                        0         
2592        2019-12-02 00:56:01.795370157+00:00  2019-12-02 00:55:30.080528292+00:0  Zero Cult/  Zero Cult - Pop Insanity (2013) FLAC-                    2578        cover.jpg   348            Pop Insani                                  2013      

[sentriz]

fair point. I could add this to the "gonicscan" binary. also need to document that. thanks!

[alex-phillips]

Any update on this? I think I need to do a re-scan to pick up new album artwork.

[sentriz]

are you using docker? if so could you try sentriz/gonic:full-scan? it adds a -full-scan cli arg to gonicscan
eg.
docker-compose exec gonic gonicscan -full-scan

[alex-phillips]

Ah, nice. When I run the command though, I get the following:

2020/04/24 11:50:39 error opening database: with gorm: database is locked

[sentriz]

ugh thats annoying. thought i fixed that with sqlite journal_mode=WAL, cache=shared etc. in the meantime stopping the gonic server and running the gonicscan again should work

[sentriz]

edit: e966fff should fix that.
i just updated sentriz/gonic:full-scan on docker hub. could you give it a go please?

[alex-phillips]

Just started, looks like it's going along fine! Will this be merged into the main branch / container at some point?

[alex-phillips]

Although it does look like it hits the db issue occassionally during scan (I'm also playing music back while doing this)

2020/04/24 20:36:19 processed folder `Opeth/Heritage`
2020/04/24 20:36:19 processed folder `Opeth/Morningrise`
2020/04/24 20:36:20 processed folder `Opeth/Ghost Reveries`
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
gorm error/src/scanner/scanner.go:384database is locked
2020/04/24 20:36:20 processed folder `Opeth/Sorceress`

[sentriz]

alright that's a pain. think the issue is the fact that two go processes are using the db at the same time. i think the easiest thing to do here is delete the gonicscan program, and just add a "full scan" option to the server's web ui. how would you feel about this?

[alex-phillips]

Sounds good to me!

[sentriz]

cool. v0.8.6 removes gonicscan and adds a full scan button to web ui - with tooltip warning
edit: ignore the commit that closed this, that should have been squashed