beet move moves all files in library, every time. beet update finds new metadata every time

[thevorpalblade]

I'm not sure if this is one issue or two, but it feels like one so I'm going to take a chance here.

Every time I issue the command beet update, the same metadata changes are made.
A Short sample:

$ beet stats
Tracks: 42211
Total time: 23.5 weeks
Approximate total size: 624.8 GiB
Artists: 5435
Albums: 3762
Album artists: 1798

$ beet move
Moving 42211 items.

Note, that all files are "moved", when using the -vv option, it appears that
most of the time is being set journaling the moves to the database.

$ beet update
Various Artists - Rolling Stone Top 500 - B.B. King / The Thrill Is Gone
  rg_album_gain: -5.6 -> -6.9
  albumartist: Various Artists -> B.B. King
  comp: True -> False
  label:  -> Mca
Various Artists - Rolling Stone Top 500 - The Beach Boys / Caroline, No
  mb_albumid:  -> c4d98de5-8365-444a-8ba7-420459ad63cf
  rg_album_gain: -5.6 -> -6.9
  mb_albumartistid:  -> MusicBrainz Album Artist Id
  albumstatus:  -> official
  albumartist: Various Artists -> 
  comp: True -> False
  albumtype:  -> compilation
Various Artists - Rolling Stone Top 500 - Neil Young / Rockin\' In The Free World
  language:  -> English
Various Artists - Rolling Stone Top 500 - Lou Reed / Walk On The Wild Side
  rg_album_gain: -5.6 -> -0.6
  rg_album_peak: 1.4 -> 1.0
  albumartist: Various Artists -> Lou Reed
<snip>

This is particularly disturbing because the metadata changes being made are not always correct, for example, Rolling Stone Top 500 is very much a compilation, so it's surprising to see the comp flag being set to false by beet update.

Here's my config.yaml:

directory: /usr/data/Music_beets
library: /usr/data/musiclibrary.blb

plugins:
    inline bpd fromfilename mbsync replaygain convert duplicates missing chroma #discogs 

replaygain:
    backend: gstreamer

import:
    write: yes
    copy: yes
    autotag: yes
    resume: ask
    incremental: yes
#    none_rec_action: asis
#    quiet_fallback: asis
    log:    /home/matthew/.beets/import.log

per_disc_numbering: yes

paths:
    default: $albumartist/$album%aunique{}/%if{$multidisc,Disc $disc/}$track $title
    singleton: Non-Album/$artist/$title
    comp: Compilations/$album%aunique{}/%if{$multidisc,Disc $disc/}$track $title

item_fields:
    multidisc: 1 if disctotal > 1 else 0


paths:
    genre::[bB]ook: /usr/data/Audiobooks/

[sampsyo]

Since this is such a complex issue, can you please put together a small reproducible case? If there's something we can run on our own machines, we might have a shot at investigating the problem.

[thevorpalblade]

Sure!
The following test case reproduces the move bug, but does not seem to catch the update bug, so that probably means they are separate.

Make a directory at /usr/data/Music_beets, with 755 permissions
make sure /usr/data is readable and writable as whatever user beets is running as

Using the config file above, run an import on a single flac file (I can provide the one I used, I don't think it should be picky). I had to also create ~/.beets at this point, so the log file could be written.

After this, beet -vv move gives:

Sending event: pluginload
inline: adding item field multidisc
library database: /usr/data/musiclibrary.blb
library directory: /usr/data/Music_beets
Sending event: library_opened
Moving 1 item.
moving: /usr/data/Music_beets/__/00.flac
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: cli_exit
[539][matthew.stargazer: /

Even when done repeatedly.

[sampsyo]

Ah! This is just a case of misleading output. "Moving N items" actually means "I'm going to try to move N items." The same is true of that "moving:" output in verbose mode. Nothing actually happens if the file is in the right place already.

#1583 proposes to fix this.

[thevorpalblade]

Interesting!
beet move takes a fairly long time to finish, even when it's a null operation, it seems like.
Is it really looping through each individual file and checking if it needs to be moved? I suppose
it's an uncommon enough action that it's not a terrible thing for it to be slow.

OK, I'll see if I can reproduce the beet update component of this bug report in a smaller test environment. I'll keep you posted.

[sampsyo]

Yep, that's exactly it. The algorithm goes like this: fetch a track from the database, generate its "destination path," check whether it matches the current path, and move if not. Repeat.

And awesome; thanks for looking into the update issue.

[sampsyo]

Alright, it seems like we have this half sorted out. Matt, please reopen if the update nonsense reappears reliably.