Can't Remove a File Path from the Beets DB

[Stunner]

Problem

I have an erroneous file path that is stuck in the beets db:

$ beet stats -e
could not get size of /Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.1.mp3: [Errno 2] No such file or directory: '/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.1.mp3'
Tracks: 8990
Total time: 2.6 weeks (1562021.03 seconds)
Total size: 45.3 GiB (48671997103 bytes)
Artists: 492
Albums: 938
Album artists: 389
$

Running this command in verbose (-vv) mode:

$ beet -vv stats -e
user configuration: /Users/aaron/.config/beets/config.yaml
data directory: /Users/aaron/.config/beets
plugin paths: 
Sending event: pluginload
inline: adding item field initial
inline: adding item field disc_and_track
library database: /Volumes/Drobo Vol 1/Music/beets/musiclibrary.db
library directory: /Volumes/Drobo Vol 1/Music/beets/music
Sending event: library_opened
could not get size of /Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.1.mp3: [Errno 2] No such file or directory: '/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.1.mp3'
Tracks: 8990
Total time: 2.6 weeks (1562021.03 seconds)
Total size: 45.3 GiB (48671997103 bytes)
Artists: 492
Albums: 938
Album artists: 389
Sending event: cli_exit
$

An attempt at removing the non-existant file path with beet remove doesn't work:

$ beet -vv remove '/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3'
user configuration: /Users/aaron/.config/beets/config.yaml
data directory: /Users/aaron/.config/beets
plugin paths: 
Sending event: pluginload
inline: adding item field initial
inline: adding item field disc_and_track
library database: /Volumes/Drobo Vol 1/Music/beets/musiclibrary.db
library directory: /Volumes/Drobo Vol 1/Music/beets/music
Sending event: library_opened
error: No matching items found.
$

Nether does a beet update:

$ beet -vv update Misconduct
user configuration: /Users/aaron/.config/beets/config.yaml
data directory: /Users/aaron/.config/beets
plugin paths: 
Sending event: pluginload
inline: adding item field initial
inline: adding item field disc_and_track
library database: /Volumes/Drobo Vol 1/Music/beets/musiclibrary.db
library directory: /Volumes/Drobo Vol 1/Music/beets/music
Sending event: library_opened
Sending event: database_change
Misconduct -  - 
  length: 1:57 -> 3:13
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: database_change
skipping /Volumes/Drobo Vol 1/Music/beets/music/T/The Almighty Trigger Happy/Misconduct _ The Almighty Trigger Happy [1201]/08 Broncitis.mp3 because mtime is up to date (1485557008.0)
skipping /Volumes/Drobo Vol 1/Music/beets/music/T/The Almighty Trigger Happy/Misconduct _ The Almighty Trigger Happy [1201]/09 Unjustified.mp3 because mtime is up to date (1485557008.0)
skipping /Volumes/Drobo Vol 1/Music/beets/music/M/Misconduct/Signed in Blood/05 The Struggle.mp3 because mtime is up to date (1485557631.0)
skipping /Volumes/Drobo Vol 1/Music/beets/music/M/Misconduct/Signed in Blood/06 Fading.mp3 because mtime is up to date (1485557632.0)
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
moving album 720
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: before_item_moved
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/beets-1.4.4-py2.7.egg/beets/util/__init__.py", line 470, in move
    shutil.copyfile(path, dest)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3'

Error: No such file or directory while moving /Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3 to /Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3

You'll need a file with almost no meta data in it, add it to the library and then move it (a mere rename seems like it would do the trick too) without notifying beets of the move. (At least I think that is what happened.)

$ beet ls -f '$albumartist <=> $artist <=> $album <=> $track <=> $title' /Volumes/Drobo\ Vol\ 1/Music/beets/music/_/Misconduct 
Misconduct <=> Misconduct <=>  <=> 00 <=> 
Misconduct <=> Misconduct <=>  <=> 00 <=> 
Misconduct <=> Misconduct <=>  <=> 00 <=> 
$ ls /Volumes/Drobo\ Vol\ 1/Music/beets/music/_/Misconduct 
00.1.mp3	00.2.mp3	00.4.mp3	00.5.mp3	00.mp3
$

Here is a link to download an archive of the directory of interest: https://dl.dropboxusercontent.com/u/17804136/Misconduct.zip

Setup

• OS: 10.11.6
• Python version: 2.7.10
• beets version: https://github.com/Stunner/beets/tree/develop
• Turning off plugins made problem go away (yes/no): Not applicable

My configuration (output of beet config) is:

$ beet config

paths:
    default: $initial/$artist/$album%aunique{}/$disc_and_track $title
convert:
    auto: yes
    command: ffmpeg -i $source -y -vn -aq 2 $dest
    extension: mp3
    dest: /Volumes/Drobo Vol 1/Music/beets/music
    tmpdir: /Volumes/Drobo Vol 1/Music/beets/tmp_convert
    never_convert_lossy_files: no
    album_art_maxwidth: 0
    format: mp3
    threads: 4
    copy_album_art: no
    max_bitrate: 500

    paths: {}
    pretend: no
    quiet: no
    formats:
        alac:
            command: ffmpeg -i $source -y -vn -acodec alac $dest
            extension: m4a
        aac:
            command: ffmpeg -i $source -y -vn -acodec libfaac -aq 100 $dest
            extension: m4a
        opus: ffmpeg -i $source -y -vn -acodec libopus -ab 96k $dest
        mp3: ffmpeg -i $source -y -vn -aq 2 $dest
        flac: ffmpeg -i $source -y -vn -acodec flac $dest
        ogg: ffmpeg -i $source -y -vn -acodec libvorbis -aq 3 $dest
        wma: ffmpeg -i $source -y -vn -acodec wmav2 -vn $dest
    embed: yes
item_fields:
    initial: "strippedartist = ''.join(ch for ch in artist if ch.isalnum()) or '_' \nif strippedartist[0].isdigit():\n    return '#'\nreturn strippedartist[0].upper()\n"
    disc_and_track: u'%02i.%02i' % (disc, track) if disctotal > 1 else u'%02i' % (track)
library: /Volumes/Drobo Vol 1/Music/beets/musiclibrary.db
fetchart:
    auto: yes
    enforce_ratio: yes
    sources: coverart itunes amazon albumart wikipedia google
    google_key: REDACTED
    minwidth: 0
    google_engine: 001442825323518660753:hrh5ch1gjzm
    cautious: no
    maxwidth: 0
    store_source: no
    fanarttv_key: REDACTED
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
spotify:
    mode: list
    regex: []
    artist_field: albumartist
    track_field: title
    album_field: album
    region_filter:
    show_failures: no
    tiebreak: popularity
chroma:
    auto: yes
embedart:
    auto: yes
    compare_threshold: 0
    ifempty: no
    remove_art_file: no
    maxwidth: 0
copyartifacts:
    extensions: .*
    print_ignored: yes

plugins: mbsubmit embedart fetchart chroma fromfilename badfiles web copyartifacts inline spotify mbsync edit info convert
directory: /Volumes/Drobo Vol 1/Music/beets/music

import:
    move: yes
    log: /Volumes/Drobo Vol 1/Music/beets/beets-import.log
acoustid:
    apikey: REDACTED
mbsubmit:
    format: $track. $albumartist <=> $artist <=> $album <=> $title ($length)
    threshold: medium
pathfields: {}
album_fields: {}
edit:
    itemfields: track title artist album
    albumfields: album albumartist
    ignore_fields: id path
web:
    host: 127.0.0.1
    include_paths: no
    port: 8337
    cors: ''

[sampsyo]

That's certainly strange. Does the file show up in beet ls -p?

Also, it's doubly strange that update is crashing—it's supposed to detect missing files, so I don't currently see how that entry could slip through the cracks.

I also notice you have the copyartifacts plugin enabled. Does disabling that have any effect?

[Stunner]

This is what I see with beet ls -p:

$ beet ls -p /Volumes/Drobo\ Vol\ 1/Music/beets/music/_/Misconduct 
/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.mp3
/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.1.mp3
/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3

Interestingly enough there are 5 items in the directory but beets only has a record of 3 of them... I don't know exactly what was done to get it into this state... Possibly renamed them manually or tried to use beet modify... I don't quite recall.

Removing copyartifacts doesn't do anything to resolve this issue:

$ beet version
beets version 1.4.4
Python version 2.7.10
plugins: badfiles, chroma, convert, edit, embedart, fetchart, fromfilename, info, inline, mbsubmit, mbsync, spotify, web
$ beet stats -e
could not get size of /Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3: [Errno 2] No such file or directory: '/Volumes/Drobo Vol 1/Music/beets/music/_/Misconduct/00.3.mp3'
Tracks: 9084
Total time: 2.6 weeks (1578473.28 seconds)
Total size: 46.0 GiB (49343796082 bytes)
Artists: 539
Albums: 949
Album artists: 393

Heh, maybe copyartifacts was the culprit after all? I was fiddling with the outputs to show you more and this happened:

$ beet update Misconduct
Misconduct -  - 
  deleted
$ beet -vv update Misconduct
user configuration: /Users/aaron/.config/beets/config.yaml
data directory: /Users/aaron/.config/beets
plugin paths: 
Sending event: pluginload
inline: adding item field initial
inline: adding item field disc_and_track
library database: /Volumes/Drobo Vol 1/Music/beets/musiclibrary.db
library directory: /Volumes/Drobo Vol 1/Music/beets/music
Sending event: library_opened
Sending event: database_change
Misconduct -  - 
  deleted
Sending event: database_change
Sending event: item_removed
skipping /Volumes/Drobo Vol 1/Music/beets/music/T/The Almighty Trigger Happy/Misconduct _ The Almighty Trigger Happy [1201]/08 Broncitis.mp3 because mtime is up to date (1485557008.0)
skipping /Volumes/Drobo Vol 1/Music/beets/music/M/Misconduct/Signed in Blood/06 Fading.mp3 because mtime is up to date (1485557632.0)
Sending event: database_change
Sending event: database_change
moving album 720
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: before_item_moved
Sending event: item_moved
Sending event: database_change
Sending event: cli_exit
$ beet stats -e
Tracks: 9082
Total time: 2.6 weeks (1578086.60 seconds)
Total size: 46.0 GiB (49340698552 bytes)
Artists: 539
Albums: 949
Album artists: 393

Strange, the problem appears to have disappeared... Something to note additionally is that I repeatedly run beet update and it always outputs that its making changes, but subsequent runs show the exact same output (as seen in the above snippet of terminal output). Thought I'd mention that here, I'll create a new issue reporting that issue, however.

[sampsyo]

Huh, how about that!

The thing about update showing the same changes repeatedly probably has to do with album-level normalization. This is a tricky issue, but there's a little more detail at these links:
https://groups.google.com/forum/#!topic/beets-users/h2r2AkY0Frg
#1837