Incremental import for singletons

[mried]

Hi!

I just found out that the incremental import only works for directories, hence albums. Do you see a chance to enable this feature for singletons (created using the -s command line option for example)?

I had a short look at the code. Wouldn't it enough to store the file names of imported singletons beside the directories (and check them while importing, of course)? Do I overlook something why this isn't possible?

My use case is: The music library is already organised. I want to use beets to build a database and correct some tags and/or file/folder names. This happens 'in place'. Every now and than, I add an album or a singleton. The files are added directly into the library, so there is no special 'incoming' folder. To import them into beets, I run beet import -i <library root>. All already imported directories are skipped, but singletons are imported again every time.

[sampsyo]

Yes! This would be great to add. In fact, it looks like this is halfway implemented: ImportTaskFactory.singleton already contains a check. It seems that ImportTask.save_history should also record individual items when they're imported as singletons. I say go for it!

[mried]

Eh... I must admit, it's already working perfectly 😳 I stumbled over this behaviour while using my autosingleton plugin which didn't respect the incremental setting. After I fixed my plugin, everything is fine.

No, wait! There's something strange: While digging through the code and inspecting the state.pickle file, I noticed that the original file and folder names are stored. What if the files are moved during the import? They will be imported a second time. After the second import, they won't be imported any more. Shouldn't be the new file name stored here?
Another bad thing about storing the original name is: Lets say we import a file a.mp3. It's renamed to b.mp3 while importing. Later, we add a new file to our music library: a.mp3 which would be renamed to c.mp3 while importing. But: a.mp3 will not be imported any more.
While writing this another idea comes into my mind: I guess this behaviour is exactly the one you'd expect, isn't it? It's useful if the import source and the musiclibrary are distinct. So, what's needed if the import imports the library directory itself is a way to stop importing files, which are already imported, hence present at the library. I guess, I could use the import_task_created plugin hook to implement this. Any ideas?

[sampsyo]

OK, cool—I'm glad this is working.

That's intentional. The incremental mode needs to detect already-imported files before resolving the match and adding to the library—we need to catch them on the way in, rather than on the way out of the import process. (The ordinary duplicate detection stuff plays that role.)

Recall that incremental mode tracks the full paths—which is the best we can do. There's no more reliable way to track moved but identical files short of storing checksums.

Your idea about importing files that are not already in the library sounds good. Sort of the opposite of beet import -L, which imports everything that is already in the library. Worth prototyping!