Manual entry option during import

[colin-scott]

My use case for beets is simple: I'm a DJ using beets to help me figure out which songs to mix together. I primarily use flexattrs to label songs according to their "flavor", then use echonest_tempo to pull in bpm info.

I have many single tracks are not in the MusicBrainz database (e.g. live DJ sets), and often have odd filenames.

What I would like is a way to fall back to manual data entry during import: when no MusicBrainz entry exists, and the fromfilename plugin fails to figure out the artist/title, I would like to just manually enter the artist and the title. I don't really care about the other metadata, nor do I care about directory restructuring.

Is this already possible? I spent a fair amount of time digging through the docs, but wasn't able to find anything. Right now many of my tracks are getting added as blank artist/titles, and I am not able to find them in queries.

I spent a bit of time thinking about how I would build this as a plugin, but it seems to me that beets generally uses plugins to help find MusicBrainz entries. It's not clear to me that plugins are a clean way to handle the case when no MusicBrainz entry exists.

For what it's worth, I also tried contributing my entries directly to MusicBrainz, but found that there is a peer-review process before entries are officially added.

Thanks,
Colin

[sampsyo]

Yes, a manual data entry option would be great; and no, there isn't a perfect plugin hook for this yet. Designing the right plugin interface to enable this is the first step.

Related tickets: #154 proposes another manual step during import, and #164 proposes a command for editing tags directly. These should all probably be implemented with the same underlying editing interface.

[colin-scott]

Why do we need a plugin hook for this? Suppose we did the following:

in ui/commands.py, whenever no match is found, add an option [M]anual entry. If the user presses M, then a prompt for Artist, Title, and Track appears, and the code populates an Item object based on the responses.

Plugins seems useful to me if we forsee potential use-cases that we haven't thought of yet. But in this case, it's not clear to me that there are potential use-cases beyond manual entry for the "No match found" event.

[sampsyo]

You're absolutely right; this may not cry out for a new plugin hook—(I only brought up the possibility since you mentioned you went looking for one and didn't find a suitable one. I think this could fit neatly into core beets as well.

And that workflow also sounds about right. As an extension, what about entering track titles? Would that ever be useful? How about track numbers? Along those lines, can we use a similar interface to choose a track order when applying metadata matches?

[boontdustie]

Just started checking out beets to use at Hollow Earth Radio! I, too, could use this feature. We have a lot of music that is not in the MusicBrainz database database and it would be great to be able to clean it up during import.

[kcrwfrd]

It would be really nice to be able to edit the tags as-is, or as returned by MusicBrainz, on both a per-album and per-track granularity during import.

[justmytwospence]

Is anybody working on this at the moment?

[fohrums]

My Recommendations for manual editing imported tracks

A) the use of fromfilename plugin (although I can't get it to work)
B) simple option to just 'edit'

Option A
Upon importing a track (with option to skip matching from sources, because there isn't any), this plugin will provide you the option to set the format-pattern to define which goes where.

Because fromfilename plugin doesn't have any 'configuration' option like other plugins have, there isn't a way to verify the format used within the filename to embed the tags from.
But if you look at the beetsplug/fromfilename.py it has sections for field
extraction patterns for which could possibly define the format-pattern in this following scenario:

File (w/out any tags):

1. Mack 187 & Akoza - Introduction.flac

Format:
$track. $artist - $title

Option B
same as above but by not defining the fromfilename this instead will allow you to edit the general tags yourself:
-artist
-title
-album
-track/total
-disc/total
-comment
...etc

as also suggested from #154 @dokterbob recommended an similar 'Apply and edit' option, except mine includes what could be edited and that it should be up to the user to define what they want to have the option to edit from.

Update

please see issue #1400 (as it's now a seperate issue)

[sampsyo]

@fohrums Hmm; interesting idea. I'm not sure it will be as simple as entering a template like that (since parsing is more complicated/ambiguous than templating), but it's worth considering. Even if it's just in the config file and not interactive.

It's a separate issue, though. Could you open a new ticket, please?

[reynhout]

I have a need that seems to be a special case of this issue.

Briefly, "artist" and "album" from the Musicbrainz database can represent many distinct releases, a few of which might exist in my library. Examples are singles and EPs with the same title as the full-length, foreign releases, remastered releases, etc.

Using the Musicbrainz data to re-tag these files with the same album titles will then confuse media players which rely on that tag pair to group songs. iTunes is the major offender.

So I want to be able to manually override the Musicbrainz album title (when a duplicate is identified) to something of my choice (e.g. "Album Title (EP)" or "Album Title (2005 remaster)", etc.

More details at #1438, but that's the gist.

My solution might not advance the primary goals of this issue, but I'll update this ticket when I get it working.

[reynhout]

This quick-hack version of this code was simpler than I expected. There might be something useful here:

reynhout@b1dfdca

This adds the ability to [O]verride matched Musicbrainz data (presently only album title) before writing. The same option is also presented when a database duplicate is detected.

PS: "[O]verride" is sketchy at best, from a UI perspective. Unfortunately, [E]dit, [M]odify, [R]etitle, re[T]itle, etc are all unavailable due to preexisting options. If any of this stuff is reused, more thought should be put into the nomenclature.

[addisonamiri]

Would it make more sense and/or be easier to implement creating a temporary yaml file with the current file's tags and opening it in the user's editor? Then when they save it beets can scan that file for changes and apply them. This is similar to taskwarrior's edit if anyone has used that. I attached a screenshot for reference.

[sampsyo]

Yes! This is exactly my hope for this feature—and for other cases where manual entry of complex information is needed.

[dokterbob]

Sounds like the ideal solution to me!

[diego-plan9]

Hope I'm not too late to the party, specially after noticing pull request #1687, that has some overlap with this issue and in particular with the yaml editing.

I have been working on a solution to this issue (at diego-plan9/beets) with a slightly different focus than the aforementioned pull request, trying to cater to the cases described on this issues's comments. Summing up, the main goals are:

• allow the user to edit album and track tags as part of the import process, focusing on the "album is not on musicbrainz" and "album is on musicbrainz, but not 100% accurate" cases.
• leave the decision to use a yaml file or the standard command line input to the user.
• provide enough flexibility for advanced users (edit single track, edit all tags, etc) while still providing (hopefully) sane defaults for the most common cases.

stdin vs. yaml handling

uiyaml.py

In the hopes of providing a seamless way of switching between using stdin or yaml for input, the plugin makes use of "batch input": a function that accept a list of "fields" (each one uniquely identified by a tuple), and depending on the user's choice, either prompts him to fill them sequentially (by calling ui.input_()) or generates the temporary yaml file, launching the editor and reading its contents again.

While it is a bit raw (specially, the using of tuples as keys is a bit cumbersome at the moment) it seemed to me like a reasonable compromise for achieving that feature, which might also be useful for other beets components. If useful, there is also room for short-term improvement (having some basic validators, handling of empty/default values, etc), and I have toyed with the idea of including some conditional dependency handling (such as, if field A is X, then skip field B), but seems a bit far-fetched at the moment.

yaml output issues

While I think it would be ideal to achieve something like on the screenshot posted by @addisonamiri, it seems that the yaml module currently used by beets has some limitations. I'm not sure if all of them could be overcome by tweaking a Dumper, in particular, comment support and some display issues such as tabulation of fields. I'm not sure if it is reasonable to expect that all users will be yaml-savvy enough to edit a yaml file without causing it to be invalid, nor if introducing a dependency on another yaml package (such as ruamel.yaml) would be justified.

General design

init.py

The task on the plugin consist roughly of the following steps:

1. copy the candidate selected by the user (task.match) or create a new AlbumMatch candidate based on task.items
2. perform some Diagnostics on the candidate, fixing some common issues
3. prompt the user for input on what to edit (or apply/discard)
4. read user input and push changes to candidate
5. GOTO 2

The rationale behind using an AlbumMatch as the central object is due to trying to conceptually mimic the importer default behaviour (ie. match a bunch of files to a MusicBrainz album), plus taking advantage of existing functionality (show_change(), distances, using TrackInfo fields instead of the more verbose Items, etc).

Diagnostics

diagnoser.py

Although it was not discussed on this issue, I have also included some facilities for performing diagnostics on a candidate. This feature is still in progress and basically a personal code/ideas playing ground of sorts, and I'm still undecided about its desirability or implementation details. Basically, the rationale was to be able to perform a series of simple tests and fixes, specially oriented towards the "album is not on musicbrainz" (and probably very incomplete filename tags) case, such as:

• automatically populate the medium-related fields
• automatically propagate the album artist to the tracks
• allow the user to retrieve artist musicbrainz ids if not present

They are implemented as Diagnostic subclasses that can be toggled on/off individually.

Other

Please be aware that code is still not apt for review - meaning there are many TODOs on the code, UI ugliness, duplicated functionality, and in general some awkwardness as a result of a combination of experimenting different approaches and not (yet) being fully familiar with the beets codebase and style.

My main goal is to try to find out if this approach sounds reasonable and solid, and would gladly welcome input, suggestions, corrections and debate in the hopes of being able to finally contribute to beets as a developer after being a user for quite some time!

A couple of screenshots for a more visual discussion:

• Stdin, editing album info for an unmatched album with empty tags
  
• Yaml, editing "typical fields" for an unmatched album with existing tags
  

[sampsyo]

Very cool! I'm glad you're working on this too.

Can we join forces with the pull request for the first milestone? The idea would be to get only the bare minimum functionality landed, and then later work on other details. In my view, the bare minimum would have:

• $EDITOR editing
• explicit invocation with a command
• not on-the-fly editing during import (this can come later)
• maybe not even selection of which tags to edit

If we can collaborate on that common functionality to get it ready as soon as possible, it will be easier to see how to proceed with the rest of the functionality.

If you're interested, I can put the current PR in a branch and we can collaborate on it there.

[sampsyo]

Oh, and thanks for the pointer to ruemel.yaml! This looks awesome!

[diego-plan9]

Sure thing, I'd be happy to chime in and help in any way you deem suitable towards the first milestone and hopefully successive ones!

As for ruemel.yaml, I can't personally vouch for it as haven't really used it myself (it is mainly a result of some quick googling, tuned towards searching a solution for the yaml comments issue), but I can look a bit more into it if needed as well.

[sampsyo]

@diego-plan9, there's now a branch here and a pull request at #1706. Can I persuade you to collaborate with us on the first-stage implementation there?

[diego-plan9]

Absolutely, I'm already persuaded! Thanks for creating the branch and the pull request, I'll move there.

[Schweinepriester]

Really great that this is being worked on :)

[diego-plan9]

Now that the basic version of the plugin is available as a command and dust settled a bit, I'm wondering if it is a good time for starting "round two" of improvements to the edit plugin? In particular, I'd personally welcome an interactive implementation (ie. just add a prompt choice for invoking the existing functionality during an interactive import session, hopefully tackling the scenario that gives name to this issue!).

A number of other improvements or issues have been suggested throughout the different issues and pull request, and if some other item is considered more urgent I have no problem in switching priorities - here is a (not comprehensive, and not in a particular order) recap:

• Reintroduce the "summary"/comprehension/reduce similar fields option implemented by @jmwatte at the first stages of Edit plugin #1706.
• Add more tests.
• Improve the YAML generation (prettify, user-friendly, robustness).
• Provide a "series of prompts" mode as an alternative to the "edit yaml externally" mode. Manual entry option during import #396 (comment)
• Provide easy access to a number of seemingly common scenarios instead of fully editing the album (swap tracks, rename just album name, ...)?
• ...

[sampsyo]

Great! Let's do it. In my view, we can now attack each new idea independently---no need for an omnibus "edit v2" update.

Let's certainly do the importer-prompt thing. Thanks to your efforts with the new plugin hook, @diego-plan9, this should be straightforward.

Beyond that, tests would obviously be a great idea, but I don't have a strong feeling about the priorities among the other possible features.

And as an aside: #154, the special case where you want to change the order of tracks in a given proposed match, is an often-requested feature that could dovetail with this.

[sampsyo]

Finally implemented, thanks to @diego-plan9! Barely in time for @colin-scott to get his Ph.D. 😃