Attachments to items and albums #111

Open
ghost opened this Issue Feb 28, 2013 · 61 comments

Projects

None yet
@ghost
ghost commented Feb 28, 2013

This issue was automatically migrated from Google Code.
Original author: adrian.sampson (November 17, 2010 23:05:05)
Original issue: google-code-export/beets#109

@jonathanthomas83

I wish to add my vote to this, I am a FLAC (100% log & CUE) man, and until I can move log & CUE files with the beets import, I am unable to use beets (at least, without manually moving those files after import).

However, a suggestion on google code worries me slightly. No changes should ever be made to log files, the point of them is to be accurate at the time of the rip. That means they should never be changed, not even if file names change later on.

So making changes to log & CUE sheets aren't an issue for me, nor is storing them in the database. I simply would love to move them along with the rest of the files, upon import. With options would be a great bonus.

Many thanks.

@jonathanthomas83

If there is any way I can help, please let me know. I'm not a Python developer, but if there's anything I can do to help, I will try my very best.

@sampsyo
Member
sampsyo commented Sep 11, 2013

If you do have extra time, it can never hurt to have some help thinking though exactly how the feature will work in the form of a set of requirements. Something like the following process would be helpful:

  • Brainstorm a list of all the possible things a user—not just yourself—might want to do with an attachments-like feature.
  • Prioritize this list roughly. Draw a line above which things seem important enough to address and below which the use cases are more optional.
  • Make a second list describing the set of concrete bits of functionality that beets would need to cover all the use cases.
  • Finally, translate these features into detailed designs for the command-line flags, an config options that form the user interface to the functionality. Write proto-documentation for these notional options to describe what they do.

It's not coding, but requirements design like this can sometimes be more helpful than the code itself! Let me know if you're interested.

@jonathanthomas83

I'm very interested, it'll help me get on my feet with this stuff. A bit of the language you've used is new to me, being a web designer/HTML/CSS monkey!

What are command-line flags? Proto-documentation? And I'll probably need a it of help with "config options that form the user interface to the functionality"... other than that, I get the gist of what you're asking.

I will try my very best to help you out! :-)

@sampsyo
Member
sampsyo commented Sep 11, 2013

Great! I'm happy to clarify; sorry for the use of jargon.

command-line flags: Here I mean the relevant commands and options for the beet CLI program. For example, might we want a beet attach command for adding new files to an album?

config options: Like these.

proto-documentation: Here I was just suggesting that we write up documentation for any commands or config options we invent even though they don't yet exist. Writing documentation before code can help clarify what we want the code to do.

Here's a wiki page to start taking notes in.

@jonathanthomas83

Thank you very much and sorry for the lack of understanding. I appreciate your clarification. I'll get cracking on that and will hopefully help out when I get a spare five minutes. Thank you! :-)

@jonathanthomas83

Little bit of work on the use cases, not a definitive list and will add to it in the coming few days.

@sampsyo
Member
sampsyo commented Sep 12, 2013

Looking good! This is a great start.

I wanted to make one comment on how things are shaping up here: I see the "core" functionality of attachments as being separate from anything we add for dealing with cue files (splitting/joining album FLACs, etc.). The former will become part of beets core; the latter will be a plugin (or possibly multiple plugins) that uses the attachments functionality.

@sampsyo
Member
sampsyo commented Sep 12, 2013

Relevant ticket on a potential cuesheet plugin: #136.

@jonathanthomas83

Totally agree with your above sentiment. It makes sense for the attachments stuff to be core with further cue sheet functionality offered via plugins. I will endeavor to write a little more later today if I get a chance.

@puretokyo

Per my duplicate issue identified in the previous post, I propose that the mechanism be flexible to allow users to specify arbitrary filetypes that could be attached whenever an album is copied or moved. For instance, filetypes could be arbitrarily specified in config.yaml as follows:

attachfiletypes: jpg png log cue nfo m3u
@jonathanthomas83

Sounds good to me. I'm sure that's not meant to be a definitive list but I'd like to toss "pdf" into the mix too! :-)

@alleyoopster

Want to add my vote for this option. I have found recently I am spending a lot of time going through each processed album and moving the extra files from a CD rip (log, cue, m3u, txt, jpg sometimes in a separate directory), to a directory on my collection. This would be a great assett to get this implemented.

I would like to suggest the option of not only being able to add specific file types, but the option of adding everything. Maybe

attachfiletypes: *

I would like to see it remove the directory also on a move, but that is not a major thing.

thanks

@holms
holms commented Nov 3, 2013

I would also vote for attachfiletypes: * because it's a usual case when you have directory name 'Artworks' with full cd scans. Actually there's nice guides on what.cd how a scene organizes music album :) would be handy I think.

@alleyoopster

Has anyone got any workarounds? I have many folders with odd files I need to move to my music collection and looking for some time saving. :)

@jonathanthomas83

No workarounds from me, it's a chore but as soon as I run "beet import" I drag and drop the attachments into the same folder as the music. However, this isn't ideal because it's not going to be associated with the music in the beets database. A couple of days ago I added a section on how "beet attach" should handle these cases to the wiki page...

https://github.com/sampsyo/beets/wiki/Attachments#beets-functionality

@zeltak
zeltak commented Nov 16, 2013

+1 for this issue, i discovered beets today and very anxious to start using it but have thousands of file (lyrics, cue etc) associated with my thousands of albums and dont want to loose it..any update on this?

best

Z

@sampsyo
Member
sampsyo commented Nov 18, 2013

Thanks for your vote of confidence, but we're still working on it. If you'd like to contribute, the wiki for the use cases would be a great place to start.

@zeltak
zeltak commented Nov 18, 2013

Thx!

As requested I added a non technical (i cant code at all unfortunately :) ) use case (see also pasted below at the end of the post):

I would also like to take the time to thank you alot for beets, i just discovered it and it looks amazing. I cant code but would love to continue and help out and contribute in other areas needed.

A quick related question. since it will take some time (any idea how much very roughly..weeks. months?) to implement the move associated files, how do you recommend i proceed with my use? im really dying to start using beets but i am dependent on associated files in my library (artist images, album images, lyrics files etc). if i import currently all my files into beets i will obviously loose these files. since i have thousands of entry's in my music library i cant manually move the associated files. should i just wait for that move feature or is there another clever way of doing this?

Thx

Z

WIKI entry:
Plugin for moving additional files (like cuesheets, scans etc.) into library on import
This would work in the following way: in the config file you would enable the "move additional files" plugin (maybe "filemove" plugin for short) and configure it based on file type. Also it would be very useful to define the level of files, that is the album folder level (move the album folder files to the associated $album folder and the artist level (move the files found in the artist root folder to the $albumartist/$artist) folder. Also a move/copy option and a delete previous/empty folder could also be great in terms of the config file i assume it would look like this plugins: bpd fetchart embedart replaygain mpdupdate fromfilename filemove filemove: type: *.jpg *.png *.lyric *.lyrics level: 2

Then when you start beet import it will move/copy the audio files and arrange them but in addition will move/copy the associated files to the respective locations based on the configs that is: album level files -> album folder artist levelfiles -> artist root folder

@jonathanthomas83

zeltak, see #111 (comment) on how I manually manage attachments, it's not great.

@aristidesfl

+1 to attach everything. I just want all the extras which come with the folder

@ccruzen
ccruzen commented Jan 21, 2014

Very interested to see this feature added. Literally the only thing keeping me from using Beets full time for my music org.

@sbarakat

Has there been any progress on this?

I've just been writing a plugin to do exactly this. What I've got so far is a plugin that copies any non imported files (album artefacts eg. .log, .cue, etc.) to the destination directory on import. It's a bit rough at the moment, but does the job. I then found this feature request and wondered if anyone has done an implementation of it yet.

I'm happy to work on the plugin and share it (I have a need for it, so I'll be doing it anyway). I develop in Python, so this isn't an issue.

@sampsyo
Member
sampsyo commented Feb 13, 2014

We've made progress on the infrastructure level—flexible attributes have come a long way and we've revamped the data model to make this more feasible. But we're not all the way there yet; contributions are still welcome!

And yes, please do publish your plugin! We'll certainly link to it from the docs at the very least.

@jonathanthomas83

Anything I can do to help guys? I think there's a few of us willing
attachments on, if there's anything we can do, let us know! :-)

Sami, well done on getting a plugin sorted! :-)

On 13 February 2014 01:29, Adrian Sampson notifications@github.com wrote:

We've made progress on the infrastructure level--flexible attributes have
come a long way and we've revamped the data model to make this more
feasible. But we're not all the way there yet; contributions are still
welcome!

And yes, please do publish your plugin! We'll certainly link to it from
the docs at the very least.

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-34939334
.

@jonathanthomas83

Sorry, on a related note. Does anyone have any thoughts or feelings about
log files and changing file names via beets...ie. the true spirit of log
files is to maintain everything untouched once the log file has been
created, so a mismatch between file names and the log file after a beet
import kind of defeats the object.

What are people's thoughts on this? I mean, would you still let beets
change file names regardless and still include the log file or would you
deem it all unnecessary...I guess I'm talking to the
audio/FLAC/EAC/pedantic enthusiasts among us.

My approach would be to let beets handle to file names for good structural
integrity, and I won't mind about the log file mismatch, but is that the
best way to go about this?

On 13 February 2014 12:17, Jonathan Thomas jonathanthomas83@gmail.comwrote:

Anything I can do to help guys? I think there's a few of us willing
attachments on, if there's anything we can do, let us know! :-)

Sami, well done on getting a plugin sorted! :-)

On 13 February 2014 01:29, Adrian Sampson notifications@github.comwrote:

We've made progress on the infrastructure level--flexible attributes have
come a long way and we've revamped the data model to make this more
feasible. But we're not all the way there yet; contributions are still
welcome!

And yes, please do publish your plugin! We'll certainly link to it from
the docs at the very least.

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-34939334
.

@aristidesfl

I agree, let beets change the file names, and keep the log file for reference of the original names.

@Hawke
Hawke commented Feb 13, 2014

Yep, from my limited perspective it seems to be the norm to leave the logs alone (due to checksums on EAC logs), regardless of actual file naming

@jonathanthomas83

So is it widely accepted that people will alter the file names then?

On 13 February 2014 16:17, Hawke notifications@github.com wrote:

Yep, from my limited perspective it seems to be the norm to leave the logs
alone (due to checksums on EAC logs), regardless of actual file naming

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-34994691
.

@Hawke
Hawke commented Feb 13, 2014

Yep.

@jonathanthomas83

Cool, thanks for the responses guys, much appreciated.

On 13 February 2014 16:34, Hawke notifications@github.com wrote:

Yep.

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-34996743
.

@sampsyo
Member
sampsyo commented Feb 13, 2014

Thanks for volunteering! Here are a few things that could use contributions:

  • Continue fleshing out the interface design wiki page. There's a good start there but it could really use some love to turn this into an awesome API and UI proposal.
  • On the implementation side, we should build attachments on the flexible attributes API (e.g., item.attachements should be a list of filenames). The main hurdle here is that each flexible attribute can only have one value in the current library. Some refactoring work is necessary to permit list-valued fields.
@sbarakat

I've published the plugin. I've called it copyartifacts to avoid confusion with the attachments work that's currently going on in the core. All it does currently is copy all other files that haven't been imported by beets, sort of like @alleyoopster suggestion attachfiletypes: *

Give it a go and let me know how it works out. This is the first time I've packaged a module so I hope it works. I'm running Ubuntu 12.04 so YMMV.

Also with regards to the EAC logs, I'm of the opinion that these should remain as is and not altered in any way. I could potentially write a plugin that parses the logs and notifies if any of the tracks were inaccurately ripped or errors occurred. Would that be useful? Perhaps that's a little off topic for this discussion though.

I'll read though the design wiki and see what else I can implement. I'm wanting to be able to rename files based on the tracks in the folder, eg.

  • renamed by beets: 01 - My Track.mp3 -> 01 - Artist - My Track.mp3
  • renamed by copyartifacts: Album.log -> 00 - Artist - Album.log

Also @sampsyo, my sincere thanks for writing beets! I've only discovered it this past week and it's miles ahead of any other music library organiser.

@sampsyo
Member
sampsyo commented Feb 14, 2014

@sbarakat Too cool! This looks really great; I'll add a link to the docs.

On attachment file naming: this is a great point and something that I don't believe is well covered by the current design doc on the wiki. Any chance I could convince you to write some notes on how you think this should work (e.g., how it should be configured)?

@sbarakat

I've updated the plugin to allow importing of white listed file extensions. The design wiki is great! And a lot of the ideas I had are already on there, I'll get to updating it as well as implementing stuff as time goes on.

cue, log and nfo support are major things that are holding me back from running beets over my whole library, so I'll dedicating some time to get it working properly.

@jonathanthomas83

Thank you Sami, really appreciate your efforts.

On 16 February 2014 21:59, Sami Barakat notifications@github.com wrote:

I've updated the plugin to allow importing of white listed file
extensions. The design wiki is great! And a lot of the ideas I had are
already on there, I'll get to updating it as well as implementing stuff as
time goes on.

cue, log and nfo support are the two major things that are holding me back
from running beets over my whole library, so I'll dedicating some time to
get it working properly.

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-35217032
.

Many thanks,
Jonathan

Twitter @jonthomas83 http://twitter.com/jonthomas83

@taylorthurlow

Did some writing on the CLI side of things, located here:
https://github.com/sampsyo/beets/wiki/Attachments

I'm going to poke around the source code a bit, but I'm more of a Java guy and don't know nearly as much Python. Though I think as a language I prefer Python...

@holms
holms commented Feb 28, 2014

Where's a code?

@jonathanthomas83

Nice one, Taylor!

On 28 February 2014 23:03, Roman Gorodeckij notifications@github.comwrote:

Where's a code?

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-36403804
.

@taylorthurlow

Just added in a bit about beet list now too.

Thanks for the kind words.

@sampsyo
Member
sampsyo commented Mar 1, 2014

These are great design notes; thanks for contributing, @taylorthurlow!

Attachment types are an important part of the story. A plugin for managing cue files, for example, might want to find all the cue sheets for albums matching a query—it doesn't care about attached lyric files or whatever. So this is not just about album art: all attachments should be indexed by a type string.

@geigerzaehler
Collaborator

Hi @taylorthurlow and thanks for writing up the CLI stuff. I just extended it a bit.

  • I removed the -a, -d, and -r options. I think they should be implict. If the attachment argument is a file, act like the -a option was given. If the argument is a directory, act like -d and -r were given.
  • I added the -l option to attach files that are already in the album directory.

Let me know if that is ok with you.

@jonathanthomas83

Being someone who has manually copied files for a few months now, since I
started using Beets, the option to attach files that are already in the
album directory is very important to me.

On 3 March 2014 18:08, geigerzaehler notifications@github.com wrote:

Hi @taylorthurlow https://github.com/taylorthurlow and thanks for
writing up the CLI stuff. I just extended it a bit.

I removed the -a, -d, and -r options. I think they should be implict.
If the attachment argument is a file, act like the -a option was
given. If the argument is a directory, act like -d and -r were given.

I added the -l option to attach files that are already in the album
directory.

Let me know if that is ok with you.

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-36538980
.

@taylorthurlow

@geigerzaehler Your edits make perfect sense, not sure why I didn't see that earlier. Thanks for the proofread!

@Night-Man

As long as we're talking about attachments, adding albums with artwork scans would be awesome. Especially if I could query for which ones have scans.

@geigerzaehler
Collaborator

@Night-Man I updated the wiki with some cover art specific cli examples. Is this what you imagined?

@jonathanthomas83

Great work, and great ideas. Can I throw into the mix a way of listing all
albums with and without a certain attachment...for example...if I wanted to
see how many albums I had in my collection without ".log" and ".cue" files.
I couldn't work out if this would work with your example list query, for
example...?

beet list --attachments "log, cue"

On 4 March 2014 17:12, geigerzaehler notifications@github.com wrote:

@Night-Man https://github.com/Night-Man I updated the wikihttps://github.com/sampsyo/beets/wiki/Attachmentswith some cover art specific cli examples. Is this what you imagined?

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-36648742
.

@jonathanthomas83

Sorry for the double email, I guess that would give all the albums with, so
would need an inverse version of that command...apologies.

On 4 March 2014 17:33, Jonathan Thomas jonathanthomas83@gmail.com wrote:

Great work, and great ideas. Can I throw into the mix a way of listing all
albums with and without a certain attachment...for example...if I wanted to
see how many albums I had in my collection without ".log" and ".cue" files.
I couldn't work out if this would work with your example list query, for
example...?

beet list --attachments "log, cue"

On 4 March 2014 17:12, geigerzaehler notifications@github.com wrote:

@Night-Man https://github.com/Night-Man I updated the wikihttps://github.com/sampsyo/beets/wiki/Attachmentswith some cover art specific cli examples. Is this what you imagined?

Reply to this email directly or view it on GitHubhttps://github.com/sampsyo/beets/issues/111#issuecomment-36648742
.

@taylorthurlow

A lot of great edits on the CLI documentation.

@vext01
vext01 commented Jun 1, 2014

Registering interest in this. See #797. My music collection has other files that came with the albums:

  • Album art
  • PDF flyers or other PDF cruft.
  • LSDJ SAV files.
  • ...

I would like to keep those. Ideally in the same dir as the rest of the album.

@holms
holms commented Jun 2, 2014

@vext01 i believe you will be able to register any kind of filename you want manually in your configuration file. my personal interest is covers, cue files, log files, and maybe whole directory with cover files. usually 300dpi scans being attached with album :)

@vext01
vext01 commented Jun 2, 2014

@holms what configuration directive am I looking for?

@holms
holms commented Jun 2, 2014

@vext01 we still don't have attachments ready? or i'm missing something?

@vext01
vext01 commented Jun 18, 2014

Any news on this?

@sampsyo
Member
sampsyo commented Jun 18, 2014

@vext01 Whenever there's news, it will be posted here! If you're interested in this feature, the best thing you can do is help out.

@Profpatsch
Contributor

The easiest way to handle this is to give an option to simply copy any files over that are not recognized (and not clutter). That way nothing gets lost.

@sampsyo
Member
sampsyo commented Jun 21, 2014

@Profpatsch Yes, that's what the stopgap copyartifacts plugin (above) does. This ticket concerns a more complete solution, which would keep track of files in the database also.

@untitaker untitaker referenced this issue in google-code-export/beets May 9, 2015
Open

Attachments to items and albums #109

@NeuroWinter

Has there been any implementation of this as of yet?

@benpye
benpye commented Oct 5, 2016

I feel like I'm just repeating the last comment, but I am looking for an organisation solution and beets seems great, however, I would like to keep cue+log files. Is there any workflow that can currently do that with beets or is it going to be impossible without the attachment support? And if the latter, is there any indication that this is a feature that is going to be undertaken.

@Stunner
Stunner commented Feb 5, 2017

@benpye this task is being undertaken, however slowly as is characteristic with all projects in the open source domain. You can find the in-progress plugin for this here, I have been using it for a little while now and I have no issues with it.

@tom--
tom-- commented Feb 5, 2017 edited

UPDATE: ignore me. i hadn't spotted the wip plugin

Perhaps a set of regex substitutions?

When beets processes a dir, for each file in the dir, if the file is not an audio file processed by core beets functionality, and if the file matches one of the configured regexes, then the plugin copies the file to the corresponding destination of the audio files and applies the re.sub.

keepfile:
    - ["^artist\.(?:jpg|png)$", "\0"]
    - ["^.+?\.cue$", "\0"]

Perhaps the substitution strings can include path format variables and functions too.

I'm not familiar with Python regexing but do you get the idea?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment