iTunes-beets bi-directional sync plugin

[flight16]

Use case

You have an iOS device that you want to use to play and rate songs from your beets library.

Assumptions

1. Beets is your master library containing your original files, metadata, ratings, and play counts.
2. Because beets is your master library, your iTunes library is disposable (ie. it can be deleted and regenerated on demand from beets).

Workflow A: Typical Workflow

1. Create a new, empty iTunes library
2. beets itunes_sync
   1. Transcodes audio (perhaps using existing Export plugin)
   2. Adds tracks to iTunes, with metadata, via AppleScript or Windows SDK.
   3. Smart playlists are sent to iTunes as regular playlists
3. You sync iTunes to iOS, play and rate a few tracks.
4. beets itunes_sync
   1. Ratings and play counts are written back to beets.

Notes:

• 4.i can be disabled via the config
• In step 2.ii, a UUID (or a unique beet ID?) is embedded in the exported track's comment field (can be user-definable). This UUID is used to insulate the sync from filename changes.
• We can perform step 4.c accurately if we take a snapshot of the play counts before the sync.
• 2.i is skipped if it already exists in the iTunes library.
• It would be cool to recreate the smart playlists in iTunes in 2.iii, but my guess (haven't looked at code) is that isn't realistic. Is it even supported in iTunes's API?

Workflow B: Something broke or a configuration changed and you want to force a refresh

1. beets itunes_full_refresh
   1. Wipes out all tracks from iTunes and runs itunes_sync

Further thoughts

• It would be cool to design this in such a way so it's easy to create additional customizations for other media players.
• This plugin's scope is a superset of Plugin to sync metadata from other applications. #1386. It has the following differences:
  • Plugin to sync metadata from other applications. #1386 appears to be a one-way sync from a 3rd party player to beets. This plugin would be bi-directional.
  • Plugin to sync metadata from other applications. #1386 syncs metadata only and doesn't handle transcoding (necessary since iTunes doesn't support FLAC).
  • Plugin to sync metadata from other applications. #1386 appears to store 3rd part player metadata in custom beets fields. In this plugin I propose a customizable mapping is exposed so users can shuffle play count and ratings into whatever beets field they choose.

[sampsyo]

Seems cool. I'd be interested to see how much this would overlap with #1386—bi-directional sync could conceivably be useful for other players too. Building this in a player-generic way would be ideal.

An endeavoring git-peruser could find something similar for very far back in the beets revision history. For a time, a file (called itunes.py, I think?) implemented a partial iTunes XML export for beets libraries.

[pprkut]

#1386 is really meant as a metadata sync plugin. I don't really see transcoding in there as in the majority of cases it wouldn't make sense. Even when talking about iTunes I don't really see the point. Why would you want to constantly translate between different formats in different players? Sounds like a lot of unnecessary complexity to me.

Re bi-directional sync, there's no limitation in #1386 to not make that happen. The way I imagined it would be something like a -w switch which instead of reading values from an external application into beets would write values from beets into the external application, if supported. Complexity here really depends on the external application.

The storing of player metadata in custom beets fields is something that I don't see easily avoided. It's just much easier and safer to do it this way internally. However that is exposed to the user is a different story, but that's independent of the plugin implementation since it's UI stuff.

[flight16]

Even when talking about iTunes I don't really see the point. Why would you want to constantly translate between different formats in different players? Sounds like a lot of unnecessary complexity to me.

This is a key feature to me. It's needed to allow the single-command regeneration of an iTunes library. (I don't want to fiddle with other export plugins and make sure directory paths match up). It does add complexity to the plugin, but it removes complexity from the user.

There are several formats that iTunes doesn't support, such as FLAC. I like to have my master library as lossless and then transcode when needed. The plugin wouldn't transcode supported formats like mp3.

The transcoding isn't constant. The song is only converted when needed: the first time it is copied and does not yet exist in the destination. (eg. iTunes).

Media Monkey and JRiver Media Center both have similar options to transcode your library to a folder on disk based on a filter of non-supported formats, and then keep it up to date. This is very useful if you have a portable music player that is too small to hold your entire lossless collection.

The storing of player metadata in custom beets fields is something that I don't see easily avoided.

I assumed that beets supports rating and playcount fields. Examining the output of 'beet fields', it seems I was incorrect.

It's just much easier and safer to do it this way internally.

If you expose the mapping to the user, then they could choose what they want to do. For example, I'd place my rating in 'rating', while you might place yours in amarok_rating. My goal in placing it in 'rating' is to create a player-independent library of metadata. I want to avoid player-specific fields and naming in my beets library.

@pprkut It sounds like your aim is to create something more minimal implementation and a bit more manual usage, where I'm aiming for a fully integrated solution that is a bit more complex, but automatic from the user's perspective. If so, I think our designs and use cases might be quite different. Perhaps I will start by building a prototype, and we can see if we can't share some code while building two plugins? What do you think?

[pprkut]

It sounds like your aim is to create something more minimal implementation and a bit more manual usage, where I'm aiming for a fully integrated solution that is a bit more complex, but automatic from the user's perspective. If so, I think our designs and use cases might be quite different. Perhaps I will start by building a prototype, and we can see if we can't share some code while building two plugins? What do you think?

Go ahead! Who would I be to try to stop you? It's open-source, go scratch your itch! :)

I'm a Slacker, for me "more manual" means "more powerful", so in that case I agree ;)
My aim is not to create a minimal, but a versatile solution. That's why I don't see why one couldn't implement itunes syncing on top of it. But I know too little about itunes to comment on what exactly would be needed.

[tomjaspers]

While it isn't a quite what you described, #1450 does implement a one-way sync (i.e., sync itunes ratings etc. to beets). Maybe it'll help you a bit, @flight16 .

[2600box]

was any progress made on this?

I would be satisfied with a workflow that makes a small portion of my beets library available to iTunes for syncing. transcoding FLAC to AAC. I haven't tried, but regenerating the entire library in iTunes would be too slow to be practical.

To put it another way, I would like an easy workflow to sync FLAC to iOS Music.app.

[AlexChalk]

Necro bumping (it's an open feature request, so I hope it's ok 😀).

I think the feature as described by the OP would be really cool, but maybe it's worth thinking about the simplest useful version of this and building from there.

If a simple iTunes -> beets sync is supported, I think the next logical step would be a simple beets -> iTunes sync. Are there any good resources to start looking at for info on building itunes xml, or previous itunes export projects people can link to, or just other (non-itunes) export code that follows a similar pattern? Thanks!