New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support local tracks in spotify playlists #519

Closed
kemurphy opened this Issue Sep 22, 2013 · 18 comments

Comments

6 participants
@kemurphy

If a spotify playlist contains a local track (spotify:local URI), mopidy should search the other backends for a matching track (the URI provides album, artist, track name, and duration). @jodal thinks this would require refactoring core. It would at least require a way to avoid deadlock if we wanted to do what the spotify desktop client does and search spotify for a matching track. Searching other backends for matching tracks currently doesn't cause problems if the backends list is made available where it needs to be.

@txomon

This comment has been minimized.

Show comment
Hide comment
@txomon

txomon Sep 22, 2013

Member

I would rather create a way to store local playlists with different backend's tracks. The next step then would be to let something like extending PlaylistController to have a part for PlaylistStorageProvider or so...

Member

txomon commented Sep 22, 2013

I would rather create a way to store local playlists with different backend's tracks. The next step then would be to let something like extending PlaylistController to have a part for PlaylistStorageProvider or so...

@adamcik

This comment has been minimized.

Show comment
Hide comment
@adamcik

adamcik Sep 22, 2013

Member

First of all this will need to have some way of translating from spotify local urls to our local uri scheme. Should probably be easy enough, we just need to figure out how to handle the fact that we have relative uris and spotify I assume has absolute ones.

As for handling of the data back and forth, and option we've discussed with respect to search results is returning more light tracks or track references. Essentially making sure we just return say and uri and maybe a title. If we where to go the same route with this API we could actually have the backend simply give core all the URIs that the playlist contains, and then leave it to the core to lookup what on earth all the URIs actually are. This way it wouldn't really mater if playlists mix backends.

Member

adamcik commented Sep 22, 2013

First of all this will need to have some way of translating from spotify local urls to our local uri scheme. Should probably be easy enough, we just need to figure out how to handle the fact that we have relative uris and spotify I assume has absolute ones.

As for handling of the data back and forth, and option we've discussed with respect to search results is returning more light tracks or track references. Essentially making sure we just return say and uri and maybe a title. If we where to go the same route with this API we could actually have the backend simply give core all the URIs that the playlist contains, and then leave it to the core to lookup what on earth all the URIs actually are. This way it wouldn't really mater if playlists mix backends.

@txomon

This comment has been minimized.

Show comment
Hide comment
@txomon

txomon Sep 22, 2013

Member

+1 for the track references... I think we should have something similar to a heat-list, so that we see which improvements are more desired...

Member

txomon commented Sep 22, 2013

+1 for the track references... I think we should have something similar to a heat-list, so that we see which improvements are more desired...

@kemurphy

This comment has been minimized.

Show comment
Hide comment
@kemurphy

kemurphy Sep 23, 2013

I think it would be wrong for core to be responsible for translating a spotify:local URI into a canonical URI. The spotify:local URI doesn't contain any path information -- just track metadata -- so a search is necessary. I think there should be something like a UriResolverProvider in each backend, and that can hand core either a resolved or an unresolved track. Core could then take all unresolved tracks and turn them into search queries.

I think it would be wrong for core to be responsible for translating a spotify:local URI into a canonical URI. The spotify:local URI doesn't contain any path information -- just track metadata -- so a search is necessary. I think there should be something like a UriResolverProvider in each backend, and that can hand core either a resolved or an unresolved track. Core could then take all unresolved tracks and turn them into search queries.

@txomon

This comment has been minimized.

Show comment
Hide comment
@txomon

txomon Sep 23, 2013

Member

Indeed, library.lookup would be the solution. The only thing I see would be
how to put which backends are admitted. As spotify only admits it's own
tracks, but local would admit any.

Member

txomon commented Sep 23, 2013

Indeed, library.lookup would be the solution. The only thing I see would be
how to put which backends are admitted. As spotify only admits it's own
tracks, but local would admit any.

@adamcik

This comment has been minimized.

Show comment
Hide comment
@adamcik

adamcik Sep 23, 2013

Member

I was thinking in terms of the spotify backend translating this, not the core.

Other thing that comes do mind is that if spotify gives us absolute file paths and metadata we could simply be creating file:// tracks and be done with it. Of course this doesn't solve the larger issue of playlists having tracks from any mix of backends.

And yes returning track references (or whatever they end up being called) and looking them up in core would be the way to go.

Member

adamcik commented Sep 23, 2013

I was thinking in terms of the spotify backend translating this, not the core.

Other thing that comes do mind is that if spotify gives us absolute file paths and metadata we could simply be creating file:// tracks and be done with it. Of course this doesn't solve the larger issue of playlists having tracks from any mix of backends.

And yes returning track references (or whatever they end up being called) and looking them up in core would be the way to go.

@kingosticks

This comment has been minimized.

Show comment
Hide comment
@kingosticks

kingosticks Sep 24, 2013

Member

Do I have this right?

  • The playlist provider for each backend presents the list of playlists stored on the backend (be it local, spotify, soundcloud whatever).
  • Each playlist will contain track references that may or may not be stored on the backend
  • All track references pass through the backend's UriResolverProvider (or something) on their way to the core
  • In the case of spotify:
    • An absolute path (e.g. spotify:track:3E9PkzyiRkpVg15fPVHZLf) can be passed straight through.
    • A local path (e.g. spotify:local:Architects:Daybreaker:Daybreak:212) should trigger a LibraryController.lookup(...) (or maybe a search, in theory you could search all backends for the track as a spotify local track may not be local on this machine, but it may be available on another backend or even spotify) which can then provide the absolute URI for the track.
Member

kingosticks commented Sep 24, 2013

Do I have this right?

  • The playlist provider for each backend presents the list of playlists stored on the backend (be it local, spotify, soundcloud whatever).
  • Each playlist will contain track references that may or may not be stored on the backend
  • All track references pass through the backend's UriResolverProvider (or something) on their way to the core
  • In the case of spotify:
    • An absolute path (e.g. spotify:track:3E9PkzyiRkpVg15fPVHZLf) can be passed straight through.
    • A local path (e.g. spotify:local:Architects:Daybreaker:Daybreak:212) should trigger a LibraryController.lookup(...) (or maybe a search, in theory you could search all backends for the track as a spotify local track may not be local on this machine, but it may be available on another backend or even spotify) which can then provide the absolute URI for the track.
@kemurphy

This comment has been minimized.

Show comment
Hide comment
@kemurphy

kemurphy Sep 24, 2013

@kingosticks Yeah that's roughly what I had in mind. Any playlist in any backend is just a list of URIs, so for each URI that core sees, it should ask the appropriate backend and the backend can decide whether it can provide either a track or metadata for search (or nothing). It should work such that the local playlist backend can have local: URIs and spotify: URIs (spotify:track: and/or spotify:local:), and each URI would be handled by the appropriate backend. Today's library.lookup already almost works like that; it just needs core to be able to interpret different return values to go fetch tracks elsewhere.

@kingosticks Yeah that's roughly what I had in mind. Any playlist in any backend is just a list of URIs, so for each URI that core sees, it should ask the appropriate backend and the backend can decide whether it can provide either a track or metadata for search (or nothing). It should work such that the local playlist backend can have local: URIs and spotify: URIs (spotify:track: and/or spotify:local:), and each URI would be handled by the appropriate backend. Today's library.lookup already almost works like that; it just needs core to be able to interpret different return values to go fetch tracks elsewhere.

@kingosticks

This comment has been minimized.

Show comment
Hide comment
@kingosticks

kingosticks Sep 24, 2013

Member

And this request for a track/metadata should happen when the playlists load/refresh or when the URI is (first) played? if you do it at playlist container load/refresh it could be very slow for a playlist containing many spotify:local tracks, especially if you consider a playlist container refresh occurs whenever any spotify playlist changes (IIRC). If you defer it until the track is played for the first time there might still be a noticeable lag for the user when changing songs.

Member

kingosticks commented Sep 24, 2013

And this request for a track/metadata should happen when the playlists load/refresh or when the URI is (first) played? if you do it at playlist container load/refresh it could be very slow for a playlist containing many spotify:local tracks, especially if you consider a playlist container refresh occurs whenever any spotify playlist changes (IIRC). If you defer it until the track is played for the first time there might still be a noticeable lag for the user when changing songs.

@adamcik

This comment has been minimized.

Show comment
Hide comment
@adamcik

adamcik Sep 24, 2013

Member

Sounds about right, but I would not recommend having a resolver in the backend as this would have the backend talking to core, which is just asking for deadlocks. Way I was thinking is that we just return the references and then have core do the resolving so the core talks to backends bit stays intact.

Other option would be to return proper tracks where we can and then unresolved ones for the types the backend can't handle, leaving it to core to then resolve them.

Member

adamcik commented Sep 24, 2013

Sounds about right, but I would not recommend having a resolver in the backend as this would have the backend talking to core, which is just asking for deadlocks. Way I was thinking is that we just return the references and then have core do the resolving so the core talks to backends bit stays intact.

Other option would be to return proper tracks where we can and then unresolved ones for the types the backend can't handle, leaving it to core to then resolve them.

@kingosticks

This comment has been minimized.

Show comment
Hide comment
@kingosticks

kingosticks Sep 24, 2013

Member

I like the 2nd option @adamcik. It sounds like there'd be a bit of back and forth in some situations but maybe that's unavoidable.

  1. core requests playlists from backends
  2. local playlist contains a spotify:local URI (spotify:local:Architects:Daybreaker:Daybreak:212) but the local backend can't produce a concrete track for this
  3. core uses _get_backend(...) and gets the spotify backend to resolve the track
  4. spotify can't provide a concrete track, but can parse the URI and extract search terms (i.e. [Artist=Architects, Album=Daybreaker, Title=Daybreaker]) for core
  5. core then does a search (over all backends?) and finally gets a concrete track back

I'm still not sure when this search should happen.

Member

kingosticks commented Sep 24, 2013

I like the 2nd option @adamcik. It sounds like there'd be a bit of back and forth in some situations but maybe that's unavoidable.

  1. core requests playlists from backends
  2. local playlist contains a spotify:local URI (spotify:local:Architects:Daybreaker:Daybreak:212) but the local backend can't produce a concrete track for this
  3. core uses _get_backend(...) and gets the spotify backend to resolve the track
  4. spotify can't provide a concrete track, but can parse the URI and extract search terms (i.e. [Artist=Architects, Album=Daybreaker, Title=Daybreaker]) for core
  5. core then does a search (over all backends?) and finally gets a concrete track back

I'm still not sure when this search should happen.

@kemurphy

This comment has been minimized.

Show comment
Hide comment
@kemurphy

kemurphy Sep 24, 2013

@kingosticks A hard track should be required by the time the track is enqueued to play. Shouldn't need to force URI resolution just to browse a playlist, but waiting to resolve until the track is about to play is probably waiting too long.

@adamcik core shouldn't talk to backends while they're running, but backends can certainly return values that instruct core to do more work (and perhaps invoke different parts of the backends)! This was the intention.

@kingosticks A hard track should be required by the time the track is enqueued to play. Shouldn't need to force URI resolution just to browse a playlist, but waiting to resolve until the track is about to play is probably waiting too long.

@adamcik core shouldn't talk to backends while they're running, but backends can certainly return values that instruct core to do more work (and perhaps invoke different parts of the backends)! This was the intention.

@adamcik

This comment has been minimized.

Show comment
Hide comment
@adamcik

adamcik Sep 26, 2013

Member

Hmm, the fact that the uri format is spotify:local:An+artist+name:An+album+name:A+track+name:duration essentially a search might require me to rethink some of my suggestions. I've been assuming that there is some sort of path so far...

Member

adamcik commented Sep 26, 2013

Hmm, the fact that the uri format is spotify:local:An+artist+name:An+album+name:A+track+name:duration essentially a search might require me to rethink some of my suggestions. I've been assuming that there is some sort of path so far...

@txomon

This comment has been minimized.

Show comment
Hide comment
@txomon

txomon Sep 30, 2013

Member

Well, songs and all the rest of resources usually have a sort of "slug" that gives you a lower-cased, dash-separated name to refer to.

Anyway, I think the best would be to use the identifier that spotify uses for everything (their link uris)

Member

txomon commented Sep 30, 2013

Well, songs and all the rest of resources usually have a sort of "slug" that gives you a lower-cased, dash-separated name to refer to.

Anyway, I think the best would be to use the identifier that spotify uses for everything (their link uris)

@kingosticks

This comment has been minimized.

Show comment
Hide comment
@kingosticks

kingosticks Sep 30, 2013

Member

@kemurphy, yeh those are good points. So resolving at a time between browsing and playing - perhaps that's a job more suited to TracklistController ?

@adamcik any more thoughts? The rough program flow I suggested previously does account for the absence of any path.

Maybe the time it takes to do all the searches might still be an issue for big playlists. But it's not like you need all the songs ready to go at once, if you could search for them in the order they would be played (possibly accounting for shuffling etc) perhaps that would be reasonable.

Member

kingosticks commented Sep 30, 2013

@kemurphy, yeh those are good points. So resolving at a time between browsing and playing - perhaps that's a job more suited to TracklistController ?

@adamcik any more thoughts? The rough program flow I suggested previously does account for the absence of any path.

Maybe the time it takes to do all the searches might still be an issue for big playlists. But it's not like you need all the songs ready to go at once, if you could search for them in the order they would be played (possibly accounting for shuffling etc) perhaps that would be reasonable.

@ezman

This comment has been minimized.

Show comment
Hide comment
@ezman

ezman Nov 19, 2013

Very interested in this feature, has anything been done on this yet? If not I might start taking a look at this.

ezman commented Nov 19, 2013

Very interested in this feature, has anything been done on this yet? If not I might start taking a look at this.

@jodal

This comment has been minimized.

Show comment
Hide comment
@jodal

jodal Sep 8, 2014

Member

Moving this from the Spotify to Core label, as this issue is mostly a question of how to make this possible in Mopidy core.

Member

jodal commented Sep 8, 2014

Moving this from the Spotify to Core label, as this issue is mostly a question of how to make this possible in Mopidy core.

@jodal jodal added A-core and removed Spotify labels Sep 8, 2014

@jodal jodal added this to the v1.0 - Core cleanup milestone Sep 8, 2014

@adamcik

This comment has been minimized.

Show comment
Hide comment
@adamcik

adamcik Feb 25, 2015

Member

I think this is one of the issues we've mostly decided to give up on. Closing and if someone disagrees reopen.

Member

adamcik commented Feb 25, 2015

I think this is one of the issues we've mostly decided to give up on. Closing and if someone disagrees reopen.

@adamcik adamcik closed this Feb 25, 2015

@jodal jodal removed this from the v2.0 - Core cleanup milestone Mar 18, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment