Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Soundcloud backend #339

Closed
wants to merge 32 commits into from

3 participants

@dz0ny
Collaborator

Implements #293

Done:

Todo:

  • Tests

Ideas:

  • Liking (user can add song to playlist liked)
  • Playlists to Sets cross saving
docs/modules/backends/soundcloud.rst
@@ -0,0 +1,8 @@
+.. soundcloud-backend:
+
+*************************************************
+:mod:`mopidy.backends.soundcloud` -- Soundcloud backend
@jodal Owner
jodal added a note

Just doing some random review here... not going to look through everything now.

SoundCloud should be written with a capital "C". "***" rows should have the same length as the header text itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/actor.py
@@ -0,0 +1,27 @@
+from __future__ import unicode_literals
+
+import logging
+
+import pykka
+
+from mopidy.backends import base
+from mopidy import settings
@jodal Owner
jodal added a note

Sort this block of imports. Should be in the opposite order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/library.py
((8 lines not shown))
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundcloudLibraryProvider(base.BaseLibraryProvider):
+ def __init__(self, *args, **kwargs):
+ super(SoundcloudLibraryProvider, self).__init__(*args, **kwargs)
+
+ def find_exact(self, **query):
+ return self.search(**query)
+
+ def search(self, **query):
+ if not query:
+ return
+
+ for (field, val) in query.iteritems():
+ if field == "any":
@jodal Owner
jodal added a note

We prefer single quotes over double quotes. Same for the argument to split() below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jodal jodal commented on the diff
mopidy/backends/soundcloud/library.py
((3 lines not shown))
+import logging
+
+from mopidy.backends import base
+from mopidy.models import SearchResult
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundcloudLibraryProvider(base.BaseLibraryProvider):
+ def __init__(self, *args, **kwargs):
+ super(SoundcloudLibraryProvider, self).__init__(*args, **kwargs)
+
+ def find_exact(self, **query):
+ return self.search(**query)
+
+ def search(self, **query):
@jodal Owner
jodal added a note

Should maybe add some TODOs here if there are more stuff than we can make work with the SoundCloud search API. I'm thinking of things like:

  • exact searches
  • search specific fields like artist, title, album, year, etc.
  • search for multiple terms (e.g. {'any': ['foo', 'bar']} returning sound files matching both 'foo' and 'bar')

If some of these features are not supported by SoundCloud, it should probably be documented in a docstring on the function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/playlists.py
((18 lines not shown))
+
+ def delete(self, uri):
+ pass # TODO
+
+ def lookup(self, uri):
+ for playlist in self._playlists:
+ if playlist.uri == uri:
+ return playlist
+
+ def refresh(self):
+ logger.info('Loading playlists from Soundcloud')
+
+ playlists = []
+
+ playlist = Playlist(
+ uri="soundcloud://playlists/liked",
@jodal Owner
jodal added a note

Not that we have any strong conventions on our custom URIs yet, but typically in URIs, playlists would here be interpreted as the host name. Maybe use three slashes, or skip the slashes entirely and go with colons. I'm not sure, I just think that we should align how we construct custom URIs across the backends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
@@ -0,0 +1,147 @@
+#!/usr/local/bin/python
+# -*- coding: utf-8 -*-
+#
@jodal Owner
jodal added a note

No need for these three lines, as there is no module level code to run here and no non-ASCII chars AFAICT.

@dz0ny Collaborator
dz0ny added a note

I was trying to add some indication to song names of where the come from (like ☀☁)

@jodal Owner
jodal added a note

Unit tests are perfect for experimenting with such things ;-)

I think that such markers on tracks needs to be implemented in the various frontends. E.g. the web clients got enough information to annotate the tracks on their own, while e.g. MPD would need Unicode char hacks like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((16 lines not shown))
+ ## TODO: merge this to util library
+ def __init__(self, ctl=8, ttl=3600):
+ self.cache = {}
+ self.ctl = ctl
+ self.ttl = ttl
+ self._call_count = 1
+
+ def __call__(self, func):
+ def _memoized(*args):
+ self.func = func
+ now = time.time()
+ try:
+ value, last_update = self.cache[args]
+ age = now - last_update
+ if self._call_count >= self.ctl or \
+ age > self.ttl:
@jodal Owner
jodal added a note

Parenthesis is better than backslash, e.g.:

if (self._call_count >= self.ctrl
        or age > self.ttl):
    # ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((36 lines not shown))
+ return value
+
+ except (KeyError, AttributeError):
+ value = self.func(*args)
+ self.cache[args] = (value, now)
+ return value
+
+ except TypeError:
+ return self.func(*args)
+ return _memoized
+
+
+class SoundcloudClient(object):
+
+ CLIENT_ID = "93e33e327fd8a9b77becd179652272e2"
+ CLIENT_SECRET = "f1a2e1ff740f3e1e340e6993ceb18583"
@jodal Owner
jodal added a note

Where do these come from? Are they associated with your SoundCloud account or something like that, or did you register Mopidy as a client?

@dz0ny Collaborator
dz0ny added a note

Registered custom client.

@jodal Owner
jodal added a note

If there's any way to associate a client to multiple SoundCloud user accounts, you can associate it with the stein.magnus@jodal.no account as well. Wasn't immediately obvious to me without registering yet another app if that's possible.

@dz0ny Collaborator
dz0ny added a note

No they are tied to one oner only. I was thinking of maybe providing defaults, overridable by user settings.

@jodal Owner
jodal added a note

No problem. Will Mopidy be visible in the SoundCloud app directory, or is that a separate registration or a curated directory?

@dz0ny Collaborator
dz0ny added a note

No idea, but that probably requires some communication with SoundCloud team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((45 lines not shown))
+ return _memoized
+
+
+class SoundcloudClient(object):
+
+ CLIENT_ID = "93e33e327fd8a9b77becd179652272e2"
+ CLIENT_SECRET = "f1a2e1ff740f3e1e340e6993ceb18583"
+
+ def __init__(self, username):
+ super(SoundcloudClient, self).__init__()
+ self.user_id = self.get_userid(username)
+
+ def get_userid(self, username):
+ try:
+ user = self._get("resolve.json?url=http://soundcloud.com/%s" % username)
+ return user.get("id")
@jodal Owner
jodal added a note

Should use single quotes on the above two lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((48 lines not shown))
+class SoundcloudClient(object):
+
+ CLIENT_ID = "93e33e327fd8a9b77becd179652272e2"
+ CLIENT_SECRET = "f1a2e1ff740f3e1e340e6993ceb18583"
+
+ def __init__(self, username):
+ super(SoundcloudClient, self).__init__()
+ self.user_id = self.get_userid(username)
+
+ def get_userid(self, username):
+ try:
+ user = self._get("resolve.json?url=http://soundcloud.com/%s" % username)
+ return user.get("id")
+ except Exception:
+ raise logger.error('Can\'t get id for %s, status code %s' % (
+ username, user.status_code))
@jodal Owner
jodal added a note

But here double quotes to avoid the escaping of single quote inside the string is totally OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((42 lines not shown))
+
+ except TypeError:
+ return self.func(*args)
+ return _memoized
+
+
+class SoundcloudClient(object):
+
+ CLIENT_ID = "93e33e327fd8a9b77becd179652272e2"
+ CLIENT_SECRET = "f1a2e1ff740f3e1e340e6993ceb18583"
+
+ def __init__(self, username):
+ super(SoundcloudClient, self).__init__()
+ self.user_id = self.get_userid(username)
+
+ def get_userid(self, username):
@jodal Owner
jodal added a note

I think this should be named get_user_id, like the variable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((87 lines not shown))
+ return tracks
+
+ def _get(self, url):
+
+ if '?' in url:
+ url = "%s&client_id=%s" % (url, self.CLIENT_ID)
+ else:
+ url = "%s?client_id=%s" % (url, self.CLIENT_ID)
+
+ url = 'https://api.soundcloud.com/%s' % url
+
+ logger.debug('Requesting %s' % url)
+ req = requests.get(url)
+ if req.status_code != 200:
+ raise logger.error('Request %s, failed with status code %s' % (
+ url, req.status_code))
@jodal Owner
jodal added a note

Is there some useful error response body to include in a debug level message here?

@dz0ny Collaborator
dz0ny added a note

No only standard http messages in the body, like { reason: "401 Unathorized" } etc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/settings.py
@@ -326,3 +326,8 @@
'rtmps',
'rtsp',
)
+
+#: Your `Soundcloud.comLast.fm <http://www.soundcloud.com/>`_ username.
@jodal Owner
jodal added a note

s/Soundcloud.comLast.fm/SoundCloud/

@dz0ny Collaborator
dz0ny added a note

:)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jodal
Owner

I haven't tried running the backend, but I generally like what I see. As mentioned on IRC, tests would be a huge plus.

Great work! :-)

mopidy/settings.py
@@ -326,3 +326,31 @@
'rtmps',
'rtsp',
)
+
+#: Your `SoundCloud.com <http://www.soundcloud.com/>`_ authentication token.
+#:
+#: Get yours at http://www.mopidy.com/authenticate.html
+#:
+#: Used by :mod:`mopidy.frontends.soundcloud`.
@adamcik Owner
adamcik added a note

s/frontends/backends/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/settings.py
((11 lines not shown))
+
+#: Extra playlists from `SoundCloud.com <http://www.soundcloud.com/explore>`
+#:
+#: Note: this might take a while to load. You can add more if you want,
+#: for example:
+#: if you want Smooth Jazz from https://soundcloud.com/explore/jazz%2Bblues
+#: your entry would be u'jazz%2Bblues/Smooth Jazz'
+#:
+#: Default::
+#:
+#: SOUNDCLOUD_EXPLORE = [
+#: u'electronic/Ambient',
+#: u'pop/New Wave',
+#: u'rock/Indie',
+#: ]
+#: Used by :mod:`mopidy.frontends.soundcloud`.
@adamcik Owner
adamcik added a note

s/frontends/backends/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/actor.py
((9 lines not shown))
+
+from .library import SoundcloudLibraryProvider
+from .playlists import SoundcloudPlaylistsProvider
+from .soundcloud import SoundcloudClient
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundcloudBackend(pykka.ThreadingActor, base.Backend):
+ def __init__(self, audio):
+ super(SoundcloudBackend, self).__init__()
+
+ if not settings.SOUNDCLOUD_AUTHTOKEN:
+ logger.error(("In order to use SoundCloud backend "
+ "you must provide settings.SOUNDCLOUD_AUTHTOKEN. "
+ "Get yours at http://www.mopidy.com/authenticate.html"))
@adamcik Owner
adamcik added a note

We could also ship the js-page as part of mopidy so the user just opens some file://... url, but lets go with just the mopidy.com on for now.

@dz0ny Collaborator
dz0ny added a note

That would'n work, SoundCloud has restrictions in place. First there must be callback.html file present and second callback.html file must have same URI as one set in API settings page on SoundCloud. If those conditions aren't met, authorization request will be denied.

@adamcik Owner
adamcik added a note

Ah, good to know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/__init__.py
@@ -0,0 +1,25 @@
+"""A backend for playing music from Soundcloud.
+
+This backend handles URIs starting with ``soundcloud:``.
+
+See :ref:`music-from-soundcloud-storage` for further instructions on using this
+backend.
+
+**Issues:**
+
+https://github.com/mopidy/mopidy/issues?labels=Soundcloud+backend
@jodal Owner
jodal added a note

I've created a label now. Please do a s/Soundcloud/SoundCloud/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/modules/backends/soundcloud.rst
@@ -0,0 +1,8 @@
+.. soundcloud-backend:
+
+*************************************************
+:mod:`mopidy.backends.soundcloud` -- SoundCloud backend
+*************************************************
@jodal Owner
jodal added a note

*** rows should have equal length with the header text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/modules/backends/soundcloud.rst
@@ -0,0 +1,8 @@
+.. soundcloud-backend:
+
+*************************************************
+:mod:`mopidy.backends.soundcloud` -- SoundCloud backend
+*************************************************
+
+.. automodule:: mopidy.backends.soudcloud
@jodal Owner
jodal added a note

s/soudcloud/soundcloud/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jodal jodal commented on the diff
mopidy/backends/soundcloud/actor.py
((9 lines not shown))
+
+from .library import SoundcloudLibraryProvider
+from .playlists import SoundcloudPlaylistsProvider
+from .soundcloud import SoundCloudClient
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundcloudBackend(pykka.ThreadingActor, base.Backend):
+ def __init__(self, audio):
+ super(SoundcloudBackend, self).__init__()
+
+ if not settings.SOUNDCLOUD_AUTH_TOKEN:
+ logger.error(("In order to use SoundCloud backend "
+ "you must provide settings.SOUNDCLOUD_AUTH_TOKEN. "
+ "Get yours at http://www.mopidy.com/authenticate"))
@jodal Owner
jodal added a note

What happens if you ignore this error and just try to use the SoundCloud backend? I guess something will crash with an AttributeError on backend.sc_api now being available, which isn't very nice. The backend should shut down properly after logging the error.

@jodal Owner
jodal added a note

I just ran into this case. Because I got a traceback about sc_api missing, it took focus away from the error message right above, stating what the root cause was.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/actor.py
((2 lines not shown))
+
+import logging
+import pykka
+
+from mopidy import settings
+from mopidy.backends import base
+
+
+from .library import SoundcloudLibraryProvider
+from .playlists import SoundcloudPlaylistsProvider
+from .soundcloud import SoundCloudClient
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundcloudBackend(pykka.ThreadingActor, base.Backend):
@jodal Owner
jodal added a note

s/Soundcloud/SoundCloud/

@jodal Owner
jodal added a note

And same for the rest of the class names. I won't comment on all of them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((195 lines not shown))
+ # thus prevent user from selecting track
+ if not is_search:
+ if ' - ' in name:
+ name = name.split(' - ')
+ track_kwargs[b'name'] = name[1]
+ artist_kwargs[b'name'] = name[0]
+ elif 'label_name' in data and data['label_name'] != '':
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data['label_name']
+ else:
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data.get('user').get('username')
+
+ album_kwargs[b'name'] = 'SoundCloud'
+ else:
+ ## NOTE mpdroid removes ☁ from track name, probably others too
@jodal Owner
jodal added a note

As discussed on IRC, logic/hacks like this should be added by the frontends, based upon the frontends capabilities and probably the URI scheme of the track.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Janez Troha added some commits
Janez Troha Fix: SoundCloud naming d7f093b
Janez Troha Fix: removing // from URI 272c185
Janez Troha Review: resolve uri 7f94568
Janez Troha Merge branch 'develop' of git://github.com/mopidy/mopidy.git into sou…
…ndcloud_backend
3386353
Janez Troha Fix: Double class, super methods 9f4dfb9
Janez Troha Merge branch 'soundcloud/resolve_uri' into soundcloud_backend
Conflicts:
	mopidy/backends/soundcloud/playlists.py
8e50a94
Janez Troha Fix: docs 023a038
Janez Troha Add: More debug information 1ab2740
Janez Troha Fix: Playlist resolving
Rename: Method for fetching displaying user liked tracks
Add: Test for avaliabilty of streamable track
ea17017
Janez Troha Fix: Only add albumart if album name is defined 5dcd5fd
Janez Troha Fix: SoundCloud api unavailable, leaves backend in broken state
Fix: Document issue with playlists for mobile clients
Fix: User is resolved before api exists
df3cf02
Janez Troha Add: Playlists tests f27d2fe
mopidy/backends/soundcloud/playlists.py
((15 lines not shown))
+ super(SoundCloudPlaylistsProvider, self).__init__(*args, **kwargs)
+ self._playlists = []
+ self.refresh()
+
+ def create(self, name):
+ pass # TODO
+
+ def delete(self, uri):
+ pass # TODO
+
+ def lookup_get_tracks(self, uri):
+ # TODO: Figure out why some sort of internal cache is used for retrieving
+ # track-list on mobile clients. If you wan't this to work with mobile
+ # clients change defaults to streamable=True
+ if 'soundcloud:exp-' in uri:
+ logger.info('Detected lookup for explore %s' % uri)
@adamcik Owner
adamcik added a note

Info logging also of this is likely to be to verbose in the long run, I suspect this should be debug to avoid noise in the logs. For development you should be able to set the log level for this logger to debug in your settings file by importing logging etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/playlists.py
((17 lines not shown))
+ self.refresh()
+
+ def create(self, name):
+ pass # TODO
+
+ def delete(self, uri):
+ pass # TODO
+
+ def lookup_get_tracks(self, uri):
+ # TODO: Figure out why some sort of internal cache is used for retrieving
+ # track-list on mobile clients. If you wan't this to work with mobile
+ # clients change defaults to streamable=True
+ if 'soundcloud:exp-' in uri:
+ logger.info('Detected lookup for explore %s' % uri)
+ return self.create_explore_playlist(uri, True)
+ elif 'soundcloud:u-liked' in uri:
@adamcik Owner
adamcik added a note

Why abbreviate, user-liked etc. is much easier to understand and a byte or two lost here won't really matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((41 lines not shown))
+ value = self.func(*args)
+ self.cache[args] = (value, now)
+ return value
+
+ except TypeError:
+ return self.func(*args)
+ return _memoized
+
+
+class SoundCloudClient(object):
+
+ CLIENT_ID = '93e33e327fd8a9b77becd179652272e2'
+
+ def __init__(self, token):
+ super(SoundCloudClient, self).__init__()
+ self.SC = requests.Session()
@adamcik Owner
adamcik added a note

SC in caps doesn't really follow our style guide as caps are normally reserved to constants.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
mopidy/backends/soundcloud/soundcloud.py
((203 lines not shown))
+ artist_kwargs[b'name'] = name[0]
+ elif 'label_name' in data and data['label_name'] != '':
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data['label_name']
+ else:
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data.get('user').get('username')
+
+ album_kwargs[b'name'] = 'SoundCloud'
+
+ if 'date' in data:
+ track_kwargs[b'date'] = data['date']
+
+ if remote_url:
+ logger.info("Adding streamable track %s" % track_kwargs[b'name'])
+ track_kwargs[b'uri'] = '%s?client_id=%s' % (
@adamcik Owner
adamcik added a note

Based on what I was telling you on IRC, at this point building a the uri as the actual http would cause trouble when running without the stream backend. Idea I'm suggesting as a possible alternative here is to construct something like 'soundcloud:track:%s:%s' % (stream_url, client_id) where stream url is the url with the http:// part stripped.

As for the rest of the urls soundcloud:set:foo etc might also work, loosely inspired by spotify urls, but now I'm wandering very close to bikesheding. Though, there might be some merit to using the same format where you have scheme and then a type and then a specifier using : as a delimiter as it should be among the safer chars to use for such a purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Janez Troha added some commits
Janez Troha Add: Custom audio implementation
Fix: Prevent parsing of track if it isn't avaliable
a5b864b
Janez Troha Fix: Rename variable SC to http_client
Remove: Debug information for playlist, client
9aed118
Janez Troha Merge branch 'develop' of git://github.com/mopidy/mopidy.git into sou…
…ndcloud_backend
14286bf
@dz0ny dz0ny closed this
@dz0ny dz0ny deleted the unknown repository branch
@adamcik
Owner

Postponed while we fix extension support I hope?

@dz0ny
Collaborator

yes :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 14, 2013
  1. Initial Soundcloud backend

    Janez Troha authored
  2. Only add streamable tracks

    Janez Troha authored
  3. PEP8 fixes

    Janez Troha authored
  4. Add Soundcloud branding

    Janez Troha authored
  5. Added sets

    Janez Troha authored
Commits on Mar 15, 2013
  1. Add user stream playlist

    Janez Troha authored
  2. Use oAuth login, add Explore section as playlists

    Janez Troha authored
Commits on Mar 16, 2013
  1. Fix: documentation for settings

    Janez Troha authored
    Fix: explore playlists with empty tracks
  2. Fix: handle 404, missing or deleted tracks

    Janez Troha authored
  3. fix: capitalize class

    Janez Troha authored
    fix: larger playlists
  4. add: Basic SounCloud JSON track parser tests

    Janez Troha authored
Commits on Mar 19, 2013
  1. Fix: Library behavior on searching

    Janez Troha authored
  2. fix: variable naming

    Janez Troha authored
  3. Fix: Libray search for mpris

    Janez Troha authored
Commits on Mar 22, 2013
  1. Fix: SoundCloud naming

    Janez Troha authored
  2. Fix: removing // from URI

    Janez Troha authored
  3. Review: resolve uri

    Janez Troha authored
  4. Merge branch 'develop' of git://github.com/mopidy/mopidy.git into sou…

    Janez Troha authored
    …ndcloud_backend
  5. Fix: Double class, super methods

    Janez Troha authored
Commits on Mar 23, 2013
  1. Merge branch 'soundcloud/resolve_uri' into soundcloud_backend

    Janez Troha authored
    Conflicts:
    	mopidy/backends/soundcloud/playlists.py
  2. Fix: docs

    Janez Troha authored
  3. Add: More debug information

    Janez Troha authored
  4. Fix: Playlist resolving

    Janez Troha authored
    Rename: Method for fetching displaying user liked tracks
    Add: Test for avaliabilty of streamable track
  5. Fix: Only add albumart if album name is defined

    Janez Troha authored
Commits on Mar 24, 2013
  1. Fix: SoundCloud api unavailable, leaves backend in broken state

    Janez Troha authored
    Fix: Document issue with playlists for mobile clients
    Fix: User is resolved before api exists
  2. Add: Playlists tests

    Janez Troha authored
Commits on Mar 26, 2013
  1. Add: Custom audio implementation

    Janez Troha authored
    Fix: Prevent parsing of track if it isn't avaliable
  2. Fix: Rename variable SC to http_client

    Janez Troha authored
    Remove: Debug information for playlist, client
Commits on Mar 31, 2013
  1. Merge branch 'develop' of git://github.com/mopidy/mopidy.git into sou…

    Janez Troha authored
    …ndcloud_backend
This page is out of date. Refresh to see the latest.
View
1  .travis.yml
@@ -5,6 +5,7 @@ install:
- "sudo wget -q -O /etc/apt/sources.list.d/mopidy.list http://apt.mopidy.com/mopidy.list"
- "sudo apt-get update || true"
- "sudo apt-get install $(apt-cache depends mopidy | awk '$2 !~ /mopidy/ {print $2}')"
+ - "sudo apt-get install python-requests"
before_script:
- "rm $VIRTUAL_ENV/lib/python$TRAVIS_PYTHON_VERSION/no-global-site-packages.txt"
View
8 docs/modules/backends/soundcloud.rst
@@ -0,0 +1,8 @@
+.. soundcloud-backend:
+
+*******************************************************
+:mod:`mopidy.backends.soundcloud` -- SoundCloud backend
+*******************************************************
+
+.. automodule:: mopidy.backends.soudcloud
+ :synopsis: Backend for the SoundCloud music streaming service
View
25 mopidy/backends/soundcloud/__init__.py
@@ -0,0 +1,25 @@
+"""A backend for playing music from SoundCloud.
+
+This backend handles URIs starting with ``soundcloud:``.
+
+See :ref:`music-from-soundcloud-storage` for further instructions on using this
+backend.
+
+**Issues:**
+
+https://github.com/mopidy/mopidy/issues?labels=SoundCloud+backend
+
+**Dependencies:**
+
+.. literalinclude:: ../../../requirements/soundcloud.txt
+
+**Settings:**
+
+- :attr:`mopidy.settings.SOUNDCLOUD_AUTH_TOKEN`
+- :attr:`mopidy.settings.SOUNDCLOUD_EXPLORE`
+"""
+
+from __future__ import unicode_literals
+
+# flake8: noqa
+from .actor import SoundCloudBackend
View
41 mopidy/backends/soundcloud/actor.py
@@ -0,0 +1,41 @@
+from __future__ import unicode_literals
+
+import logging
+import pykka
+
+from mopidy import settings
+from mopidy.backends import base
+
+
+from .library import SoundCloudLibraryProvider
+from .playlists import SoundCloudPlaylistsProvider
+from .soundcloud import SoundCloudClient
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundCloudBackend(pykka.ThreadingActor, base.Backend):
+
+ def __init__(self, audio):
+ super(SoundCloudBackend, self).__init__()
+
+ if not settings.SOUNDCLOUD_AUTH_TOKEN:
+ logger.error(("In order to use SoundCloud backend "
+ "you must provide settings.SOUNDCLOUD_AUTH_TOKEN. "
+ "Get yours at http://www.mopidy.com/authenticate"))
@jodal Owner
jodal added a note

What happens if you ignore this error and just try to use the SoundCloud backend? I guess something will crash with an AttributeError on backend.sc_api now being available, which isn't very nice. The backend should shut down properly after logging the error.

@jodal Owner
jodal added a note

I just ran into this case. Because I got a traceback about sc_api missing, it took focus away from the error message right above, stating what the root cause was.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ else:
+ self.sc_api = SoundCloudClient(settings.SOUNDCLOUD_AUTH_TOKEN)
+ self.library = SoundCloudLibraryProvider(backend=self)
+ self.playback = SoundCloudPlaybackProvider(audio=audio, backend=self)
+ self.playlists = SoundCloudPlaylistsProvider(backend=self)
+
+ self.uri_schemes = ['soundcloud']
+
+
+class SoundCloudPlaybackProvider(base.BasePlaybackProvider):
+
+ def play(self, track):
+ id = track.uri.split(';')[1]
+ logger.info('Getting info for track %s with id %s' % (track.uri, id))
+ track = self.backend.sc_api.get_track(id, True)
+ return super(SoundCloudPlaybackProvider, self).play(track)
View
43 mopidy/backends/soundcloud/library.py
@@ -0,0 +1,43 @@
+from __future__ import unicode_literals
+
+import logging
+
+from mopidy.backends import base
+from mopidy.models import SearchResult
+
+logger = logging.getLogger('mopidy.backends.soundcloud')
+
+
+class SoundCloudLibraryProvider(base.BaseLibraryProvider):
+ def __init__(self, *args, **kwargs):
+ super(SoundCloudLibraryProvider, self).__init__(*args, **kwargs)
+
+ def find_exact(self, **query):
+ return self.search(**query)
+
+ def search(self, **query):
@jodal Owner
jodal added a note

Should maybe add some TODOs here if there are more stuff than we can make work with the SoundCloud search API. I'm thinking of things like:

  • exact searches
  • search specific fields like artist, title, album, year, etc.
  • search for multiple terms (e.g. {'any': ['foo', 'bar']} returning sound files matching both 'foo' and 'bar')

If some of these features are not supported by SoundCloud, it should probably be documented in a docstring on the function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ if not query:
+ return
+
+ for (field, val) in query.iteritems():
+
+ # TODO: Devise method for searching SoundCloud via artists
+ if field == "album" and query['album'] == "SoundCloud":
+ return SearchResult(
+ uri='soundcloud:search',
+ tracks=self.backend.sc_api.search(query['artist']) or [])
+ elif field == "any":
+ return SearchResult(
+ uri='soundcloud:search',
+ tracks=self.backend.sc_api.search(val[0]) or [])
+ else:
+ return []
+
+ def lookup(self, uri):
+ try:
+ id = uri.split(':')[1]
+ logger.info(u'SoundCloud track id for %s: %s' % (uri, id))
+ return [self.backend.sc_api.get_track(id)]
+ except Exception as error:
+ logger.error(u'Failed to lookup %s: %s', uri, error)
+ return []
View
95 mopidy/backends/soundcloud/playlists.py
@@ -0,0 +1,95 @@
+from __future__ import unicode_literals
+
+import logging
+
+from mopidy import settings
+from mopidy.backends import base, listener
+from mopidy.models import Playlist
+
+logger = logging.getLogger('mopidy.backends.soundcloud.playlists')
+
+
+class SoundCloudPlaylistsProvider(base.BasePlaylistsProvider):
+
+ def __init__(self, *args, **kwargs):
+ super(SoundCloudPlaylistsProvider, self).__init__(*args, **kwargs)
+ self._playlists = []
+ self.refresh()
+
+ def create(self, name):
+ pass # TODO
+
+ def delete(self, uri):
+ pass # TODO
+
+ def lookup_get_tracks(self, uri):
+ # TODO: Figure out why some sort of internal cache is used for retrieving
+ # track-list on mobile clients. If you wan't this to work with mobile
+ # clients change defaults to streamable=True
+ if 'soundcloud:exp-' in uri:
+ return self.create_explore_playlist(uri, True)
+ elif 'soundcloud:user-liked' in uri:
+ return self.create_user_liked_playlist(True)
+ elif 'soundcloud:user-stream' in uri:
+ return self.create_user_stream_playlist(True)
+ else:
+ return []
+
+ def lookup(self, uri):
+ for playlist in self._playlists:
+ if playlist.uri == uri:
+ # Special case with sets, which already contain all data
+ if 'soundcloud:set-' in uri:
+ return playlist
+ logger.debug('Resolving with %s', playlist.name)
+ return self.lookup_get_tracks(uri)
+
+ def create_explore_playlist(self, uri, streamable=False):
+ uri = uri.replace('soundcloud:exp-', '')
+ (category, section) = uri.split(';')
+ logger.debug('Fetching Explore playlist %s from SoundCloud' % section)
+ return Playlist(
+ uri='soundcloud:exp-%s' % uri,
+ name='Explore %s on SoundCloud' % section,
+ tracks=self.backend.sc_api.get_explore_category(
+ category, section) if streamable else []
+ )
+
+ def create_user_liked_playlist(self, streamable=False):
+ username = self.backend.sc_api.get_user().get('username')
+ logger.debug('Fetching Liked playlist for %s' % username)
+ return Playlist(
+ uri='soundcloud:user-liked',
+ name="%s's liked on SoundCloud" % username,
+ tracks=self.backend.sc_api.get_user_favorites() if streamable else []
+ )
+
+ def create_user_stream_playlist(self, streamable=False):
+ username = self.backend.sc_api.get_user().get('username')
+ logger.debug('Fetching Stream playlist for %s' % username)
+ return Playlist(
+ uri='soundcloud:user-stream',
+ name="%s's stream on SoundCloud" % username,
+ tracks=self.backend.sc_api.get_user_stream() if streamable else []
+ )
+
+ def refresh(self):
+ self._playlists.append(self.create_user_liked_playlist())
+ self._playlists.append(self.create_user_stream_playlist())
+
+ for (name, uri, tracks) in self.backend.sc_api.get_sets():
+ scset = Playlist(
+ uri='soundcloud:set-%s' % uri,
+ name=name,
+ tracks=tracks
+ )
+ self._playlists.append(scset)
+
+ for cat in settings.SOUNDCLOUD_EXPLORE:
+ exp = self.create_explore_playlist(cat.replace('/', ';'))
+ self._playlists.append(exp)
+ logger.info('Loaded %d SoundCloud playlist(s)', len(self._playlists))
+ listener.BackendListener.send('playlists_loaded')
+
+ def save(self, playlist):
+ pass # TODO
View
250 mopidy/backends/soundcloud/soundcloud.py
@@ -0,0 +1,250 @@
+#!/usr/local/bin/python
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import logging
+import requests
+import time
+
+from requests.exceptions import RequestException
+from mopidy.models import Track, Artist, Album
+from urllib import quote_plus
+
+logger = logging.getLogger('mopidy.backends.soundcloud.client')
+
+
+class cache(object):
+ # TODO: merge this to util library
+
+ def __init__(self, ctl=8, ttl=3600):
+ self.cache = {}
+ self.ctl = ctl
+ self.ttl = ttl
+ self._call_count = 1
+
+ def __call__(self, func):
+ def _memoized(*args):
+ self.func = func
+ now = time.time()
+ try:
+ value, last_update = self.cache[args]
+ age = now - last_update
+ if (self._call_count >= self.ctrl
+ or age > self.ttl):
+ self._call_count = 1
+ raise AttributeError
+
+ self._call_count += 1
+ return value
+
+ except (KeyError, AttributeError):
+ value = self.func(*args)
+ self.cache[args] = (value, now)
+ return value
+
+ except TypeError:
+ return self.func(*args)
+ return _memoized
+
+
+class SoundCloudClient(object):
+
+ CLIENT_ID = '93e33e327fd8a9b77becd179652272e2'
+
+ def __init__(self, token):
+ super(SoundCloudClient, self).__init__()
+ self.http_client = requests.Session()
+ self.http_client.headers.update({'Authorization': 'OAuth %s' % token})
+ self.user = self.get_user()
+ logger.debug('User id for username %s is %s' % (
+ self.user.get('username'), self.user.get('id')))
+
+ @cache()
+ def get_user(self):
+ try:
+ return self._get('me.json')
+ except Exception as e:
+ logger.error('SoundCloud Authentication error: %s' % e)
+
+ # Private
+
+ @cache()
+ def get_user_stream(self):
+ # User timeline like playlist which uses undocumented api
+ # https://api.soundcloud.com/e1/me/stream.json?offset=0
+ # returns five elements per request
+ tracks = []
+ for sid in xrange(0, 2):
+ stream = self._get('e1/me/stream.json?offset=%s' % sid * 5)
+ for data in stream.get('collection'):
+ try:
+ kind = data.get('type')
+ # multiple types of track with same data
+ if 'track' in kind:
+ tracks.append(self.parse_track(data.get('track')))
+ if kind == 'playlist':
+ tracks.extend(self.parse_results(
+ data.get('playlist').get('tracks')))
+ except Exception:
+ # Type not supported or SC changed API
+ pass
+
+ return self.sanitize_tracks(tracks)
+
+ @cache()
+ def get_sets(self):
+ playlists = self._get('users/%s/playlists.json' % self.user.get('id'))
+ tplaylists = []
+ for playlist in playlists:
+ name = '%s on SoundCloud' % playlist.get('title')
+ uri = playlist.get('permalink')
+ tracks = self.parse_results(playlist.get('tracks'))
+ logger.debug('Fetched set %s with id %s' % (name, uri))
+ tplaylists.append((name, uri, tracks))
+ return tplaylists
+
+ def get_user_favorites(self):
+ favorites = self._get('users/%s/favorites.json' % self.user.get('id'))
+ return self.parse_results(favorites)
+
+ # Public
+
+ @cache(ctl=100)
+ def get_track(self, id, streamable=False):
+ try:
+ # TODO better way to handle deleted tracks
+ return self.parse_track(self._get('tracks/%s.json' % id), streamable)
+ except Exception:
+ return
+
+ @cache()
+ def get_explore_category(self, category, section):
+ logger.debug("get_explore_category %s %s" % (category, section))
+ # Most liked by category in explore section
+ tracks = []
+ for sid in xrange(0, 2):
+ stream = self._get('explore/sounds/category/%s?offset=%s' % (
+ category.lower(), sid * 20))
+ for data in stream.get('collection'):
+ if data.get('name') == section:
+ for track in data.get('tracks'):
+ tracks.append(self.get_track(track.get('id')))
+ return self.sanitize_tracks(tracks)
+
+ @cache()
+ def search(self, query):
+ 'SoundCloud API only supports basic query no artist,'
+ 'album queries are possible'
+ # TODO: add genre filter
+ res = self._get(
+ 'tracks.json?q=%s&filter=streamable&order=hotness' %
+ quote_plus(query))
+
+ tracks = []
+ for track in res:
+ tracks.append(self.parse_track(track, False, True))
+ return self.sanitize_tracks(tracks)
+
+ def parse_results(self, res):
+ tracks = []
+ for track in res:
+ tracks.append(self.parse_track(track))
+ return self.sanitize_tracks(tracks)
+
+ def _get(self, url):
+
+ # TODO: Optimize
+ if '?' in url:
+ url = '%s&client_id=%s' % (url, self.CLIENT_ID)
+ else:
+ url = '%s?client_id=%s' % (url, self.CLIENT_ID)
+
+ url = 'https://api.soundcloud.com/%s' % url
+
+ logger.debug('Requesting %s' % url)
+ req = self.http_client.get(url)
+ if req.status_code != 200:
+ raise logger.error('Request %s, failed with status code %s' % (
+ url, req.status_code))
+ try:
+ return req.json()
+ except RequestException as e:
+ logger.error('Request %s, failed with error %s' % (
+ url, e))
+
+ def sanitize_tracks(self, tracks):
+ return filter(None, tracks)
+
+ def parse_track(self, data, remote_url=False, is_search=False):
+ if not data:
+ return
+ if not data['streamable']:
+ return
+ if not data['kind'] == 'track':
+ return
+ if not self.can_be_streamed(data['stream_url']):
+ return
+
+ # NOTE kwargs dict keys must be bytestrings to work on Python < 2.6.5
+ # See https://github.com/mopidy/mopidy/issues/302 for details.
+
+ track_kwargs = {}
+ artist_kwargs = {}
+ album_kwargs = {}
+
+ if 'title' in data:
+ name = data['title']
+
+ # NOTE On some clients search UI would group results by artist
+ # thus prevent user from selecting track
+
+ if ' - ' in name:
+ name = name.split(' - ')
+ track_kwargs[b'name'] = name[1]
+ artist_kwargs[b'name'] = name[0]
+ elif 'label_name' in data and data['label_name'] != '':
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data['label_name']
+ else:
+ track_kwargs[b'name'] = name
+ artist_kwargs[b'name'] = data.get('user').get('username')
+
+ album_kwargs[b'name'] = 'SoundCloud'
+
+ if 'date' in data:
+ track_kwargs[b'date'] = data['date']
+
+ if remote_url:
+ track_kwargs[b'uri'] = self.get_streamble_url(data['stream_url'])
+ else:
+ track_kwargs[b'uri'] = 'soundcloud:song;%s' % data['id']
+
+ track_kwargs[b'length'] = int(data.get('duration', 0))
+
+ if artist_kwargs:
+ artist = Artist(**artist_kwargs)
+ track_kwargs[b'artists'] = [artist]
+
+ if album_kwargs:
+ if 'artwork_url' in data and data['artwork_url']:
+ album_kwargs[b'images'] = [data['artwork_url']]
+ else:
+ image = data.get('user').get('avatar_url')
+ if image:
+ album_kwargs[b'images'] = [image]
+ else:
+ album_kwargs[b'images'] = []
+
+ album = Album(**album_kwargs)
+ track_kwargs[b'album'] = album
+
+ track = Track(**track_kwargs)
+ return track
+
+ def can_be_streamed(self, url):
+ req = self.http_client.head(self.get_streamble_url(url))
+ return req.status_code == 302
+
+ def get_streamble_url(self, url):
+ return '%s?client_id=%s' % (url, self.CLIENT_ID)
View
28 mopidy/settings.py
@@ -318,3 +318,31 @@
'rtmps',
'rtsp',
)
+
+#: Your `SoundCloud.com <http://www.soundcloud.com/>`_ authentication token.
+#:
+#: Get yours at http://www.mopidy.com/authenticate
+#:
+#: Used by :mod:`mopidy.backends.soundcloud`.
+SOUNDCLOUD_AUTH_TOKEN = None
+
+#: Extra playlists from `SoundCloud.com <http://www.soundcloud.com/explore>`
+#:
+#: Note: this might take a while to load. You can add more if you want,
+#: for example:
+#: if you want Smooth Jazz from https://soundcloud.com/explore/jazz%2Bblues
+#: your entry would be u'jazz%2Bblues/Smooth Jazz'
+#:
+#: Default::
+#:
+#: SOUNDCLOUD_EXPLORE = [
+#: u'electronic/Ambient',
+#: u'pop/New Wave',
+#: u'rock/Indie',
+#: ]
+#: Used by :mod:`mopidy.backends.soundcloud`.
+SOUNDCLOUD_EXPLORE = [
+ u'electronic/Ambient',
+ u'pop/New Wave',
+ u'rock/Indie',
+]
View
3  requirements/soundcloud.txt
@@ -0,0 +1,3 @@
+requests >= 1.1.0
+# Requests is an Apache2 Licensed HTTP library, written in Python, for human beings.
+# Available as the python-requests package from Ubuntu universe.
View
1  tests/backends/soundcloud/__init__.py
@@ -0,0 +1 @@
+from __future__ import unicode_literals
View
52 tests/backends/soundcloud/playlists_test.py
@@ -0,0 +1,52 @@
+# encoding: utf-8
+
+from __future__ import unicode_literals
+
+from mopidy import settings
+from mock import patch
+from tests import unittest
+from mopidy.models import Playlist
+from mopidy.backends.soundcloud import soundcloud
+from mopidy.backends.soundcloud.playlists import SoundCloudPlaylistsProvider
+
+
+class SoundCloudClientTest(unittest.TestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ settings.SOUNDCLOUD_AUTH_TOKEN = '1-11-1111'
+ settings.SOUNDCLOUD_EXPLORE = []
+ with patch.object(soundcloud.SoundCloudClient, '_get') as get:
+ get.return_value.status_code = 200
+ get.return_value.content = {'username': 'mopidy', 'user_id': 1}
+ cls.sc_api = soundcloud.SoundCloudClient(settings.SOUNDCLOUD_AUTH_TOKEN)
+ cls._scp = SoundCloudPlaylistsProvider(backend=cls)
+
+ @classmethod
+ def tearDownClass(cls):
+ cls._scp = None
+ settings.runtime.clear()
+
+ def test_explore_returns_empty(self):
+
+ 'create_explore_playlist should return empty tracks,\
+ when stream-able is False'
+ result = self._scp.create_explore_playlist('mopidy;Love is in the air')
+ self.assertIsInstance(result, Playlist)
+ self.assertEqual(result.tracks, ())
+ self.assertEqual(result.uri, 'soundcloud:exp-mopidy;Love is in the air')
+ self.assertEqual(result.name, 'Explore Love is in the air on SoundCloud')
+
+ def test_lookup_sets_returns_tracks(self):
+
+ 'lookup must return tracks, when lookup is soundcloud:set-*'
+ self._scp._playlists = [Playlist(
+ uri='soundcloud:set-love',
+ name='Sets',
+ tracks=['track1']
+ )]
+ result = self._scp.lookup('soundcloud:set-love')
+ self.assertIsInstance(result, Playlist)
+ self.assertEqual(result.tracks, ('track1',))
+ self.assertEqual(result.uri, 'soundcloud:set-love')
+ self.assertEqual(result.name, 'Sets')
View
45 tests/backends/soundcloud/trackparser_test.py
@@ -0,0 +1,45 @@
+# encoding: utf-8
+
+from __future__ import unicode_literals
+
+from mock import patch
+from tests import unittest
+from mopidy.backends.soundcloud import soundcloud
+
+
+class SoundCloudClientTest(unittest.TestCase):
+
+ def setUp(self):
+ with patch.object(soundcloud.SoundCloudClient, '_get') as get:
+ get.return_value.status_code = 200
+ get.return_value.content = {'username': 'mopidy', 'user_id': 1}
+ self.SCC = soundcloud.SoundCloudClient('1-11-1111')
+
+ def test_get_user(self):
+
+ "get_user should return username mopidy"
+
+ with patch.object(soundcloud.SoundCloudClient, '_get') as get:
+ get.return_value = {'username': 'mopidy', 'user_id': 1}
+ self.assertEqual(self.SCC.get_user().get('username'), 'mopidy')
+
+ def test_return_none_no_data(self):
+
+ "parse_track should return None if data is empty"
+
+ self.assertEqual(self.SCC.parse_track(None), None)
+
+ def test_return_none_if_not_track(self):
+
+ "parse_track should return None if data is not track or not streamable"
+
+ payload = {'kind': 'playlist', 'streamable': False}
+ self.assertEqual(self.SCC.parse_track(payload), None)
+
+ def test_sanitize_tracks(self):
+
+ "sanitize_tracks should return only valid data"
+
+ payload = ["Track A", None, "Track B"]
+ result = ["Track A", "Track B"]
+ self.assertEqual(self.SCC.sanitize_tracks(payload), result)
Something went wrong with that request. Please try again.