Getting higher usage rate limits #644
Comments
|
Requesting a users recently played tracks once per minute seems unnecessary for your use case. A track must be played for at least 30 seconds to be included in the playback history, and you can fetch up to 50 recently played items. Thus, it follows that you could safely limit your requests to once every 24 minutes per user without missing anything. With 400 users, this would work out at one request every 3.6 seconds, which I'm sure wouldn't trigger Spotify's rate limiting policy. Play history objects from the web API include a timestamp which you can pass directly on when scrobbling, so the user's Last.fm profile will reflect accurate listening times. Of course, you sacrifice a feeling of realtime processing, but when you don't have access to realtime APIs there's not much you can do about it. Processing data in guaranteed-to-work batch updates is better than an app which stops functioning entirely due to rate limiting, and you can make it clear to your users that updates to their Last.fm profile will not be immediate. |
|
I have already thought about such a solution but 24 minutes are a lot. It would completely remove any "feeling of realtime". That’s why it’s not realy a good solution to me. Spotify offers full lastfm support in the desktop/smartphone clients but not in the connect devices which are used a lot nowadays. I dont see why scrobbling should be either not possible or that poorly supported. |
|
Wouldn't requiring users to log in spread out rate limits equally on users? I'm aware rate limits are lower then, but the total might still be better. Does it make sense? |
|
@andersborgabiro Users do have to login in order to grant the required permissions. That does not change anything about the rate limit / api key. |
|
But don't you get a rate limit per user then, so it scales with amount of users? My app would probably also break if that's not the case. |
|
Im not sure what exactly you mean. My serice only asks for permission once. So I just use the token and refresh it every time I need to make a request. "Rate limiting is applied on an application basis (based on client id), regardless of how many users are using it." |
|
I probably don't understand your use case. Anyway, I rewrote my app yesterday to use an absolute minimum of requests, in cases by chunking data to the limit of each request. That also sped up the app significantly. This might not apply in your case. |
|
Your app has an per app limit. It does not matter how many users you have. At least that's what Spotify says. I cant decrease my requests without making my service less useful and even if I could it wouldnt fix the problem. It would just give me a month and then I would hit the limit again because I get new users every day. I'll probably just stop working on the project. There is no point in putting effort in a project if it is already doomed to failure. Another service which was doing pretty much the same as mine has already been shut down because of the rate limit. Sorry, but I'm extremely disappointed by Spotify and so are most of the users. If Spotify is not willed to offer simple features, they should at least support developers who do. |
|
In my case it didn't change what the app did, it only decreased the amount of requests, but with many users (and how the rate limit works) I will eventually hit the roof anyway. Registering several apps and switching between them randomly per user might help for a while, but such manual configuration doesn't scale well. Spotify could of course increase this easily, so maybe nagging helps. My app is based on a long-time request from users for a certain feature, so as you say, others are complementing Spotify with features they should already support. |
|
I need to request data for every single user that's why I need so many requests. The only temporary solution would be to request the data only every 10 minutes which would kill the feel of realtime completely. And even that would only fix the problem for a while. Switching app keys is not realy a solution since it's not allowed and it wont work afaik. The limit is bound to your servers ip. Spotify even says that they are willed to increase the rate limit in "special cases" but there is not even an email address given. |
|
So that means if I make several completely different Spotify server-based apps they would share the same rate limit, unless they would be hosted on different servers. That's bad. Direct access from a mobile app would use different IP addresses though. |
|
At least afaik. Even if it would work, it would certainly not be allowed. |
|
Hi! If you have a use case that requires higher rate limits, please get in touch with us via dm on Twitter @SpotifyPlatform. Polling recently played once per minute doesn't strike me as a use case that we're likely to increase rate limits for though to be completely honest. We're aware of feature requests for real time playback notifications, which is likely to be a better solution for you this use case. |
When a spotify user registers with listen along, we are polling spotify for their playback state every 5 seconds. Spotify rate limits API requests in a vague way. https://developer.spotify.com/documentation/web-api/ (Search that page for "Rate Limiting" for the relevant section) It's suggested here that the rate limit would be hit after making requests for 400 users every 3.6 seconds. spotify/web-api#644 This is useful information because listen along makes a request for each user every 5 seconds. So we'll start hitting the rate limit around that number of users. There was another github issue page where someone said they hit the API rate limit when making requests for each of 400 users every 10 seconds. I can't find the link to that other source but I think given the two sources, the information is probably true. It may not be. This commit may be a waste of time. However, that spotify page (https://developer.spotify.com/documentation/web-api/) also says the rate limiting is done on a per client id basis. Client IDs correspond to apps registered in the spotify developer dashboard. It looks like I can create an unlimited number of these apps. There may be a limit, however I've had ten apps registered with spotify before on a single account. I also pay for two accounts to test listen along. This commit allows me to dynamically assign users different spotify registered apps so that I can work around the rate limiting. It feels a bit sleezy, but it'd be better for everyone (spotify included for less traffic) if spotify had some sort of web socket interface to reach out to me and let me know when someone does something like skips a song or pauses, which they don't do. So this is my only option if I'd like to scale above ~400 users. If my monitoring alerts send me an email saying I'm hitting the rate limit on the spotify apps I'm using for listen along, I can manually create new apps in the dashboard, add them to listen along as `SpotifyApp` models and they'll load balance themselves by always registering new users in the `SpotifyApp` with the fewest users attached. It requires the occasional manual process, but only every 400 users or so and it's the best option as far as I can see. Relevant Github Issue: spotify/web-api#492
|
@gLarsVegas How did you handle this? |
|
@gLarsVegas ^ I am interested as well |
I built a SDA which offers users to keep track of their spotify playback and additionally scrobble it to last.fm. Due to the high demand for such a service, a lot of users registered. Currently the service has about 400 users. In the last few days the service started hitting the API rate limit. Obviously I have to request the current/recent playback quite often since it can’t be cached for hours or even days like meta data or something similar. I tried to find a reasonable compromise so I request the playback every minute. But due to the climbing amount of users that’s too often for the rate limit.
I was looking for some contact email address when I stumbled upon this issue comment by @thelinmichael #228 (comment)
Can anybody give me more details about how I can apply for such a "special case".
The text was updated successfully, but these errors were encountered: