Skip to content
This repository has been archived by the owner on Jan 22, 2022. It is now read-only.

webclient.get_all_songs and get_all_playlists broken due to Google changes #221

Closed
bubbleguuum opened this issue Feb 26, 2014 · 19 comments
Closed

webclient.get_all_songs and get_all_playlists broken due to Google changes #221

bubbleguuum opened this issue Feb 26, 2014 · 19 comments

Comments

@bubbleguuum
Copy link

It seems the /music/services/loadalltracks and /music/services/loadplaylist endpoints do not work anymore and return a 404.
For fetching tracks, the Google Music Web app make this request instead:

https://play.google.com/music/services/streamingloadalltracks?u=0&xt=CjUKATASMEFNLVdiWGpyYlF6Ym8yeWJCRmZROTBsZnJaRkVDOHRNUFE6MTM5MzQzNzYyNTAyOA%3D%3D&json=%7B%22tier%22%3A2%2C%22requestCause%22%3A1%2C%22requestType%22%3A1%2C%22sessionId%22%3A%22uc59ldv9aj8v%22%7D&format=jsarray

Which returns some html and javascript containing the tracks json:

<html><head></head><body>
<script>window.parent['slat_progress'](0.02);</script>
<script type='text/javascript'>
window.parent['slat_process']([[["63f1b9d1-94e8-3c6d-8fb6-0b57f9416d03","Strider's World",,"Omar-S","Just Ask The Lonely","",,,,,"Fxhe Records (005)","Techno",,513000,4,10,1,1,2005,0,,,2,0,1352827997810338,1352828328741433,,,,2,"",,,"A3snjywo67zzg7swl7keobkhxjy",243,1352839630711000,"//lh3.googleusercontent.com/c0mNO8Wjn_rnoaTlftjt9Om0XUDz974b_PxFqqYPXSwf8DDX2DGqeS9b3CP3q8mcGLS425YSJQ",,,[]
],
...
</html>
@bubbleguuum
Copy link
Author

here's the decoded GET request:

https://play.google.com/music/services/streamingloadalltracks?u=0&xt=CjUKATASMEFNLVdiWGpyYlF6Ym8yeWJCRmZROTBsZnJaRkVDOHRNUFE6MTM5MzQzNzYyNTAyOA==&json={"tier":2,"requestCause":1,"requestType":1,"sessionId":"uc59ldv9aj8v"}&format=jsarray

@bubbleguuum
Copy link
Author

After testing, changing the code to make the request below (with the usual auth headers but without that json query parameter which doesn't seem to do anything) seems to work and return the html mentionned in the first post. After that, it should just be a matter of extracting the json from the html.

https://play.google.com/music/services/streamingloadalltracks?u=0&xt=CjUKATASMEFNLVdiWGpyYlF6Ym8yeWJCRmZROTBsZnJaRkVDOHRNUFE6MTM5MzQzNzYyNTAyOA==&format=jsarray

@simon-weber simon-weber mentioned this issue Feb 27, 2014
Open
@simon-weber
Copy link
Owner

Thanks for looking into this. There were a lot of changes -- the mobileclient was broken as well (#220).

some comments:

  • json in the query string, huh? that's pretty funky
  • the jsarray format makes me sad. Have you tried sending format=json instead?

@danielmj
Copy link

Has anyone figured out how to do continuations?

EDIT: Whoops I didnt realize that this request pulls all song data. No more need for continuation tokens

@danielmj
Copy link

Simon, I've created a recursive Objective-C class to parse jsarrays. It works pretty well for what I need. Would you like it?

@bubbleguuum
Copy link
Author

format=json returns an error 400 and omitting it as well.

@ankurwcities ankurwcities mentioned this issue Feb 27, 2014
Closed
@ankurwcities
Copy link

Does anybody know parameter at which index corresponds to continuationToken in this new response?

@jkiddo
Copy link

jkiddo commented Feb 27, 2014

I made a quick hack to get it working again in my project. It ain't pretty, but it works for now, and I'm able to download songs once again

@ankurwcities
Copy link

Hi jkiddo, I think you removed the recursive call to getSongs() with a valid continuationToken & hence it won't work for more than 1000 songs. Is this true?

@reduxd reduxd mentioned this issue Feb 28, 2014
Closed
@jkiddo
Copy link

jkiddo commented Feb 28, 2014

@ankurwcities - as @danielmj states, the continuationToken is no longer used - which means that all songs are fetched in a single call

@ankurwcities
Copy link

Actually I tested for 1060 songs out of which it could retrieve only 1000 songs. That's the reason I was worried. Is it working for you for more than 1000 songs?

@jkiddo
Copy link

jkiddo commented Feb 28, 2014

Well ... can't say anything about that - I tested on my own account and I fetched more that 5000 tracks

@ankurwcities
Copy link

Oh, ok..It was my mistake in parsing. Response contains set of 1000 songs each starting with "['slat_process']". I had to loop for parsing of more songs.

@simon-weber
Copy link
Owner

Hm, I suppose I'll have to push a major release for this? From what you folks are describing, there's not a simple way for me to support the incremental option anymore.

re @danielmj: thanks, but I'd really like to lean on existing code if possible. The last time I looked at this I came up with a hack that used the Python tokenizer - mentioned in #129 (comment), though I don't remember where the code is edit: here it is - which I think is slightly cleaner.

@jkiddo
Copy link

jkiddo commented Mar 1, 2014

I don't recall that the field descriptors are included in the jsarray format, so you will need to supply some extra information in order to parse it as json, if im not mistaken

@simon-weber
Copy link
Owner

@jkiddo: @lukegb originally extracted them from the webclient code; I have a list here. Hopefully the definitions haven't changed, but even if they have it should be relatively straightforward for me to pull them out again.

@simon-weber
Copy link
Owner

On second thought, I think it's smarter to just remove these methods; they're duplicated in the mobileclient, and I declared them deprecated a while back.

It should really easy for current users to switch over, since - unlike with mobileclient streaming - the auth requirement is identical.

Thoughts? Would people be outraged if I just removed these? =)

@JustinB4
Copy link

JustinB4 commented Mar 7, 2014

@simon-weber I hadn't realized the methods were deprecated. When I saw your comment I went back through my scripts and updated them to use mobileclient. It didn't take too long. It seems reasonable that if you can do it with mobileclient and webclient is broken that you should just remove the weblicent methods.

@simon-weber
Copy link
Owner

@nughtin I've had a lingering todo item to actually raise DeprecationWarning, but never got around to it. Admittedly, I didn't think they'd break as soon as they did.

When I remove these, I'll add warnings to the other deprecated methods.

@p0lar-bear p0lar-bear mentioned this issue Mar 10, 2014
Closed
@jekapps jekapps mentioned this issue May 11, 2015
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@jkiddo @bubbleguuum @simon-weber @JustinB4 @danielmj @ankurwcities