Can't play songs larger than 32 MB

[derat]

Requests to the /song_data endpoint for large files result in 500 errors.

The client sends a request with Range: bytes=0-. In the App Engine logs, I can see that the server dutifully reads and returns the data without reporting any problems, e.g. Creating reader for bucket:song.mp3 with size 55596839. However, dev tools shows that Chrome is getting back a 500.

Per https://cloud.google.com/appengine/docs/standard/go111/how-requests-are-handled:

Dynamic responses are limited to 32MB. If a script handler generates a response larger than this limit, the server sends back an empty response with a 500 Internal Server Error status code. This limitation does not apply to responses that serve data from the Blobstore or Cloud Storage .

I don't understand what the "does not apply" clause is referring to. Is it not talking about App Engine anymore, and instead referring to requests to a Cloud Storage endpoint? The linked page doesn't have any useful information.

[derat]

https://support.google.com/chrome/thread/25510119?hl=en has some discussion about how Chrome requests media files. It sounds like the initial full-range request may just be used to check whether the server supports range requests, and might be aborted before the download finishes.

However, this doesn't exactly match what I'm seeing. With a smaller file, the server returns a 206 with Content-Range: bytes 0-9366265/9366266, and I'm not seeing Chrome make any other requests. I don't think I've noticed Chrome ever actually making use of range requests with music files -- it seems like it just asks for bytes=0- and gets the full file. When I initially wasn't handling range requests on the server, Chrome wasn't supporting seeking, though, so it seems necessary to reply with a 206.

Maybe Chrome only aborts the initial request if the file is huge (e.g. video).

[derat]

More discussion at https://stackoverflow.com/questions/39961598/video-seeking-in-google-chrome-how-to-handle-continuous-partial-content-request.

[derat]

Sigh. As best as I can tell, Chrome's networking/media-fetching code is buggy and inefficient, and devtools and/or App Engine logging are full of lies.

When I try to play a 29127465-byte file, dev tools claims that Chrome is sending a single request with Range: bytes=0- that receives a 206 with Content-Range: bytes 0-29127464/29127465 and X-Cloud-Trace-Context: 09ad9285d9700617e833f38eef8a339b;o=1. The Timing tab still says CAUTION: request is not finished yet! after I've been listening to the song for 5 minutes. App Engine says that it sent the full file for trace_id=09ad9285d9700617e833f38eef8a339b, but it shows another request 12 seconds later for trace_id=2af7011c170c86b817c59952630cf5b2 sending the full file, and then another a second after that for trace_id=a859e2f2593b489c7b519341a75c803f again with the full file.

I can seek forward to the end of this file (48:32) without any more requests being made, but when I seek backward devtools shows a bunch of new requests being sent:

• range: bytes=19628032-
• range: bytes=19529728-
• range: bytes=19431424-
• range: bytes=19333120-
• range: bytes=19234816-
• range: bytes=19136512-
• range: bytes=19038208-
• range: bytes=18907136-
• range: bytes=18808832-
• range: bytes=18710528-

All of those but the last one are red, which I assume is Chrome's way of saying it aborted them. The last one is shown as successful, with the server returning a 206 with Content-Range: bytes 18710528-29127464/29127465.

In the App Engine logs, I can see that all of those results actually got sent to the server and resulted in big reads from Cloud Storage.

[derat]

When I play a song with size 55596839, Chrome just shows a single request for range bytes=0-39387135 with trace ID feba9de9ee1bd1bd2c84267db4aa0bf7. The specific range is weird -- why isn't Chrome sending its usual bytes=0- here? Chrome and App Engine agree that the server sent back 0-33521663/55596839, but Chrome just shows a single request while the server shows two requests: feba9de9ee1bd1bd2c84267db4aa0bf7 and then d2dd1f90d65c864b12005324f3c59061 with the same 31.97 MB size.

15 minutes later, Chrome shows another request, this one for bytes=33521664-. The server just shows a single request this time, this one resulting in a 21.05 MB response with trace ID e5db23328d24bd1b95387b6d582cefa3 and range 33521664-55596838/55596839.

Cloud Logging's show-new-logs-when-scrolling-down behavior is super-buggy and often appends old logs instead of new ones, and App Engine requests often get split across multiple top-level entries, some of which are missing trace IDs, so it's really hard to figure out what's actually going on here.

[derat]

I think that this works in the web and Android clients now.