Json parsing crash in get_segments_internal

[stephen304]

Here's a new crash on my long-running castblock, running commit 243e756 , it seems to be related to the json parsing in https://github.com/erdnaxeli/castblock/blob/main/src/sponsorblock.cr#L49

[erdnaxeli]

Maybe the API have changed. I will look at it, thanks for the report.

[erdnaxeli]

Hmm I can't reproduce it. I added some logging and published new images, if it happens again please comment again with the logs.

At least now if this happens it will retry to download segments.

[stephen304]

Will try that thanks. I had tried to log the payload but the last payload was 404 and more than 30 seconds prior so it might not have been it. It might just be an intermittent issue with the api but hopefully we'll find out.

…

On Mon, Nov 29, 2021, 7:35 AM Alexandre Morignot ***@***.***> wrote: Hmm I can't reproduce it. I added some logging and published new images, if it happens again please comment again with the logs. At least now if this happens it will retry to download segments. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#37 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHPXG4WRX6OGZOZFYACSV3UONXRDANCNFSM5I4SDSNQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[stephen304]

Here is the debug log:

Nov 29 13:23:37 pihole castblock[115378]: 2021-11-29T18:23:37.564760Z ERROR - castblock.sponsorblock: Error when reading JSON from Sponsorblock -- error: "Unexpected char '\u001F' at line 1, column 1", json: "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0000\u0003$\xCEMn\xC20\u0010\u0006л|k\v\xCD\xF8/\u0013\xAF\xD9t_V\x88\x85=\u001EPDKP\u0012\x90PջWmO\xF0\xDE\xF1\vZ7\xBB\xCC\xCB\v\u0005\xD3m[f8Tݦ\xF9\xF6\xFE\xBA\e\n\xD6\xEBt\x87\xC3j\x97O\xBBm(ǜwa\u001C\xDC@\xBB\u001C\xE4\xE4p8\xBC\xEDQ\x90DX\xABJ\xF3\x83x\u0019{\xEB<\x92)\x87\x98\xD9s&\xF3>\xE6\xD8\u0013S\xB3@\xA6C7\xB3@Lާ\xE0\xAB)\u001C>f\xBDZGa\x87\xE7\xBCيB\u000Eϩۼ\u007F,\xF57\x85\x928\xEERd\x87\xC7j˟<\xF6\xA41\x86\xDEc\rYZ\u0014&k\xE7\xACL\xE1\xCCV){\u0011\xABF\x92H$6\xEA\xEA\a\xC9!\xA7\xA6\x91)\x8Fp\xE8\xB6\xEA2\xDD\xFF\u0005\xE0\xFB\xF4\u0003\u0000\u0000\xFF\xFF\u0003\u0000X\u0016\u001Dv\u0019\u0001\u0000\u0000"

The video ID is CaLOiGEDPJQ which is 404, and trying again with the same video it's fine. Maybe something weird happens occasionally with the API on videos without segments...

Here's another one, looks like it's different junk each time (this time with a backtick so I block quoted it)

Nov 29 14:35:11 pihole castblock[115378]: 2021-11-29T19:35:11.418729Z  ERROR - castblock.sponsorblock: Error when reading JSON from Sponsorblock -- error: "Unexpected char '\u001F' at line 1, column 1", json: "\u001F\x8B\b\u0000\u0000\u0000\u0000\u0000\u0000\u0003$λn\xC30\f@\xD1\u007F\xE1,\u0004zҢ\xE7,ٛ)\xC8@Qt`\xA4\xB5\f\xCB\t\u0010\u0014\xFD\xF7\"\xEDz\x97s/\xDF \xBC\xEB\xADm/\u0018\xA1\xAFm\xE9m\u0003\u0003,\xFBܖ\x8Fת\xEF|\x9FW0\xD0\xF5\xF6\xA5\xCB\u000E\xE3\xC5\a<\u0004g|v\x87\xE8\xC3\xD5\xC0\xF9|:\xC2\b\x89S\xA9\x91C\xCD\u0018J\u0015\x9Fu\xCAI2\xD6\"6\x95RT\tUp\xB29\xDB:y\x97À\xEC\x88XBƉ\x82\u0003\u0003\x9FM\xEEZat\u0006\x9Em\xD7\u000E\xA3\xB3\u0006\x9Es\xD5v|l\xFCނ1\u000E\xE1\xE0\xA33\xF0\xE8\xBA\xFDєi\xA8\xA1\bFbtL^\xBD\x948\u0014\xA6\x9C\xAA-\xC2d\x85(:\xAE\xB9\xF8\xA8\u008E\u0019\u0013S\xC54\xA0%\u000F\u0006\xAAv\xD9\xE6\xF5_\u0000\xF8\xB9\xFE\u0002\u0000\u0000\xFF\xFF\u0003\u0000'\x8B\xF2\x94\u001D\u0001\u0000\u0000"

[erdnaxeli]

Wow, I didn't expect that 😮

I will try to look at it by the end of the week.

[mchangrh]

the response for skipSegments has changed but I don't think it should invalidate any JSON schemas

ajayyy/SponsorBlockServer#380
ajayyy/SponsorBlockServer#398

[erdnaxeli]

Those new fields should just be ignored.