Consistent broken payloads with WebcastLinkMicBattle

[shaiyon]

Describe the bug

Consistent broken payloads with WebcastLinkMicBattle events. Sometimes it goes through normally, but around half the time this broken payload decoding varint error comes up. Could this be a new schema that hasn't been accounted for yet, maybe the battle ending? Don't mind tackling this myself if you can point me in the right direction.

Expected Behaviour

Tracking a battle beginning or ending.

Actual Behaviour

[TikTokLive] ERROR from U.s.D.H.T.T.c.client.py:406 — Traceback (most recent call last):
  File "/Users/shaiyon/Documents/project_name/TikTokLive/TikTokLive/client/client.py", line 404, in _parse_webcast_response
    proto_event: ProtoEvent = event_type().parse(response.payload)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1015, in parse
    value = self._postprocess_single(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 955, in _postprocess_single
    value = cls().parse(value)
            ^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1015, in parse
    value = self._postprocess_single(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 955, in _postprocess_single
    value = cls().parse(value)
            ^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 1008, in parse
    decoded, pos = decode_varint(parsed.value, pos)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shaiyon/Documents/project_name/project_name/lib/python3.12/site-packages/betterproto/__init__.py", line 487, in decode_varint
    raise ValueError("Too many bytes when decoding varint.")
ValueError: Too many bytes when decoding varint.

Broken Payload:
b'\n6\n\x14WebcastLinkMicBattle\x10\x91\x96\x8a\xe8\x8a\xd1\xfa\xbaf\x18\x9f\x96\x88\xc4\xd4\xeb\xf7\xbaf \xf9\x81\xf1\xe4\x832(\x01\xd0\x01\x01\x10\x90\x96\xa9\x96\xce\xaa\xfa\xbaf\x1a"\x08\x90\x96\xa9\x96\xce\xaa\xfa\xbaf\x10\xc9\xb5\xde\xe4\x832\x18\xad\x02 \x90\x96\x86\xdc\xad\xa3\xfa\xbaf(\x03@\x01 \x05*\x1a\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x0e\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x10\x01\x18g*\x19\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\r\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x18\xd4\tJ\xd2\x0f\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\xc5\x0f\n\x8f\x05\x08\x86\x88\xbf\xbe\xe6\x8a\xe9\x9aa\x10"\x1a\x0bWeedman 902"\xde\x04\n\x8c\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc952~c5_100x100.webp?lk3s=a5d48078&nonce=84167&refresh_token=374e5f3f0b9a24d454b8401ff9b53ea3&x-expires=1719176400&x-signature=0Qxc%2FC5VdRsIHgWy1SnS5clhUIA%3D&shp=a5d48078&shcp=d2876c74\n\x8e\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc952~c5_100x100.jpeg?lk3s=a5d48078&nonce=35952&refresh_token=ba58d331c490adf86203d6c51cbdb0e2&x-expires=1719176400&x-signature=wRPa4W%2FxNevnV1Md%2BraqaFttUYU%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/612ef28ac1850f3be59cc872f5ddc9522\x137004685486460486662\n\x95\x05\x08\x85\x88\xd8\x84\xc0\xb0\xe3\xfa^\x10\x13\x1a\x11McDaddy Rico \xf0\x9f\x98\x9c"\xde\x04\n\x90\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a28~c5_100x100.webp?lk3s=a5d48078&nonce=45835&refresh_token=78f8bfddbcaafea70c9f2096015e1c0f&x-expires=1719176400&x-signature=XnQi%2BCjk%2Bau8pn36VirHu9h3%2Bm4%3D&shp=a5d48078&shcp=d2876c74\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a28~c5_100x100.jpeg?lk3s=a5d48078&nonce=85970&refresh_token=167fa817f6a7ab8add1492ee3dfef286&x-expires=1719176400&x-signature=WCdnp0TD8Bk2XQP7Dgi67Oe9Q6Q%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/d7eda1280decc4753f4379f4cf9c8a282\x136842530806944367621\n\x82\x05\x08\x86\x88\xd9\xa0\xe9\xca\xfe\xc4_\x10\x04\x1a\x04Matt"\xd8\x04\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d8~c5_100x100.webp?lk3s=a5d48078&nonce=91314&refresh_token=025cfb9b99037eb70e2a4412a1f27639&x-expires=1719176400&x-signature=FD1r9vjXJ7LED7EFm9mZqOtqJTE%3D&shp=a5d48078&shcp=d2876c74\n\x8a\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d8~c5_100x100.jpeg?lk3s=a5d48078&nonce=58945&refresh_token=5aac7f50d07c95c5fb988536acd0c3ae&x-expires=1719176400&x-signature=O0o5KcDxBSWSLTkKs4tlY4QtYpU%3D&shp=a5d48078&shcp=d2876c74\x12<100x100/tos-maliva-avt-0068/c8bfdd9fb5dd79db486a6002275271d82\x136884308755171132422\x10g\x1a\x12191139995515252736J\xe0\x10\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\xd3\x10\n\xbb\x05\x08\x9a\x88\xb0\xac\x83\xaf\xcf\xc1d\x10\x84\x01\x1a\x17\xe2\x9c\x8c\xef\xb8\x8f only pind \xe2\x9c\x8c\xef\xb8\x8f"\xfd\x04\n\x9a\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a7~c5_100x100.webp?lk3s=a5d48078&nonce=49965&refresh_token=3c437998130aceeda15c044634a4510d&x-expires=1719176400&x-signature=tkRA%2BTMJmTGRHJnosiXSlt382p4%3D&shp=a5d48078&shcp=d2876c74\n\x97\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a7~c5_100x100.jpeg?lk3s=a5d48078&nonce=3339&refresh_token=70c01f3aff4ba2562ba535e411a178fe&x-expires=1719176400&x-signature=gdZQkrFPsNqu2YyrGGVmRDyw1nI%3D&shp=a5d48078&shcp=d2876c74\x12D100x100/tos-useast2a-avt-0068-euttp/dfdb58b99833dc3bf6f28876c31e20a72\x137242700212245627930\n\xca\x05\x08\x9a\x88\x81\xf6\xc5\xf8\xe2\xdad\x10+\x1a \xd8\xb4\xdb\x81\xd8\xb1\xdb\x8c\xd8\xa7\xd8\xb1\xf0\x9f\x96\xa4\xf0\x9f\x87\xb5\xf0\x9f\x87\xb0\xf0\x9f\x87\xae\xf0\x9f\x87\xb9"\x84\x05\n\x9a\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a~c5_100x100.webp?lk3s=a5d48078&nonce=45406&refresh_token=98efd4b9a64574d4474925d709fc206e&x-expires=1719176400&x-signature=zJjOUAMlJdA%2FBhU6mk7ubMGQh2w%3D&shp=a5d48078&shcp=d2876c74\n\x9e\x02https://p16-sign-useast2a.tiktokcdn.com/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a~c5_100x100.jpeg?lk3s=a5d48078&nonce=43494&refresh_token=674b71e9c15cf1fa21822b19067767a7&x-expires=1719176400&x-signature=3iMC3k8NSzFVa%2BWtLnF%2FJr%2FVqEE%3D&shp=a5d48078&shcp=d2876c74\x12D100x100/tos-useast2a-avt-0068-euttp/ec3a871ab0731dec13e6f49dcef0952a2\x137256860050096931866\n\xad\x05\x08\x82\x88\xb8\xde\x9e\xf6\x9b\xf3d\x10\x15\x1a+\xf0\x9d\x95\xaf\xf0\x9d\x96\x8a\xf0\x9d\x96\x8b\xf0\x9d\x96\x86\xf0\x9d\x96\x9a\xf0\x9d\x96\x91\xf0\x9d\x96\x99\xf0\x9d\x96\x8a\xf0\x9d\x96\x97 \xe2\x9a\x94\xef\xb8\x8f"\xdc\x04\n\x8e\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb90.webp?lk3s=a5d48078&nonce=97307&refresh_token=06350644f79b0267366fa174dd92212f&x-expires=1719176400&x-signature=sKsq9pdRkFUEoHm96H%2BatUfLXIo%3D&shp=a5d48078&shcp=d2876c74\n\x8b\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb90.jpeg?lk3s=a5d48078&nonce=4507&refresh_token=d2b3d9b541d6d8ed21f9b9809832970a&x-expires=1719176400&x-signature=feSYTqEpzF6ZKPvjLeqp5tfy8ZI%3D&shp=a5d48078&shcp=d2876c74\x12;100x100/tos-alisg-avt-0068/880a4e347cba416093dca474eb3cbb902\x137270621458392286210\x10\xd4\t\x1a\x137146125739820598273R\x8d\x05\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x80\x05\n\xfd\x04\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x08Ali Jutt\x1a\xd8\x04\n\x8d\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a.webp?lk3s=a5d48078&nonce=5851&refresh_token=ed9c80d14d540205f33c6765fc2a0e25&x-expires=1719176400&x-signature=ZPBwd6o9qEccH5K8ax7A%2BDMxgpI%3D&shp=a5d48078&shcp=d2876c74\n\x90\x02https://p16-sign-sg.tiktokcdn.com/aweme/100x100/tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a.jpeg?lk3s=a5d48078&nonce=36074&refresh_token=0a27140104c3f0844eedf0d37f679b39&x-expires=1719176400&x-signature=m6RFfrjBL%2F8DlG%2Fz71b0Xr2FBL4%3D&shp=a5d48078&shcp=d2876c74\x123tos-alisg-avt-0068/26c7da7443e82dc7520571415689543a"\x0cali_jutt1982R\x89\x05\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\xfc\x04\n\xf9\x04\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x06Tori V\x1a\xd5\x04\n\x8d\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579~c5_100x100.webp?lk3s=a5d48078&nonce=2537&refresh_token=e139ec435edd7cbc07cc99838f08736e&x-expires=1719176400&x-signature=B5VNzKSI3WzL3Xi07%2F38%2F3XgYPk%3D&shp=a5d48078&shcp=d2876c74\n\x8c\x02https://p16-sign-va.tiktokcdn.com/tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579~c5_100x100.jpeg?lk3s=a5d48078&nonce=37294&refresh_token=4ba4c2710ae7f9698b6476b10c1823c7&x-expires=1719176400&x-signature=rIqT4trBOabq4S84dcPBywmHi%2BI%3D&shp=a5d48078&shcp=d2876c74\x124tos-maliva-avt-0068/285745437d005f2527db37af8f0b2579"\rtorivofficialj\x9f\x03\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x12\x92\x03\x08\x81\x88\xa1\x92\xfa\xef\x88\x96c\x10\x01\x18\x01(\x012\xff\x02sslocal://webcast_lynxview_popup?url=https%3A%2F%2Flf16-gecko-source.tiktokcdn.com%2Fobj%2Fbyte-gurd-source-sg%2Ftiktok%2Ffe%2Flive%2Ftiktok_live_revenue_match%2Fpages%2Flive_match_streak%2Ftemplate.js&height=832rpx&radius=16rpx&container_bg_color=FFFFFF&bdhm_bid=tiktok_live_revenue_match&bdhm_pid=tiktok_live_revenue_match_pages_live_match_streak&server_room_id=7383053092951968513j\x9d\x03\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x12\x90\x03\x08\x80\xe0\xb9\xf8\x82\x9a\xc4\xd3\x02\x10\x02(\x012\xff\x02sslocal://webcast_lynxview_popup?url=https%3A%2F%2Flf16-gecko-source.tiktokcdn.com%2Fobj%2Fbyte-gurd-source-sg%2Ftiktok%2Ffe%2Flive%2Ftiktok_live_revenue_match%2Fpages%2Flive_match_streak%2Ftemplate.js&height=832rpx&radius=16rpx&container_bg_color=FFFFFF&bdhm_bid=tiktok_live_revenue_match&bdhm_pid=tiktok_live_revenue_match_pages_live_match_streak&server_room_id=7383052755885230879'

Package Version

Latest code from main.

Operating System

MacOS

Anything Else

Thank you!

[isaackogan]

Hi Shaiyon,

Yes it's likely precisely that. @jwdeveloper @kholerpop1 do you have any updated proto files? I'd like to sift through and update the proto.

Shaiyon, if you would like to tackle this yourself you should likely fork the library and intercept the payload before it becomes a dictionary. Encode this in base64 and use https://protobuf-decoder.netlify.app/ to read the raw structure. You can look for the difference in the proto structure and update it manually if you would like. I'll be happy to merge that until we get a 'real' proto update.

[kohlerpop1]

@isaackogan https://github.com/jwdeveloper/TikTokLiveJava/tree/master/API/src/main/proto has our most up to date proto. I have made some changes recently regarding WebcastLinkMicBattle however I do not claim it to be perfect.

P.S. You spelled my name wrong! Lol

[isaackogan]

@isaackogan https://github.com/jwdeveloper/TikTokLiveJava/tree/master/API/src/main/proto has our most up to date proto. I have made some changes recently regarding WebcastLinkMicBattle however I do not claim it to be perfect.

P.S. You spelled my name wrong! Lol

No one is perfect. For example, you got your name wrong. I got it right. I name you kholer as of today.

[shaiyon]

@isaackogan seems like the .proto files are a bit outdated compared to the Java version, so I updated those to match. Having some trouble identifying which scripts to run to build the docs/python/ts files, where should I start?

[isaackogan]

@shaiyon dont worry about the docs, make a PR and I'll build them. It's a shitshow from 2y ago...