Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streams going live occasionally sends online, offline, online notifications. #230

Closed
thedist opened this issue Sep 30, 2020 · 11 comments
Closed
Labels
product: eventsub EventSub (Webhooks and WebSockets)

Comments

@thedist
Copy link

thedist commented Sep 30, 2020

Brief description
When a stream goes live, there have been instances where webhook subscriptions send an Online, Offline, then Online notification. Because order isn't guaranteed it's possible some users may not receive them in that order but will still receive them.

How to reproduce
Create webhook subscriptions to the streams changed topic and wait.

Expected behavior
1 single notification when a stream goes live, unless the streamers connection itself has an outage.

Additional context or questions
Each notification has a unique notification ID, and if there was an issue with duplication that wouldn't explain the offline notification. Additionally, the payload of the live notifications are identical.

@thedist thedist added the product: eventsub EventSub (Webhooks and WebSockets) label Sep 30, 2020
@BarryCarlyon
Copy link
Contributor

BarryCarlyon commented Sep 30, 2020

I have also observed the same issue the last week or so.

I have observed issue normally occurs for streams that start 10pm BST thru 3am BST.
But it's not 100% reliable to occurs.

From today:

"twitch-notification-id":"5133eda7-c84e-4817-931f-3b049c72a85f"
{"data":[{"game_id":"510590","id":"2707678081","language":"en","started_at":"2020-09-30T22:27:26Z","tag_ids":null,"thumbnail_url":"https://static-cdn.jtvnw.net/previews-ttv/live_user_cohhcarnage-{width}x{height}.jpg","title":"Cozy Hades! - Song of Horror starts TOMORROW! Star Wars Friday! \\o/ - Creator Code: Cohh #EpicPartner - !Coffee - !Corsair - !Discord","type":"live","user_id":"26610234","user_name":"CohhCarnage","viewer_count":0}]}
"twitch-notification-id":"24db2798-0977-4d25-ae25-563903d5fd66
{"data":[]}
"twitch-notification-id":"bd8d842b-d3a3-418a-80dc-2f1f0c386da5",
{"data":[{"game_id":"510590","id":"2707678081","language":"en","started_at":"2020-09-30T22:27:26Z","tag_ids":null,"thumbnail_url":"https://static-cdn.jtvnw.net/previews-ttv/live_user_cohhcarnage-{width}x{height}.jpg","title":"Cozy Hades! - Song of Horror starts TOMORROW! Star Wars Friday! \\o/ - Creator Code: Cohh #EpicPartner - !Coffee - !Corsair - !Discord","type":"live","user_id":"26610234","user_name":"CohhCarnage","viewer_count":0}]}

@kav2k
Copy link

kav2k commented Sep 30, 2020

Seeing this as well. The notifications come through with the same stream ID:

Sep 30 19:49:55 {"data":[{"game_id":"33331","id":"39549635548","language":"en","started_at":"2020-09-30T17:49:14Z","tag_ids":null,"thumbnail_url":"https://static-cdn.jtvnw.net/previews-ttv/live_user_kikoskia-{width}x{height}.jpg","title":"Let\'s Stream XCOM: Enemy Within!","type":"live","user_id":"8421033","user_name":"Kikoskia","viewer_count":0}]}
Sep 30 19:50:36 {"data":[]}
Sep 30 19:51:07 {"data":[{"game_id":"33331","id":"39549635548","language":"en","started_at":"2020-09-30T17:49:14Z","tag_ids":null,"thumbnail_url":"https://static-cdn.jtvnw.net/previews-ttv/live_user_kikoskia-{width}x{height}.jpg","title":"Let\'s Stream XCOM: Enemy Within!","type":"live","user_id":"8421033","user_name":"Kikoskia","viewer_count":0}]}

Earliest case I can find is Sept 18th.

@GStefanowich
Copy link

Went back and checked my logs, as I've been seeing this issue a lot lately with the streamers my bot tracks the live status for. I've all but eliminated polling for keeping track of livestream changes, but this has been troublesome.

I've seen issues as early as Sept 4th, where I received the stream down event a second before the stream up event.

[02:52:06|05SE'20] ChicagoJames [70506325] went offline at 2020-09-05 02:52:06.
[02:52:07|05SE'20] ChicagoJames [70506325] went live, playing Apex Legends, at 2020-09-05 02:49:25.

Below are the remaining times I've received a stream down within a minute or so of a stream up, it seems to have effected about 80% of the channels I have webhooks for.

My site logic currently rejects both duplicate Webhook IDs and Stream IDs, which the latter may need to be changed. There wasn't any issue until recently.

Dates & times are all listed in UTC.

Sept 16th

[00:03:07|16SE'20] QQTs [442869523] went live, playing Grounded, at 2020-09-16 00:02:25.
[00:03:54|16SE'20] QQTs [442869523] went offline at 2020-09-16 00:03:54.
[02:38:10|16SE'20] ChicagoJames [70506325] went live, playing Ori and the Will of the Wisps, at 2020-09-16 02:37:14.
[02:38:35|16SE'20] ChicagoJames [70506325] went offline at 2020-09-16 02:38:35.

Sept 17th

[23:02:18|17SE'20] BossFister [77507833] went live, playing The Witcher 3: Wild Hunt, at 2020-09-17 23:01:36.
[23:02:52|17SE'20] BossFister [77507833] went offline at 2020-09-17 23:02:52.

Sept 19th

[02:41:32|19SE'20] QQTs [442869523] went live, playing Among Us, at 2020-09-19 02:40:35.
[02:41:57|19SE'20] QQTs [442869523] went offline at 2020-09-19 02:41:57.

Sept 20th

[21:39:33|20SE'20] Goeringer [28700369] went live, playing Super Mario 3D All-Stars, at 2020-09-20 21:38:43.
[21:40:23|20SE'20] Goeringer [28700369] went offline at 2020-09-20 21:40:23.

Sept 23rd

[00:34:32|23SE'20] Colinahscopy_ [82185701] went live, playing Cities in Motion 2, at 2020-09-23 00:33:50.
[00:35:02|23SE'20] Colinahscopy_ [82185701] went offline at 2020-09-23 00:35:02.
[02:30:22|23SE'20] QQTs [442869523] went live, playing Fall Guys, at 2020-09-23 02:29:42.
[02:31:07|23SE'20] QQTs [442869523] went offline at 2020-09-23 02:31:07.

Sept 24th

[02:55:15|24SE'20] QQTs [442869523] went live, playing Grounded, at 2020-09-24 02:54:18.
[02:55:44|24SE'20] QQTs [442869523] went offline at 2020-09-24 02:55:44.

Sept 25th

[00:24:43|25SE'20] Colinahscopy_ [82185701] went live, playing Pokémon Sword/Shield, at 2020-09-25 00:23:53.
[00:25:08|25SE'20] Colinahscopy_ [82185701] went offline at 2020-09-25 00:25:08.

Sept 27th

[23:27:07|27SE'20] BossFister [77507833] went live, playing The Witcher 3: Wild Hunt, at 2020-09-27 23:26:12.
[23:27:33|27SE'20] BossFister [77507833] went offline at 2020-09-27 23:27:32.

Sept 29th

[02:26:58|29SE'20] Perkil8n [45208581] went live, playing Among Us, at 2020-09-29 02:26:16.
[02:27:32|29SE'20] Perkil8n [45208581] went offline at 2020-09-29 02:27:32.
[23:05:41|29SE'20] BossFister [77507833] went live, playing The Witcher 3: Wild Hunt, at 2020-09-29 23:04:59.
[23:06:13|29SE'20] BossFister [77507833] went offline at 2020-09-29 23:06:13.
[00:26:22|30SE'20] Colinahscopy_ [82185701] went offline at 2020-09-30 00:26:22.
[00:26:22|30SE'20] Colinahscopy_ [82185701] went live, playing Pokémon Sword/Shield, at 2020-09-30 00:25:33.

Oct 1st

[19:47:53|01OC'20] Doubl3down [58745117] went live, playing Just Chatting, at 2020-10-01 19:47:09.
[19:48:22|01OC'20] Doubl3down [58745117] went offline at 2020-10-01 19:48:22.
[23:32:17|01OC'20] BossFister [77507833] went live, playing The Witcher 3: Wild Hunt, at 2020-10-01 23:31:35.
[23:32:44|01OC'20] BossFister [77507833] went offline at 2020-10-01 23:32:44.
[23:56:52|01OC'20] Colinahscopy_ [82185701] went live, playing Pokémon Sword/Shield, at 2020-10-01 23:56:10.
[23:57:17|01OC'20] Colinahscopy_ [82185701] went offline at 2020-10-01 23:57:17.

Oct 2nd

[02:07:35|02OC'20] Twitch: Perkil8n [45208581] went live, playing Garfield Kart, at 2020-10-02 02:06:45.
[02:08:06|02OC'20] Twitch: Perkil8n [45208581] went offline at 2020-10-02 02:08:06.

@MJRLegends
Copy link

I can confirm it does seem to be in the duration of

I have observed issue normally occurs for streams that start 10pm BST thru 3am BST. which was said by BarryCarlyon above

@kav2k
Copy link

kav2k commented Oct 17, 2020

Still seeing this.

I added a check that immediately queries https://api.twitch.tv/helix/streams with the user_id for the stream status, and it dutifully reports offline (empty data) for the moment when the false "stream end" message is sent.

After that, a new update comes in, with the same stream ID and correct uptime.

@jbulava
Copy link
Member

jbulava commented Oct 27, 2020

This issue has been fixed. Thanks for your patience.

@jbulava jbulava closed this as completed Oct 27, 2020
@GStefanowich
Copy link

I experienced this again today

02:06:07 UTC

[
    {
        "game_id": "509538",
        "id": "40261982654",
        "language": "en",
        "started_at": "2020-10-29T02:05:25Z",
        "tag_ids": null,
        "thumbnail_url": "https://static-cdn.jtvnw.net/previews-ttv/live_user_qqts-{width}x{height}.jpg",
        "title": "EES PAHMPKEEN DAY 🎃🎃//!pov",
        "type": "live",
        "user_id": "442869523",
        "user_name": "QQTs",
        "viewer_count": 0
    }
]

~30 seconds later at 02:06:35 UTC

[]

And then at 02:07:20 UTC

[
    {
        "game_id": "509538",
        "id": "40261982654",
        "language": "en",
        "started_at": "2020-10-29T02:05:25Z",
        "tag_ids": null,
        "thumbnail_url": "https://static-cdn.jtvnw.net/previews-ttv/live_user_qqts-{width}x{height}.jpg",
        "title": "EES PAHMPKEEN DAY 🎃🎃//!pov",
        "type": "live",
        "user_id": "442869523",
        "user_name": "QQTs",
        "viewer_count": 1
    }
]

@sp0ck1
Copy link

sp0ck1 commented Oct 31, 2020

Just here to say that my stream today (just now, in fact) was definitely reported online on one check, offline on the next check within a few seconds, online again, and offline again once more, before finally stabilizing on the third online notification.

@xal3xhx
Copy link

xal3xhx commented Jan 12, 2021

can confirm this issue is NOT fixed

@alexweininger
Copy link

Experiencing this issue still also.

@alexweininger
Copy link

2022-02-03T22:58:47.255Z → Got a stream.online notification

~4 seconds later
2022-02-03T22:58:51.038Z → stream.offline notification

~2 minutes later
2022-02-03T23:00:40.066Z → stream.online notification

These are all for the same user, and the same stream id, and same stream start time. Each notification had a different message id. The duplicate stream.online notifications are easy to deal with, however the completely errant stream.offline notification is extremely unproductive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product: eventsub EventSub (Webhooks and WebSockets)
Projects
None yet
Development

No branches or pull requests

9 participants