Twitch bypass is circumvented, should now be removed to prevent repeated ads.

[ghost]

Prerequisites

• I verified that this is not a filter issue (MUST be reported at filter issue tracker)
• This is not a support issue or a question
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
• The issue is not present after wholly disabling uBlock Origin ("uBO") in the browser
• I checked the documentation to understand that the issue I report is not a normal behavior

I tried to reproduce the issue when...

• uBO is the only extension
• uBO with default lists/settings
• using a new, unmodified browser profile

Description

The scriptlet that blocked Twitch ads is now circumvented by Twitch. The issue now is that because UBlock is using the same device ID when requesting the video access token, the Twitch ad rate limit is not being imposed on that whitelisted device ID. Each refresh of a page after an ad is now shown, will repeat an ad because it does not apply a rate limit on Twitch's side.

Without UBlock enabled, once an ad is shown, a refresh of a page will not show an ad for a specified time limit.

The device ID bypass should now be removed to prevent excess ads.

gorhill/uBlock@cc0008d
gorhill/uBlock@0d3a193

A specific URL where the issue occurs

twitch.tv

Steps to Reproduce

1. Go to a Twitch channel with a pre-roll ad.
2. Watch it fully.
3. Refresh page and notice another ad shows.
4. Disable UBlock and repeat the above steps and after a refresh of the page, an ad does not show because the rate limit is correctly applied.

Expected behavior

The Twitch ad rate-limit should be applied, however it seems Twitch has now blocked the whitelisted device ID that was being used.

Actual behavior

Ad rate limit is not applied because of the set device ID being different to a normal user device ID. As the old whitelisted device ID is now no longer whitelisted, is should be removed.

uBlock Origin version

1.38.6

Browser name and version

Firefox (Latest Stable)

Operating System and version

Windows 10

[uBlock-user]

Still works fine at https://www.twitch.tv/esl_csgo and at https://www.twitch.tv/riotgames

[Nojuuu]

Still work fine for me too.

[ghost]

https://www.twitch.tv/burkeblack has pre-roll ads. You can force it by deleting the cookies, 'unique_id_durable' and 'unique_id' and refreshing the page. It could be that this is region based and they are slowly blocking it. I can confirm though that it was blocking ads about a week ago for me and is now no longer blocking ads. I've also confirmed that the whitelisted device ID is being used in the network connections.

This was also tested on a logged in Twitch account.

Anyone affected by it is currently getting more ads with UBlock, than without it.

[ghost]

Or cohhcarnage

[gorhill]

We should just comment out the filter for now.

[Nojuuu]

Wouldn't it be better to wait until more people confirm this issue?

[gorhill]

Yes, but there has been many complaints recently about Twitch ads.

[gorhill]

Personally I am not getting ads at the suggested URLs, but I might not wait long enough, or it might be that I am not logged out.

In any case, this is a filter issue, there is no need to remove the scriptlet if needed, just remove the filter making use of the scriptlet.

[uBlock-user]

Commented out for now...

[gorhill]

When I except the filter on my side, I now do get ads at https://www.twitch.tv/esl_csgo, so this makes it difficult to know what's the best move.

I now also get ads at https://www.twitch.tv/cohhcarnage, which I didn't get with the filter enabled.

[uBlock-user]

I now do get ads

Yes, I experience the same thing, so I added the filter to my filters too so atleast I won't have to experience ads, lets see how users react to this change.

[ghost]

It might be worth trying different regions. I'm EU. https://www.reddit.com/r/uBlockOrigin/comments/qbbxjq/comment/hh9mw95/?utm_source=share&utm_medium=web2x&context=3

I VPN'd to the US and the bypass works for me there.

[uBlock-user]

Tried them once that thread was posted, the problem there is some users still didn't bother to disable external twitch adblockers which could lead to issue you present here.

[gorhill]

It's just not external blockers which could interfere, people also need to be sure there is no leftover of old solutions in uBO, i.e. a lot of people were told to use userResourcesLocation, and now this must be undone to be sure there is no negative interference.

[ghost]

Yeah that is a possibility for some users, personally I can say that I have disabled them all. Only using UBlock, no extra scripts installed and ads are showing. Just giving you a heads up, because the rate-limit problem is the issue here, so I'm guessing more users will report repeated ads after refreshing in the future. No need to remove it yet if you are not sure, but I am one of those users affected by it, when I use UBlock only.

[uBlock-user]

because the rate-limit problem is the issue here, so I'm guessing more users will report repeated ads after refreshing in the future.

uBlockOrigin/uAssets@1b1838a

Filter has already been disabled. This change will reach all uBO users within 4 days.

[uBlock-user]

First(Second) case of reported ads after disabling the filter -- https://www.reddit.com/r/uBlockOrigin/comments/qlyg5z/twitch_ads_are_back/

[gwarser]

You mean - reported as working when filter was added back? Because https://www.reddit.com/r/uBlockOrigin/comments/pqmy4w/ublock_origin_138_announcement_thread/hiwt0o0/

[uBlock-user]

Because

Okay I didn't count that, so that makes it the above report the second report.

[ghost]

Thought I would try this again, but I'm still getting ads (While logged in). Here is the token response from the GQL request. Removed my IP and user ID, notice server_ads = true (SureStream).

"{\"adblock\":false,\"authorization\":{\"forbidden\":false,\"reason\":\"\"},\"blackout_enabled\":false,\"channel\":\"cohhcarnage\",\"channel_id\":26610234,\"chansub\":{\"restricted_bitrates\":[],\"view_until\":1924905785},\"ci_gb\":false,\"geoblock_reason\":\"\",\"device_id\":\"twitch-web-wall-mason\",\"expires\":1635964456,\"extended_history_allowed\":false,\"game\":\"\",\"hide_ads\":false,\"https_required\":true,\"mature\":false,\"partner\":false,\"platform\":\"web\",\"player_type\":\"site\",\"private\":{\"allowed_to_view\":true},\"privileged\":false,\"role\":\"\",\"server_ads\":true,\"show_ads\":true,\"subscriber\":false,\"turbo\":false,\"user_id\":,\"user_ip\":\"\",\"version\":2}"

I then enabled UBlock in private tabs, tried again in a private tab and got a response with server_ads turned off (Good). Realized it might be because I was logged out. So I logged in with my usual Twitch ID in the private tab, refreshed the page and got an ad and server_ads was turned back on. I now get an ad after every refresh.

They must be blocking the device ID bypass by user ID, maybe certain previous spade events or something add user ID's to a blacklist. I can literally log-in and log-out in the private tab and get ads off when I'm logged out and ads on when logged in. Didn't clear cookies or anything in the private tab while doing this and only had UBlock enabled in the private tab.

Here is a logged out response (No ads) and server_ads is false:

"{\"adblock\":false,\"authorization\":{\"forbidden\":false,\"reason\":\"\"},\"blackout_enabled\":false,\"channel\":\"cohhcarnage\",\"channel_id\":26610234,\"chansub\":{\"restricted_bitrates\":[],\"view_until\":1924905894},\"ci_gb\":false,\"geoblock_reason\":\"\",\"device_id\":\"twitch-web-wall-mason\",\"expires\":1635965675,\"extended_history_allowed\":false,\"game\":\"\",\"hide_ads\":false,\"https_required\":true,\"mature\":false,\"partner\":false,\"platform\":\"web\",\"player_type\":\"site\",\"private\":{\"allowed_to_view\":true},\"privileged\":false,\"role\":\"\",\"server_ads\":false,\"show_ads\":true,\"subscriber\":false,\"turbo\":false,\"user_id\":null,\"user_ip\":\"\",\"version\":2}"

I then created a new test Twitch account on the private tab, logged into it and still no ads. So it's somehow picking certain user ID's to fail on and can't be IP based. My normal Twitch ID must be blacklisted from using that 'twitch-web-wall-mason' device ID. I'm guessing that is also the reason for the other random cases too. Just no idea why and if it is timed based or they are detecting it somehow and slowly blacklisting ID's based on usage.

If I remove the OAuth header from the GQL request for the access token, while logged in, I get a response with ads turned off. If I leave the OAuth header in, I get a response with ads turned on, so has to be user ID based. For example, I can remove the OAuth header using the script and I get the site ad-free while logged in with my usual ID. Maybe this is the answer? To remove the OAuth header.

Removing the OAuth header, shouldn't have any negative affects and will fix the issues for the fringe cases. The only time it will have a negative affect is if they do block the twitch-web-wall-mason device ID, because then users will get ads on streams they are subbed to, but this could be addressed fairly quickly if it came to it.

Hope this helps

[uBlock-user]

Removing the OAuth header,

Does doing that results in server_ads = false ?

[ghost]

Yeah server_ads is false every time when I remove the OAuth token.

I added the following to the script:

if (typeof init.headers['Authorization'] === 'string') { init.headers['Authorization'] = ''; }

[uBlock-user]

Also if possible removing server_ads variable from the token param in GQL POST request might help in blocking forced ads(SureStream) for good.

[uBlock-user]

Yeah server_ads is false every time when I remove the OAuth token.

I added the following to the script:

if (typeof init.headers['Authorization'] === 'string') { init.headers['Authorization'] = ''; }

Do you mind testing this on https://www.twitch.tv/twitchmedia39 ?

[ghost]

I think the token/sig is made server-side and can't be edited as they won't match.

That test channel still shows server_ads = true, but I think it's forced to?

[uBlock-user]

I think the token/sig is made server-side

gql request is POST, so it should be possible ?

[uBlock-user]

channel":"travenura"

https://www.twitch.tv/travenura ? server_ads is true there but I don't get served ads, very strange.

[ghost]

Could be your location not having any fill, but generally server_ads = true, means SureStream(Embedded) ads is turned on. In the network tools console, if you look at the Twitch cookies, there is two device-id cookies, deleting them usually resets any rate-limits etc and sometimes forces an ad on the next refresh. Up to you if you want to test it that way though, especially if you are ad-free.

[ghost]

I think you need to consider the rate-limit issue when using the bypass device ID, for users that it is no longer working for, it's not applying a rate-limit to the amount of ads shown. So for a non subbed stream, you would get an ad after every refresh of the page, or every stream switch (if there is fill). Normally a rate-limit would be applied once an ad has fully shown. I expect Twitch is getting the brunt of the support emails and Reddit posts about the amount of ads, because people don't realize it's due to the bypass device ID. The Brave browser seems to of applied the bypass script, but without the Auth header, which explains the Turbo issue.

I can see that the filter is still enabled by default, do you have any plans to investigate it?

[uBlock-user]

@gorhill are you going to pull pixeltris/TwitchAdSolutions@aad8946 ?

[ImpalaPUA]

There are a lot of threads on /r/Twitch (I count 9 - including one mentioning Brave which I believe pulls in the same script) which would make me suspect it is the case, but you're right...

@ImpalaPUA can you confirm that you have userResourcesLocation as unset in your advanced settings of uBlock Origin (1.38.6) and that you don't have any other Twitch ad blockers enabled?

Yea, I have nothing in there and its set to unset

[pixeltris]

@ImpalaPUA Are you on 1.38.6 and don't have any other Twitch ad blocking extensions enabled?

And as follow-up; could you also tell me if you see ads in incognito (i.e. not logged into Twitch).

[uBlock-user]

Twitch Turbo issue fixed in gorhill/uBlock@ddd31f3

[pixeltris]

That isn't related to Turbo. That was a fix for sub-only VODs not working for subscribers #1789 (comment)

[uBlock-user]

Only issue/s reported on uBO subreddit was of Twitch Turbo which started after Authorisation was set to ''

subscribers #1789 (comment)

and would get fixed with users disabling the twitch filter, so was related to Authorisation commit.

[pixeltris]

Yes there's a good chance the Authorization header being set to null is impacting Turbo. Which is why I'm trying to establish if any impacted users are on 1.38.6 as that doesn't have the Authorization code.

If someone is on 1.38.6 and their Turbo is broken that is a very good indication that the bypass is no longer working (for some users). Which is also why I asked @ImpalaPUA for a follow-up for whether they see ads in incognito (no Turbo).

It should be noted that pretty much single time Twitch breaks a bypass it comes in waves (either as explicit A/B testing, or just how they deploy stuff to their servers around the world).

[ImpalaPUA]

Yes there's a good chance the Authorization header being set to null is impacting Turbo. Which is why I'm trying to establish if any impacted users are on 1.38.6 as that doesn't have the Authorization code.

If someone is on 1.38.6 and their Turbo is broken that is a very good indication that the bypass is no longer working (for some users). Which is also why I asked @ImpalaPUA for a follow-up for whether they see ads in incognito (no Turbo).

It should be noted that pretty much single time Twitch breaks a bypass it comes in waves (either as explicit A/B testing, or just how they deploy stuff to their servers around the world).

1.38.7b20 and umatrix

and ads on incognito

[pixeltris]

Great thanks. If you could do me one more favor and test 1.38.6 and let me know if you see ads there while logged in Turbo, that would be helpful.

[uBlock-user]

and umatrix

Disable uMatrix and all other extensions.

[ImpalaPUA]

Great thanks. If you could do me one more favor and test 1.38.6 and let me know if you see ads there while logged in Turbo, that would be helpful.

and umatrix

Disable uMatrix and all other extensions.

With all addons off except ublock 1.38.6 and twitch.tv#@#+js(twitch-videoad) removed from my filters

I still get ads with twitch turbo

[ImpalaPUA]

Actually when I went back to dev build and forgot to add twitch.tv#@#+js(twitch-videoad) back

I got ads through twitch turbo again

[pixeltris]

Thanks.

To sum this up...

Based on what @ImpalaPUA has observed it sounds like the there's a new check (introduced ~3 days ago) for the twitch-web-wall-mason header which always returns true to server sided ads. This check seems to be prior to any check for Turbo / subscription status (which can be confirmed by seeing ads with Turbo enabled on 1.38.6 which doesn't touch the Authorization header).

It should be noted that anyone who is impacted by this (Turbo or not) will see ads on every single stream on every page load. Normally you're given a ~10 minute cooldown after watching an ad (as mentioned by @saucettv). It's a worse experience having the ad blocking method enabled vs disabled.

Yes, it's weird there are literally 0 reports of the method being broken to uAssets or /r/uBlockOrigin. But this might be due to people using alternative methods for the past year or so, or somewhat expecting to see ads. And it might also be partly due to Twitch's shitty use of A/B testing.

*shrug*

[ImpalaPUA]

Wasn't it reported here https://www.reddit.com/r/Twitch/comments/qr1y1i/twitch_turbo_running_ads/

And isn't it repeatable by just adding and removing twitch.tv#@#+js(twitch-videoad)

[ghost]

Twitch use the users normal Device ID to record ad impressions and to apply the rate-limit. They are not doing this with the 'twitch-web-wall-mason' device ID. So as it's not actually bypassing the ads anymore for some users, having it in the scriptlet is causing more ads than would normally be shown to users that it's affecting. It is affecting users and eventually they will block it for all users. The scriptlet needs to be removed before users start disabling UBlock to actually see less ads on Twitch.

[diglarboard]

Based on what @ImpalaPUA has observed it sounds like the there's a new check (introduced ~3 days ago) for the twitch-web-wall-mason header which always returns true to server sided ads. This check seems to be prior to any check for Turbo / subscription status (which can be confirmed by seeing ads with Turbo enabled on 1.38.6 which doesn't touch the Authorization header).

I'm seeing different behavior. With a 1.38.6 I do not see ads on a subbed channels. With 1.38.7b27 I do get ads on a subbed channel. So I think it is the Authorization change that is breaking it and should be removed. It likely is causing Twitch to be unable to authentication that my user has a sub or has turbo

[pixeltris]

Do you see ads in incognito mode / logged out (both 1.38.6 and 1.38.7b27)?

[diglarboard]

Yes, so in incognito mode I get preroll ads for both versions.

I just double checked and it's definitely the case that the auth change is causing ads on my subbed channel. On 1.38.7b27 if I log in and watch a subbed channel, the https://usher.ttvnw.net/api/channel/hls/formal.m3u8 network call shows my user_id set to null and subscriber set to false. If I use 1.38.6, then my user_id is set correctly and subscriber is true.

This is also the same with the response from the GQL PlaybackAccessToken call. So the change in 1.38.7b27 is making it treat my user as logged out.

[pixeltris]

One thing I didn't ask @ImpalaPUA is if they use Brave. It pulls in the script also and might be pulling in the latest version (which is why they might be observing 1.38.6 differently).

The method is patched for you. And yes it sounds like Authorization is a regression in this case.

I recommend removing Authorization from the script to reduce the impact.

[ryanbr]

At Brave we just pulled in the latest uBO twitch update, which will roll out in the next 24/48hrs.

[pixeltris]

Thanks for the info @ryanbr unfortunately the latest update to the script wont change anything. The method is slowly being patched by Twitch. @gorhill I recommend the Authorization change is removed at minimum which will reduce the sub/Turbo issues (but non sub/Turbo users will still see way more ads than normal (ad on every page load) for those who observe the method as patched).

[ImpalaPUA]

Small update, when I upgraded to the latest development build and removed twitch.tv#@#+js(twitch-videoad) from my filters

I no longer see pre rolls with twitch turbo, so it seems to have fixed the issue

[ghost]

Twitch use the users normal Device ID to record ad impressions and to apply the rate-limit. They are not doing this with the 'twitch-web-wall-mason' device ID. So as it's not actually bypassing the ads anymore for some users, having it in the scriptlet is causing more ads than would normally be shown to users that it's affecting. It is affecting users and eventually they will block it for all users. The scriptlet needs to be removed before users start disabling UBlock to actually see less ads on Twitch.

https://www.reddit.com/r/uBlockOrigin/comments/qwr2oj/suddenly_on_twitch_im_getting_much_more_adds_why/ @gorhill

https://www.reddit.com/r/Twitch/comments/qxu6ty/is_twitch_trying_to_kill_viewership_with_preroll/