Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New youtube ads not blocked #7519

Closed
MrGamy opened this issue Jun 5, 2020 · 66 comments
Closed

New youtube ads not blocked #7519

MrGamy opened this issue Jun 5, 2020 · 66 comments

Comments

@MrGamy
Copy link

MrGamy commented Jun 5, 2020

URL(s) where the issue occurs

https://www.youtube.com/watch?v=4ehDRCE1Z38
https://www.youtube.com/watch?v=aaL7owZmbEA

Describe the issue

Certain videos on youtube seem to implement a new way of displaying ads - those ads are not being blocked by uBlock Origin at the Moment. The Channels around LTT seem to use this new method.

Versions

  • Browser/version: Firefox/77.0.1 (64-bit)
  • uBlock Origin version: 1.27.10

Settings

Used lists: Built in +EasyList, EasyPrivacy, Malware Domains & Malware Domain List, Peter Lowe's
No other additions to default settings.

Notes

@gwarser
Copy link
Contributor

gwarser commented Jun 5, 2020

Updating "uBlock filters" list does not help?

@MrGamy
Copy link
Author

MrGamy commented Jun 5, 2020

didn't help yet (tried updating first) and I only noticed it on those two videos so far. Like I said, no Ads on basically any other videos I've watched today

@Tomek-PL
Copy link

Tomek-PL commented Jun 6, 2020

I have the same issue on some videos.
I noticed that the video-ad has a yellow progress bar and cannot be fast forwarded. Once I manage to turn on my video, after scrolling to the beginning, it again activates the video ad with the yellow bar.
It's annoying

@gwarser
Copy link
Contributor

gwarser commented Jun 6, 2020

Related https://www.reddit.com/r/uBlockOrigin/comments/gx19kt/ads_on_youtube_as_of_today/

@MrGamy
Copy link
Author

MrGamy commented Jun 8, 2020

Could you tell me what you need me to do if I encounter another Ad? I've attached two Network Monitor HAR Files and hope this helps.
https://drive.google.com/drive/folders/1R6IyXLNbaKAAM2UN0ntQU1K9EtNT8T49?usp=sharing

@gwarser
Copy link
Contributor

gwarser commented Jun 8, 2020

If you updated "uBlock filters" in lates ~20 hours and still dont' have "Skip ad" button, add youtube.com#@##player-ads to My filters.

@Tomek-PL
Copy link

Tomek-PL commented Jun 8, 2020

I deleted uBlock from addons, reopen firefox and install uBlock again. No ads for over 1 day so I think it's a success

@liamengland1
Copy link
Contributor

That does not update the filters. To update the filters you must click "Purge all caches" and then "update now".

@Tomek-PL
Copy link

Tomek-PL commented Jun 8, 2020

OK, now I've also done this. Thank you for the advice

@gwarser
Copy link
Contributor

gwarser commented Jun 8, 2020

Try adding to My filters:

youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, undefined)

This should block video ads in new tabs.

@MrGamy
Copy link
Author

MrGamy commented Jun 8, 2020

Try adding to My filters:

youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, undefined)

This should block video ads in new tabs.

The above seems to have worked for me.


youtube.com#@##player-ads

doesn't seem to do anything. I'll further watch the issue and report back on any new ads popping up :) Thanks!

@liamengland1
Copy link
Contributor

If someone is still having issues they could try: dhowe/AdNauseam@e6682bb

@krystian3w
Copy link
Contributor

Very old:

committed on 28 Apr 2018

FiltersHeroes/PolishAnnoyanceFilters@beab581

@gwarser gwarser reopened this Jun 9, 2020
@gwarser
Copy link
Contributor

gwarser commented Jun 9, 2020

OK, youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, undefined) caused issues #7542 and was removed from filter list 9d7610b

Maybe some other variant of this filter will work.

Replace undefined by:
false,
true,
null - similar to undefined, may not work,
noopFunc,
trueFunc,
falseFunc or
''.

Like this:

youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, false)

@gorhill
Copy link
Member

gorhill commented Jun 12, 2020

Also, be sure you do not have another similar-purposed content blocker (ABP, AdBlock, AdGuard, etc.) along uBO; using more than one similar-purposed content blocker will always cause issues -- uBO is meant to perform optimally on its own.

@mlindner
Copy link

I'm also having the same issue on Chrome. It displays a grey overlay with a skip button in the lower right corner, but doesn't actually play the ad.

@gwarser
Copy link
Contributor

gwarser commented Jun 15, 2020

@mlindner youtube.com##.ytp-ad-module and youtube.com##.video-ads should hide these grey static ads, but when YT decide to show you video ad, you will not be able to skip it.

@sharpaper
Copy link

Came here to report the same issue (on firefox). Ads are not blocked on youtube. I can also reproduce the issue with the 2 links posted by OP.

@krystian3w
Copy link
Contributor

krystian3w commented Jun 19, 2020

IMO latest json-prune down, I see ads with /sw.js blocked (AdNauseam also fail).

https://www.youtube.com/watch?v=oBDTCSoi47E two 30 sec pre-roll ads.

@liamengland1
Copy link
Contributor

This video is also good for testing: https://www.youtube.com/watch?v=jjwilAja7Lc

@gwarser
Copy link
Contributor

gwarser commented Jun 19, 2020

Do you see ads after opening new tab or when navigating in YT (with red progress bar on top)?

@gwarser
Copy link
Contributor

gwarser commented Jun 19, 2020

  1. with new tabs: look into page source and try location initial player configuration object assignment. ytInitialPlayerResponse = {...adPlacements/playerAds...} or widnow[ytInitialPlayerResponse] = {...adPlacements/playerAds...} or somethingConfig then follow it in debugger and try to defuse by set/aopr/aopw/whatever.

  2. when navigation inside YT: set userResourcesLocation to https://gist.githubusercontent.com/gorhill/ef7af0cf1f3c9c6d1b57f355ceb4ce40/raw/61969a9fc2f66e5e5938cc5a30d12419d8288316/json-prune.js and add to My filters:

     youtube.com##+js(json-prune, [].playerResponse.adPlacements [].playerResponse.playerAds playerResponse.adPlacements playerResponse.playerAds adPlacements playerAds)
    
Details
// initial configuration handling code from my browser
// config object should be above this code
    (function() {
      var setFiller = function() {
        var fillerData = {
          browse: {
            filler: true,
            page: 'browse'
          },
          home: {
            filler: true,
            page: 'home',
            endpoint: {
              commandMetadata: {
                webCommandMetadata: {
                  url: "/",
                  webPageType: "WEB_PAGE_TYPE_BROWSE"
                }
              },
              urlEndpoint: {
                url: "/"
              }
            },
            response: {
              contents: {
                twoColumnBrowseResultsRenderer: {
                  tabs: [{
                    tabRenderer: {
                      selected: true,
                      content: {
                        richGridRenderer: {
                          contents: [],
                          continuations: [{
                            nextContinuationData: {continuation: ''}
                          }]
                        }
                      }
                    }
                  }]
                }
              }
            }
          },
          search: {
            filler: true,
            page: 'search',
            endpoint: {
              commandMetadata: {
                webCommandMetadata: {
                  url: "/results",
                  webPageType: "WEB_PAGE_TYPE_SEARCH"
                }
              },
              urlEndpoint: {
                url: "/results"
              }
            },
            response: {
              contents: {
                twoColumnSearchResultsRenderer: {
                  primaryContents: {
                    sectionListRenderer: {
                      contents: [],
                      subMenu: {
                        searchSubMenuRenderer: {}
                      }
                    }
                  },
                  secondaryContents: {
                    secondarySearchContainerRenderer: {
                      contents: []
                    }
                  }
                }
              }
            }
          },
          watch: {
            filler: true,
            page: 'watch',
            endpoint: {
              commandMetadata: {
                webCommandMetadata: {
                  url: "/watch",
                  webPageType: "WEB_PAGE_TYPE_WATCH"
                }
              },
              urlEndpoint: {
                url: "/watch"
              }
            },
            playerResponse: {
              playabilityStatus: {
                status: 'OK'
              }
            },
            response: {
              contents: {
                twoColumnWatchNextResults: {
                  results: {
                    results: {
                      contents: [{
                        videoPrimaryInfoRenderer: {},
                        videoSecondaryInfoRenderer: {}
                      }]
                    }
                  },
                  secondaryResults: {
                    secondaryResults: {}
                  }
                }
              }
            }
          }
        };

        window.ytcfg.set('FILLER_DATA', fillerData);
      };

        window.getPageData = function() {
          if (window.ytcsi) {window.ytcsi.tick("pr", null, '');}
          var endpoint = null;
            endpoint = {"commandMetadata":{"webCommandMetadata":{"url":"/watch?v=jgliCvYQ4hY","webPageType":"WEB_PAGE_TYPE_WATCH","rootVe":3832}},"urlEndpoint":{"url":"/watch?v=jgliCvYQ4hY"}};
          var data = {
              page: "watch",
              player: ytplayer.config,
              playerResponse: window.ytInitialPlayerResponse,
                url: "\/watch?v=jgliCvYQ4hY",
              response: window.ytInitialData,
              endpoint: endpoint
          };
          return {
            data: data,
            endpoint: endpoint
          };
        };

        if (window.loadDataHook) {
          var pageData = window.getPageData();
          window.loadDataHook(pageData.endpoint, pageData.data);
          window.loadDataHook = null;
        }
        setFiller();

    })();

@krystian3w
Copy link
Contributor

  • Only at the beginning,
  • video opened in a new card
  • Pre-roll ads is charged once in about 60 tests
  • no successful loading attempts during viewing

@gwarser
Copy link
Contributor

gwarser commented Jul 11, 2020

I am just trying to act according to feedback above:

  • Two users report that the filter appears to prevent ads from being shown -- so far; hoping for more follow-up feedback;

And one uses Fx 56. @BogudanPro are you on uBO legacy?

  • @okiehsch says AdGuard uses that filter, and report no ads and no "error occurred"

Tons of issues on AdGuard tracker (it's hard to track them all), but AFAIK uBO does not have problems with "error occured" (uBO can have it for few seconds max) and they still have.

  • Your json-prune filter is something which didn't exist originally, so not really full circle

Without /get_video_info there will be nothing to prune. This JSON is included in this request and sometimes coincidentally also valid video info and this causes issues ("error occurred").

Hopefully 1.28 with wildcards will fix more, but still it only fixes cases where ad info is downloaded by XHR - this is probably only the case when people navigate to video inside YT page.

For me the worst case is people being served ads which would not being served without our uBO-specific exception filter, @@||youtube.com/get_video_info.

Or complain about not seeing videos at all.

We don't need to block ||youtube.com/api/stats/watchtime?*=adunit&$important, you say it was already excepted by EasyPrivacy, so its not uBO-specific.

What I meant, some people don't have youtube.com/get_video_info request and still see ads because ad info can be smuggled in other requests including youtube.com/api/stats/watchtime.

@gorhill
Copy link
Member

gorhill commented Jul 11, 2020

Ok well you have been dealing with the Youtube issue more than I did, so it's best for me to not interfere at least until I invest as much time in this as you did. Feel free to remove the filter I added if you think it makes things worst, I don't have the same insights as you have. It seems what's really missing at this point is users suffering the ads and with minimal technical skills thoroughly collaborating with us to get to the bottom of this.

@gwarser
Copy link
Contributor

gwarser commented Jul 11, 2020

I will wait for feedback. There is always a chance I was mislead or lost in contradicting cases.

It seems what's really missing at this point is users suffering the ads and with minimal technical skills thoroughly collaborating with us to get to the bottom of this.

👍


I will probably add json-prune with wildcards enclosed inside env_firefox this evening.

@BogudanPro
Copy link

BogudanPro commented Jul 11, 2020

And one uses Fx 56. @BogudanPro are you on uBO legacy?

@gwarser, on home pc: yes, uBO 1.16.4.18 legacy. Not sure about PC where i saw ads...

@AntiSol
Copy link

AntiSol commented Jul 11, 2020

To chime in with some more info in case it matters: I'm using uBlock Origin v1.18.6 on waterfox classic 2020.06.

thoroughly collaborating with us to get to the bottom of this

I'd love to help solve this in any way that I can. What should I do if I do see an ad? Is there some info I can collect to help you track this down? Thanks! :)

@Yuki2718
Copy link
Contributor

for now I haven't taken the time to track down the reason they added the filter.

FYI: AdguardTeam/AdguardFilters@2d4f91e

@BogudanPro
Copy link

@gwarser
Copy link
Contributor

gwarser commented Jul 11, 2020

yes, uBO 1.16.4.18 legacy.

json-prune should be available, yet you are few versions behind.

@gwarser
Copy link
Contributor

gwarser commented Jul 11, 2020

I'm using uBlock Origin v1.18.6 on waterfox classic 2020.06.

json-prune is available starting with 1.23 and legacy 1.16.4.18.

On legacy builds you can try =adunit&$important,domain=youtube.com

@sharpaper
Copy link

Is there any fix for this issue? Youtube ads not blocked.

@mapx-
Copy link
Contributor

mapx- commented Jul 12, 2020

Is there any fix for this issue? Youtube ads not blocked.

did you update uBO filters ?

@ryanbr
Copy link
Contributor

ryanbr commented Jul 12, 2020

Is it fixed with latest update @sharpaper ?

@twisterniq
Copy link

Updating uBO filters doesn't help, still have the same issue. On many videos YouTube Ads aren't blocked.

1.27.10
Google Chrome 83.0.4103.116

@guft
Copy link

guft commented Jul 14, 2020

I'm getting ads on youtube (Firefox 78.0.2 64bit, uBlock Origin 1.28.0). I can reproduce the issue with the links provided. Just tried on Google Chrome ( 64bit, v83.0.4103.116, uBlock 1.27.10) and can also reproduce.

@sharpaper
Copy link

@ryanbr Under "Filter lists" I've clicked "Purge caches" and then "Update now". Still seeing ads.

@AntiSol
Copy link

AntiSol commented Jul 14, 2020

I'm now very confident that

||www.youtube.com/get_video_info*=adunit&$important

is working great. I've watched maybe 50 youtube videos since adding it. No ads, no errors.

@sharpaper
Copy link

To be fair, I do have noticed a lot fewer ads. The links reported in the ticket do not show any ad anymore. Occasionally (rarely) one shows up though, or I see a 15-seconds whitescreen with the "Skip ad >" button. But overall I think it's working again. Thanks for fixing this.

@BogudanPro
Copy link

On computer where i've seen ads was FF 56 with uBO 1.13 legacy. As i mentioned before, rule helped to get rid of ads.

@krystian3w
Copy link
Contributor

So use too old version, try override Firefox add-on signing requirement (verification):

https://web.archive.org/web/20180809075244/http://www.largrizzly.net/firefox.html

or

https://www.ghacks.net/2016/08/14/override-firefox-add-on-signing-requirement/#comment-4291219

If any people don't try publish compatible json-prune with uBO 1.13: #7519 (comment)

@twisterniq
Copy link

twisterniq commented Jul 14, 2020

Actually, at the moment, the difference is that there is a white screen with "Skip Ads" button. Should it be like it's now? I mean, before there were no white screens, no ads, we could directly watch the video.

image

@Toastbroti
Copy link
Contributor

youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, undefined)

fixes these blank ads without causing any issues for me.

@twisterniq
Copy link

twisterniq commented Jul 14, 2020

I can confirm there is no more white screen with the filter given by @Toastbroti. It works fine for me. I've tested with the two given links in the first post. Without the filter, there is always a white screen with the "Skip Ads" button. With the filter, the video starts directly.

@krystian3w
Copy link
Contributor

krystian3w commented Jul 14, 2020

And no break now homepage (only blank boxes on background)?

@Toastbroti
Copy link
Contributor

And no break now homepage (only blank boxes on background)?

It doesn't cause any issues

@lain566
Copy link
Contributor

lain566 commented Jul 14, 2020

youtube.com##+js(set, ytInitialPlayerResponse.adPlacements, undefined)

fixes these blank ads without causing any issues for me.

Here's why it was removed from the filter list #7519 (comment)
https://www.reddit.com/r/uBlockOrigin/comments/gzxzv4/youtube_on_firefox_broken/

Some people are still complaining
https://www.reddit.com/r/uBlockOrigin/comments/hq1rgl/getting_yt_ads_suddenly/fxygaes/

@mapx-
Copy link
Contributor

mapx- commented Jul 15, 2020

see YT issues
#7636

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests