GDPR dialogs / banners hiding on webpages

[peace2000]

GDPR dialogs / banners hiding on webpages conversation.

Edit, hiding outdated content.

Some time ago, Sanoma adopted new GDPR dialogs for their sites.

Unfortunately, cookies have to be accepted in order to see some embedded content, such as tweets.

I think it's really not needed to whitelist that dialog for each and every page... So I was planning to create filters for Easylist Cookie that remove the dialog only if the article doesn't have embedded stuff (tweets).

Currently, I have this set of filters:
@@||gdpr-tcfv2.sp-prod.net/wrapperMessagingWithoutDetection.js$script,domain=is.fi
is.fi##+js(rc, sp-message-open, html, stay)
is.fi##body:not(:has(.embed-cookie-consent-icon)) > div[id^="sp_message_container"]:remove()

Edit, this is needed also for Firefox so that the cookies can actually be accepted:
@@||sp-prod.net^$xhr,domain=is.fi

The first rule whitelists the dialog so that it's possible to see it. The second unlocks scrolling, and the third one removes the dialog if there's no embedded content in the article.

Would you be willing to test @Nojuuu how well this set works? Do you see any visual flickering or performance issues?

This article has a tweet: https://www.is.fi/politiikka/art-2000008974427.html (the dialog should be shown)
This has no embedded stuff https://www.is.fi/hyvaolo/art-2000008970002.html (the dialog should be hid)
And of course the main page should work as well: https://www.is.fi/

[Nojuuu]

Looks like cookie dialog can be blocked on keskustelu.kauppalehti.fi:
https://keskustelu.kauppalehti.fi/threads/taenaeaen-soi.120751/page-511

Do you know if adding domain=~keskustelu.kauppalehti.fi to this:

finnish-easylist-addition/Finland_adb.txt

Line 1369 in 3b9ac8d

@@||almamedia.fi/almacmp/$script

works?

[peace2000]

Let's begin with whitelisting easylist/easylist#12990 that PR took time to make...

I'm still unsure about this https://koneporssi.com/ I could not confirm as I didn't find embedded stuff.

[peace2000]

My PR was accepted.

What do you think about these rules?

arvopaperi.fi,iltalehti.fi,mediuutiset.fi,mikrobitti.fi,talouselama.fi,tekniikkatalous.fi,tivi.fi,www.kauppalehti.fi,www.uusisuomi.fi###alma-cmpv2-container:style(display: none)
arvopaperi.fi,mediuutiset.fi,mikrobitti.fi,talouselama.fi,tekniikkatalous.fi,tivi.fi,www.kauppalehti.fi,www.uusisuomi.fi##body:has(p:has-text(Sisältöä ei voitu ladata)) > .alma-cmpv2-container:style(display: block !important)
iltalehti.fi##body:has(.consents-checker) > .alma-cmpv2-container:style(display: block !important)
iltalehti.fi##+js(set, jwAlmaCMPLoaded, true)

[peace2000]

Added PR easylist/easylist#13004

[Nojuuu]

Sorry, didn't have time to test earlier but did now and those seem to work fine.

I could not confirm as I didn't find embedded stuff.

https://koneporssi.com/kuljetuskalusto/hakosen-uudet-jareammat-sahkokuorma-autot-kayttoon-paakaupunkiseudulla/

[peace2000]

Hi, ok. Thanks, great!

[peace2000]

https://bigbrother.fi/

It's Sanoma site so embedded social media content needs cookies accepted, though I don't see that this site uses that kind of posts at all.

@Nojuuu Do you feel safe to block GDPR scripts?

[Nojuuu]

Videos are not loading for me on mobile when blocking the script.

btw supla.fi needs checking too

E: easylist/easylist#14206

[Nojuuu]

https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#subjectremove-attrarg-subjectremove-classarg

+js(ra, ...) and +js(rc, ...) filters need to be updated at some point.

[peace2000]

Maybe 1 month after stable release is good time to do that.

[Nojuuu]

@peace2000
https://www.vr.fi/ does scrolling work for you on mobile?

[peace2000]

I'm trying to check this. I suspect that mobile browser needs something else than this:

vr.fi##+js(addEventListener-defuser, wheel, preventDefault)

As you can't use mouse wheel on a phone.

[peace2000]

vr.fi##+js(addEventListener-defuser, touchmove, preventDefault)

Does this work for you @Nojuuu

[Nojuuu]

That helps, maybe it would be good to have both? I think it's possible to put something like /touchmove|wheel/ there.

[peace2000]

Made a pr: easylist/easylist#14613 I think it's better to have separate filters with a comment.

[Nojuuu]

That's fine too :D

[peace2000]

Videos on ruutu.fi seem to be broken now if GDPR script is blocked. Earlier it worked....

@Nojuuu

easylist/easylist#14717

[peace2000]

It seems that something has happened and the dialog isn't being dealth anymore properly with trusted filters. It's hid only because of the old cosmetic rules that are still there.

Edit, fixed the filter.

[peace2000]

Though some pages don't seem to need the acceptance of the cookies.

Even if I block the GDPR script and disable trusted filters, the embedded content works on these pages:
https://www.etlehti.fi/artikkeli/vapaa-aika/katso-suomen-vanhin-varielokuva-talta-naytti-helsingin-kauppatori-vuonna-1936
https://www.hyvaterveys.fi/artikkeli/vanhemmuus/kansa-sekosi-aaro-vauvan-kuvasta-teilla-varsin-hyvantuulisen-nakoinen
https://www.vauva.fi/comment/42579908#comment-42579908

Can you reproduce this @Nojuuu ?

[peace2000]

ConsentUUID expires, so have use clicker. uBlockOrigin/uAssets@4e7ebc2

Had to be a bit creative. First we click settings open, then select Sanoman sisällönjakelukumppanit (eng: Sanoma's content delivery partners), and deselect everything else and then save. Also needed cosmetic filters to hide the dialog visually so that it's being hid quickly (it's still being clicked in the background), and also cosmetics for scrolling to avoid annoying bouncing effect the dialog causes to scrolling.

The clicker also activates on cdn.privacy-mgmt.com even if it could be used to host the GDPR dialog for other pages as well, but it's not a problem because the css selector requires .sanoma-logo-container to be there, which will not be be found from anywhere else except from Sanoma related GDPR requests.

Writing this comment here as a explanation what those rules are doing.

[Nojuuu]

@peace2000 Cookie modal back on ruutu?

[peace2000]

Hmm yes it seems.

Updated: uBlockOrigin/uAssets@e06cc8b

I'm not sure why Ruutu wasn't there at all.

[peace2000]

As uBO now has a scriptlet that allows to set cookies, many of these dialogs can be handled in a better way - no need to hack with "not:has()" filters that much... :)

@Nojuuu would you be willing to test these? (All of them are needed):

mtvuutiset.fi##+js(trusted-set-cookie, OptanonConsent, groups=1%3A1%2C2%3A0%2C3%3A0%2C4%3A0%2C5%3A1%2CBG57%3A0%2CBG58%3A0%2CBG59%3A0, 1year)

mtvuutiset.fi##+js(trusted-set-cookie, eupubconsent-v2, CPtgAUgPtgAUgAcABBENDJCgAAAAAH_AAApAAAATBAJMNS4gC7MscCbaMIoUQIwrCQqgUAFFAMLRAYQOrgp2VwE-sIkAKAUATgRAhwBRkwCAAASAJCIAJAjwQCAACAQAAgAVCIQAMbAILACwEAgAFAdCxRigCECQgyIiIhTAgKkSCgnsqEEoP9DTCEOssAKDR_xUICNZAxWBEJCwchwRICXiyQPcUb5ACMAKAUSoVqKT00BChmbLAAAA.YAAAD_gAAAAA, 1year)

mtvuutiset.fi##+js(trusted-set-cookie, OptanonAlertBoxClosed, $currentDate$, 1year)

Those filters aim to set mandatory and social media cookies accepted.

Sample page that has tweets:

https://www.mtvuutiset.fi/artikkeli/hallitusohjelma-kirvoittaa-epauskoisia-parahduksia-ja-kiitosta-lue-marinin-ja-monen-muun-kommentit-linjauksiin/8722768

trusted-set-cookie only works on own filters and on uAssets but that's ok as uAsset's Annoyance section handles GDPR dialogs now: https://github.com/uBlockOrigin/uAssets/blob/master/filters/annoyances-cookies.txt

[peace2000]

uBlockOrigin/uAssets@d453ef3

Bit lenghty rules but couldn't figure out anything else that works.

[Nojuuu]

I didn't notice any breakages without eupubconsent-v2 filter so that one might not be necessary?

[peace2000]

Interesting... Testing this again and yeah it really seems that eupubconsent-v2 isn't really needed.

Adjusted filters in uBO.

[Nojuuu]

OptanonAlertBoxClosed isn't needed either, but if that's left out then the dialog will stay. Though at the background tweets seem to unlock. Can you reproduce this as well?

Sorry, I was busy, but yeah i can reproduce it.

[peace2000]

OK :)

[peace2000]

Ok, now for Iltalehti.

These two do the trick:

iltalehti.fi##+js(trusted-set-cookie, TcString, CPtgasAPtgasABUAMAFIDICgAP_AAAAAAApAAAAMEgLgALAAqABkADwAIAAZAA0AB8AEQAJgATwA5gB-AEIANEAbIBFgC0gGKAM-AmQBeYDBACQkBAABYAFQAMgAeABAADIAGgARAAmABPADmAH4AQgA2QDFALzDQAgBsgFpEQAQBsioAYATAC0gLzGQAgAmALzHQFAAFgAVAAyACAAGQANAAfABEACYAE8AOYAfgBogDZAIsAWkAxQB1AEyALzIQBgAFgAZACYAWkAxQB1CUAkABYAGQAiABMAGyAWkAxQB1AF5lICAACwAKgAZABAADIAGgARAAmABPADmAH4AaIA2QCLAGKAXmAAA.YAAAAAAAAIAA, 1year)

iltalehti.fi##+js(trusted-set-cookie, gravitoData, {"NonTCFVendors":[{"id":1\,"name":"Facebook"\,"consent":true}\,{"id":9\,"name":"Twitter"\,"consent":true}]}, 1year)

They allow Twitter and Facebook embeds. I couldn't find anything else embedded stuff that would be in use.

Sample links:
https://www.iltalehti.fi/kotimaa/a/47433dea-ec7c-468b-9562-7a4f07ff3063

https://www.iltalehti.fi/politiikka/a/08e2a41e-a031-4303-96e1-eaa9982b86d1

[Nojuuu]

I see it but only on mobile for some reason

E; Nevermind, also seeing it if I open it in private window

[peace2000]

Updated the filter uBlockOrigin/uAssets@c63d36d

[peace2000]

Yes and it didn't work, but I updated uBO and now it seems to work. I've had problems before with some filters randomly not working with >the development build, but I'm not sure what's causing it.

I have also seen randomly cookie filters not working. I have a hunch it could be caused by the fact that usually when we use the button on the page to accept cookies, the cookie is set for the base domain. But cookie scriplets always set cookie also for the www.-part. Though this is not always an issue, the fact is that the cookie is in a slightly different place where the page expects the cookie to be.

Though Alma media sites seem to favor setting the cookie with www.-part natively but in most cases, pages don't use www.-part.

uBlockOrigin/uBlock-issues#2893

For your interest @Nojuuu

[peace2000]

Updated Alma media:
uBlockOrigin/uAssets@086cbf2

[peace2000]

Updated again, more stuff needs to be accepted so that login via various 3P-services work. Also added ampparit.com to the list due to this.

Screenshot of the current settings (what is accepted):


[Nojuuu]

Needs "Performance Cookies" for videos:
https://www.gamespot.com/videos/

[peace2000]

Concerning Gamespot, maybe performance cookies is safe enough for the page. The page anyway has multiple "always active" cookies, and I can't figure out anything else that important as videos that should be tested.

Though I can't figure out a working set of filters for that page.

[Nojuuu]

I'll see if I can come up with something later.

[peace2000]

Added rps: uBlockOrigin/uAssets@d4ae432

[Nojuuu]

Doesn't similar filters work for gamespot too?

gamespot.com##+js(trusted-set-cookie, OptanonConsent, groups=C0001%3A1%2CC0002%3A1%2CC0003%3A0%2CC0004%3A0%2CC0005%3A0, 1year)
gamespot.com##+js(trusted-set-cookie, OptanonAlertBoxClosed, $currentDate$, 1year)

[peace2000]

That works.

I'll add it: uBlockOrigin/uAssets@298c027

[Nojuuu]

historianet.fi,tieku.fi##+js(trusted-set-cookie, CookieInformationConsent, %7B%22consents_approved%22%3A%5B%22cookie_cat_necessary%22%2C%22cookie_cat_statistic%22%2C%22cookie_cat_marketing%22%5D%2C%22consents_denied%22%3A%5B%22cookie_cat_functional%22%2C%22cookie_cat_unclassified%22%5D%7D, 1year)

Embedded content.

youtube:
https://tieku.fi/terveys/aivot-ahnehtivat-rasvaa
https://historianet.fi/kulttuuri/arkeologia/3d-kuvat-paljastavat-uskomattoman-loydon-titanicin-hylysta
https://historianet.fi/kulttuuri/arkeologia/muinaisihmiset-hautasivat-vainajiaan-100-000-vuotta-ennen-homo-sapiensia

twitter:
https://tieku.fi/teknologia/twitterin-perustaja-loi-uuden-sosiaalisen-verkoston
https://tieku.fi/ilmasto/ilmastonmuutos/pahin-kuivuus-500-vuoteen-nakyy-avaruuteen-asti

Can you figure out anything better? Seems like you need to accept a lot for whatever reason...

[stephenhawk8054]

Do these filters work?

historianet.fi,tieku.fi###cookie-information-template-wrapper
historianet.fi,tieku.fi##+js(rc, noScroll, .noScroll, stay)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_functional, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_statistic, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_marketing, true)

[Nojuuu]

Seems to work for me

[peace2000]

@stephenhawk8054 That works but there's a small "bounce" effect that happens on the right side of the page every time the page is loaded by F5. (On slower pc's the effect is probably more severe).

Also, is it practical / performance-wise to have a scriptlet rc constantly active for the page when the GDPR stuff can be handled by setting a cookie?

In this case, I'd favor setting cookie instead of using other solutions.

I appreciate your efforts though. :)

[stephenhawk8054]

Actually I personally would consider setting cookie and storage item as the last methods if no solutions being reasonable, because we are setting up websites' values that will be stored on users' browser for an amount of time and across browsing other websites (the least would be session, or until user turning off browser). Other filters will just affect to the code while running on that website. I think minimizing trusted scriptlets to the lowest amount as possible is better to reduce attack surface.

My priority while crafting filters would be: normal filters -> trusted scriptlets involving code (rpnt, trusted-set-constant...) -> cookie/storage trusted scriptlets but just setting to default (or session) as expired time -> cookie trusted scriptlets set to 1year.

Of course for sites with not good intentions (anti-adb, popup, anti right click...), I also won't consider that priority too much (and actually these kinds of sites rarely needs trusted set-cookie/set-storage anyway). Or even with the sites that normal filters causing glitches like you said, I will also go for cookie/storage approach.

---

Anyway, although rc with stay or :remove-class with :watch-attr won't cause much performance degrade (and also we haven't optimized it to some specific selectors yet), do these work better?

||policy.app.cookieinformation.com/*/fi.js$script,domain=historianet.fi|tieku.fi
historianet.fi,tieku.fi##.topscroll-banner
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_functional, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_statistic, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_marketing, true)

[peace2000]

ELC already blocks scripts.

historianet.fi,tieku.fi##.topscroll-banner
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_functional, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_statistic, true)
historianet.fi,tieku.fi##+js(set, cicc.cookie_cat_marketing, true)

These work! I'll replace the previous ones.

[peace2000]

@Nojuuu

Can you test this? https://tweakers.net/nieuws/197852/portal-achtige-puzzelgame-the-entropy-centre-komt-in-2022-uit.html

tweakers.net##+js(rpnt, script, "youtube":{"approved":false, "youtube":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "omny":{"approved":false, "omny":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "pcnltelecom":{"approved":false, "pcnltelecom":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "googlemaps":{"approved":false, "googlemaps":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "streamable":{"approved":false, "streamable":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "soundcloud":{"approved":false, "soundcloud":{"approved":true, sedCount, 1)
tweakers.net##+js(rpnt, script, "knightlab":{"approved":false, "knightlab":{"approved":true, sedCount, 1)

What I'm trying to do is to set embedded content cookies accepted by replacing parts of inline script. (Setting cookies doesn't work for that site as it's not possible to set cookies for different domain (Youtube)). The link has an embedded Youtube-video.

I'm wondering whether I have performance issues on my browser installation or is it just Firefox generally.

When I load that page bypassing cache (CTRL + F5), the Youtube video will always unlock. But If I reload that page without bypassing the cache (F5) the video won't occasionally unlock on my Firefox browser but on Chrome it unlocks always.

How does it work for you?

[Nojuuu]

Works the same way for me

[stephenhawk8054]

Do these work?

tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.youtube.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.omny.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.pcnltelecom.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.googlemaps.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.streamable.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.soundcloud.approved, true)
tweakers.net##+js(set, tweakersConfig.userConfiguredConsent.knightlab.approved, true)

[peace2000]

Yes, they work fine!

[peace2000]

easylist/easylist#16469

[Nojuuu]

@peace2000
Gigantti needs statistic cookies for product reviews:

elgiganten.dk,elgiganten.se,elkjop.no,gigantti.fi##+js(trusted-set-cookie, CookieInformationConsent, %7B%22website_uuid%22%3A%22%22%2C%22timestamp%22%3A%22%22%2C%22consent_url%22%3A%22%22%2C%22consent_website%22%3A%22%22%2C%22consent_domain%22%3A%22%22%2C%22consents_approved%22%3A%5B%22cookie_cat_necessary%22%2C%22cookie_cat_statistic%22%5D%2C%22consents_denied%22%3A%5B%5D%2C%22user_agent%22%3A%22%22%7D)

With Dan Pollock's and Peter Lowe's list you also get adb complaint if you click "Kommentoi"

This seems to get rid of that, not sure if the commenting still work though

||iesnare.com^$script,redirect-rule=noopjs,domain=elgiganten.dk|elgiganten.se|elkjop.no|gigantti.fi

[peace2000]

Sorry I was busy.

Yeah, if not accepted, product review section won't appear.

Gigantti uses the same GDPR script as historianet.fi & tieku.fi.

This seems to work:

elgiganten.dk,elgiganten.se,elkjop.no,gigantti.fi##+js(set, cicc.cookie_cat_statistic, true)

Sample links:

https://www.elgiganten.dk/product/mobil-tablet-smartwatch/mobiltelefon/iphone-14-5g-smartphone-128gb-midnight/522692

https://www.gigantti.fi/product/kodinkoneet/pyykinpesu-ja-kuivaus/pyykinpesukoneet/samsung-ww95ta047ae-pyykinpesukone-ww5000t/185561

Redirecting that script isn't enough - it has to be whitelisted. I tried to send a product review - it failed if that script wasn't allowed.

On my end, only the Danish and Finnish sites have that comment-button. But I still think that it's better to include Swedish and Norwegian sites as well as they are sister sites and they could add commenting some day.

[peace2000]

easylist/easylist#16954

[peace2000]

Oh, and also power.fi suffers the same issue if that script is blocked. I some time ago whitelisted that in the Finnish list but I think should be moved to uAssets because uAssets fixes issues caused by it's default lists.

Sample link for power.fi: https://www.power.fi/keittion-pienkoneet/kahvi-ja-tee/kahvinkeittimet/moccamaster-manual-s-kahvinkeitin-kivenharmaa/p-1953935/#reviews

[peace2000]

uBlockOrigin/uAssets@6fd03b4

[Nojuuu]

This seems to work:

elgiganten.dk,elgiganten.se,elkjop.no,gigantti.fi##+js(set, cicc.cookie_cat_statistic, true)

I thought I had tried that and it didn't work, but now it seems to work, maybe I tried it without cicc.

[Nojuuu]

@peace2000

https://www.kiertokanki.com/

Needs zen_eprivacy and __kla_off cookies, otherwise the product modals won't work.

[peace2000]

Sorry I didn't understand what is "product modal", can you clarify?
@Nojuuu

[Nojuuu]

Nvm, rechecked the website now and I think we can just do this:

kiertokanki.com##.modal:has(#eprivacy)
kiertokanki.com##.modal-backdrop
kiertokanki.com##body:style(overflow: auto !important;)

"product modal", can you clarify?

For example, if you click on an item in the "Uutuustuotteet" section, it should open a modal for it, it has the same id etc. as the cookie modal.

[peace2000]

easylist/easylist#17281

[Nojuuu]

@peace2000

autodude.dk,autodude.fi,autodude.no,autodude.se,valostore.fi,valostore.no,valostore.se##+js(trusted-set-local-storage-item, cookieConsent, necessary)

Could you check if this works and maybe add it to uBO list?

[peace2000]

Added: uBlockOrigin/uAssets@cae66fe