Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while loading filters into Safari #550

Closed
zebrum opened this issue Dec 13, 2017 · 34 comments
Closed

Error while loading filters into Safari #550

zebrum opened this issue Dec 13, 2017 · 34 comments
Assignees
Labels
Milestone

Comments

@zebrum
Copy link
Member

@zebrum zebrum commented Dec 13, 2017

Since there are a lot of such errors, let's have a task. Probably iOS bug

Steps to reproduce

  1. Load new filters or UF into Safari

Expected behavior

Filters are loaded correctly

Actual behavior

Screenshot:

img_0001

Possible solution: swipe Safari to close it and restart AG.

Your environment

  • Browser: Safari
@vozersky

This comment has been minimized.

Copy link
Member

@vozersky vozersky commented Dec 13, 2017

have to get an iOs console log

@Goddess-Bastet

This comment has been minimized.

Copy link

@Goddess-Bastet Goddess-Bastet commented Dec 14, 2017

I have the same bug on my iPhone X, 5s & iPad Pro 9.7.

@ameshkov ameshkov added the P2: High label Dec 17, 2017
@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Dec 17, 2017

@zebrum do we know any stable way to reproduce this?

@BlazDT

This comment has been minimized.

Copy link

@BlazDT BlazDT commented Dec 20, 2017

Application version: 1.3.3.89
Device: iPhone
Platform: iOS
OS: 11.1.2

Helpdesk ID: 1108721

According to customer "Another issue is that switching on and off AdGuard several times shows an error message."

@zebrum

This comment has been minimized.

Copy link
Member Author

@zebrum zebrum commented Dec 20, 2017

@ameshkov yes, we do. You just need to load filter or several filters into Safari.

@IvanIin

This comment has been minimized.

Copy link
Contributor

@IvanIin IvanIin commented Dec 29, 2017

com.apple.SafariServices.Content process killed by the system due to exceeding the memory limit

Dec 29 16:31:04 Adg-iPhoneX kernel[0] <Notice>: EXC_RESOURCE -> com.apple.SafariServices.Content[8803] exceeded mem limit: ActiveSoft 150 MB (non-fatal)
Dec 29 16:31:04 Adg-iPhoneX kernel[0] <Notice>: 476630.614 memorystatus: killing_highwater_process pid 8803 [com.apple.SafariServices.Content] (highwater 3) - memorystatus_available_pages: 26612
Dec 29 16:31:04 Adg-iPhoneX Adguard Pro[8789] <Notice>: (AEService) Finishing notify Safari - reloadContentBlockerWithIdentifier. ```
@ameshkov ameshkov added this to the 2.0.0 milestone Jan 2, 2018
@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Jan 2, 2018

Basically, we need to find a way to further compress the filtering rules.

This can be solved on the JS rules converter side.

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 13, 2018

This issue also occurred in unreleased AG Pro v2.0.0 (124) while trying to save rules in the user filter or domains in the whitelist for Safari Content Blocker.

In the user filter for Safari Content Blocker:
Error: Filters cannot be loaded into Safari. Therefore, your recent changes were not applied.

Screenshot:

image


In the whitelist for Safari Content Blocker:
Error: Filters cannot be loaded into Safari. Therefore, your recent changes were not applied.

Screenshot:

image


B/R

@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Jan 20, 2018

@Mizzick this issue is what I was telling you about the filters compression. Let's try compressing Safari CB more.

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 28, 2018

Today, I just entered Blocking Lists, and then the warning suddenly jumped.
"Filters cannot be loaded into Safari. Try to clear your rules and the whitelist, or change the set of used filters."
Only report.
Do the large numbers of Subscriptions and user rules also cause the issue?
I have more than 59k rules in Blocking Lists.

Screenshot:

image


@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 28, 2018

This is not related to the Blocking List (as they are not loaded in Safari), but the Safari Content Filters. @lancelot-moon , your Safari Filters probably is "Updating" when you go to the Blocking List.

I have 59K rules in blocking list in my ipod touch and ipad pro.

I don't have this issue in iPad Pro (probably because it's a more powerful device) - I have english, spyware, annoyance, mobiles ads filter in safari content blocker.

In my iPod Touch, I encounter this in Safari Content blocker ~ so I only selected Simplified Domain Filters to mitigate the issue.

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 28, 2018

I knew Blocking Lists is not related to Safari Content Blocking.
Well, I fast entered Blocking List at that time.

Maybe it's due to my old iPhone 6 Plus.
Only have 1GB ram.
。・゚・(ノ∀`)・゚・。

Anyway, just report.
Please don't mind.

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 28, 2018

@lancelot-moon , you can deselect some filters in the content blocket to prevent this. What I did in my ipod touch(which has also 1gb of ram ) is to deselect everything except Simplified Domain Name filters and mobile ads filter.

As far as i can tell, the limit of rules is 50K, but if the rules is above 40K, it will start to complain with this. If the rules are less than 40K, eg. 35K, it will not complain.

Thanks the AdGuard DNS And the blocking list subcriptions i have activated in my ipod touch, i dont mind having less filters in safari content blocking... :-)

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 28, 2018

Thanks for your reply.

I applied about 45k rules in Safari Content Blocker.

When I just tried to disable a filter, that such warning jumped again.
The change for filters was not loaded.
Maybe it was updating at that time.
Later, I tried again.
The change for filters was saved.

What you will do is the compression of filters.
And then I think you can design that it'll jump a warning to tell users to try again later after the filters finished updating in the background if users are doing changes of filters for Safari at that time.

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 28, 2018

Lets’s wait for the AdGuard Team to find a solution for this as this. This seems to be an issue even with the stable version 1.3.3 due to too many filters selected.

If they cannot compresss it, probably just default to Simplified Domain Filters + Mobile Ads filter since this two filters are very powerful enough in my opinion. If you live in a country e.g Taiwan that has its regions specific filter, you can select your region specific filter

Just my opinion though.

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 28, 2018

No, English filter, Spyware filter, Social media filter, Annoyances filter and Safari filter are basic filters for Safari.

SDN filter only can block domains.
Do you think SDN filter can block those elements and pop-ups on pages?
Mobile Ads filter is specially for ads in apps, not for browsers.

Especially, when we enable English filter, Spyware filter, Social media filter, Annoyances filter, Safari filter and localism filter (ex: Easylist China. It's a pity there is no Taiwan localism filter now.), there are already large numbers of rules.

Is there quotas for additional other filters?
I doubt.

I don't understand why AdGuard team @zebrum decided to set Mobile Ads filter as default.

It's better to only set English filter, Spyware filter, Social media filter, Annoyances filter and Safari filter as default.

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 28, 2018

Simplified Domain Name filter is a combinatiom of whay you just said and is used by their AdGuard DNS....

Please check their website.

There is no quota on how many filtes you want to select. Only the rules.

The default AdGuard filters - English - Spyware - Social -Annoyance and safari are already big enough. But if a user add some more, then they will encpunter this issue.

I agree with AdGuard team to add Mobile Ads filter since this version of adGuard is for mobile devices. It makes sense that this is default. this is not only for ads in mobile apps...

It’s just a suggestion if they cannot compress. If this is an apple ios limitation or hardware limitation, then it is harder to fix.

This suggestion is for first time users. They can still select whatever they want but i think it’s better if we have a warning pop up informing users that selecting too many rules can potentially cause problems in updating due to ios limitation or something....

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 28, 2018

I have understood, but you don't understand what SDN filter is.
It's for DNS-level filtering.
It's a simplified filter, not a full combination.
You should open those filters and read the content.
You'll understand the difference.
Do you think SDN filter can block elements and pop-ups on pages?
Those annoyances?
In your opinion, you even think English filter is unnecessary because SDN filter is the combination of those filters.
You just ignore "Simplified".

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 28, 2018

Chill. It’s just a suggestion to mitigate this issue. As mentioned they can still select whatever they want. People do “install and forget”. It’s just defaulting (at first launch). They can still select whatever they want to choose.

Do you have anything that you can think of that can possibly mitigate this issue?

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 29, 2018

Sorry if offending you.
But you should already know SDN filter is not so powerful as what you think because it abandoned those rule syntax.
Especially, SDN fiter and Moble Ads filter can't solve anti-adblock...

SDN filter can't replace English filter, Spyware filter, Social media filter and Mobile Ads filter.

Simply, wait for the result of compression for filters.
If there is no good news for the compression, I don't suggest to set Mobile Ads filter as default.
No quota.

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 29, 2018

LOL. I’m alright not offended at all. Blame it to my Australian english. :-) we sounded weird sometimes, but not all the time!

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Jan 29, 2018

For Safari,
Moble Ads filter has about 1800 rules.
But EasyList China filter is a big guy, it has about 6200 rules.

Well, there are totally 47134 rules after enabling English filter, Spyware filter, Social media filter, Annoyances filter, Safari filter, Mobile Ads filter and EasyList China filter...

If disabling Mobile Ads filter from the above, the sums are 45334 rules.
EasyList China filter is too big.
That's why I said, no quota, unless you can compress them more.

@zznosar

This comment has been minimized.

Copy link

@zznosar zznosar commented Jan 29, 2018

@lancelot-moon you are almost at your limit mate. you will encounter issues for sure... especially when you reach 50K

@XCoderever

This comment has been minimized.

Copy link

@XCoderever XCoderever commented Jan 31, 2018

I wrote an email and they said they were fixing it... but no changes. How long has this been going on?

@Sailor-Chibimoon

This comment has been minimized.

Copy link

@Sailor-Chibimoon Sailor-Chibimoon commented Jan 31, 2018

I am here to report the same issue. I am using iPhone 7 Plus. As mentioned by the previous poster I deselected some filters. It stopped from appearing. I have a total count of 39K + rules

@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Feb 1, 2018

Here's an update on this.

It appears that the issue is caused not just by some per-process memory limit, but by the limit on the free RAM on a device. They've changed the content blocker compilation process in iOS 11.2 and due to that, the whole "compile & load" process is failing much more often.

There's more to it. On AdGuard's side a memory intensive "adguard rules --> content blocker rules" conversion process takes place.

So here's what we should do:

  1. Do not trigger loading filters to Safari until the memory used for conversion is freed up;
  2. Change the server-side optimization process so that the default set of filters was smaller;
@ExpertContributor

This comment has been minimized.

Copy link

@ExpertContributor ExpertContributor commented Feb 6, 2018

The following solves (works around) the issue for me:

  1. Settings ➡️ Safari ➡️ Content Blockers ➡️ AdGuard 📴
  2. Perform action with AdGuard (update/load or unload filters/block object)
  3. Return to settings as before and switch it on.
  4. Close Safari and then reopen it

This works regardless of how many filters I have, and the number of filters does not appear to alter the frequency of the errors occurring.

The presence of other ad blockers also appears to be irrelevant, as does the presence of an active VPN.

DNS logs shine no light on the issue either.

@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Feb 6, 2018

@IvanIin let's close this issue once the part 1 is resolved. It partly solves it, and the filters optimization will be addressed on the server side.

Also, I need you to prepare a simple iOS content blocker extension with 50k rules JSON that we can use to report this bug to Apple.

@kathampy

This comment has been minimized.

Copy link

@kathampy kathampy commented Feb 16, 2018

The workaround is to disable the AdGuard Pro Content Blocker in iOS Settings, update the filters in AdGuard Pro, and enable the Content Blocker again. Otherwise the error occurs 100% of the time.

@zebrum zebrum added current beta and removed current beta labels Feb 16, 2018
@Eugene-Savenko

This comment has been minimized.

Copy link
Member

@Eugene-Savenko Eugene-Savenko commented Feb 17, 2018

@kathampy AFAIK it doesn't load updates to Safari when you're doing so, you just avoid the error messages but the rules remain not updates. But if you unload both AdGuard and Safari from the memory, open AdGuard and check updates, it might work.

@kathampy

This comment has been minimized.

Copy link

@kathampy kathampy commented Feb 17, 2018

@evgeniy-ADG Yes I notice ad-blocking wasn't working.

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Mar 1, 2018

@ameshkov

Sir,
I use AG Pro beta v2.0.0(140).
Abnormal large numbers of rules occurred in my Safari filters.
I just enable English filter, Spyware filter, Social media filter, Annoyances filter, Safari filter and EasyList China filter.
But it shows that there are 85061 rules enabled and 50000 active rules.

Error: Filters cannot be loaded into Safari. Therefore, your recent changes were not applied.

Screenshots:

image


@ameshkov

This comment has been minimized.

Copy link
Member

@ameshkov ameshkov commented Mar 1, 2018

@lancelot-moon wait a bit, optimization algorithm bug on the server side, will be resolved soon

@lancelot-moon

This comment has been minimized.

Copy link

@lancelot-moon lancelot-moon commented Mar 1, 2018

Thanks!
I also think if something wrong happened in your server, especially you're updating Safari filters' names and descriptions recently.

Only enable English filter, but it has 57629 rules. XD

Screenshots:

image


Before I read your reply, I just sent the bug report.
Please ignore it.

B/R

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.