New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Constant requests to a tracking server causes CPU overuse #1179

Closed
ameshkov opened this Issue Apr 20, 2017 · 9 comments

Comments

Projects
None yet
2 participants
@ameshkov
Member

ameshkov commented Apr 20, 2017

The easiest way to reproduce this issue is to block api.segment.io, install LastPass and monitor the filtering log. It constantly tries to contact that server. Also mixpanel.com is rather aggressive.

For now I've changed the rule to ||api.segment.io/v1/ so that it didn't work when HTTPS filtering cannot be used to emulate server's response.

Possible solutions

In any case, we should implement some sort of a rate limiter. For instance, if 10 similar requests are blocked within a short period of time, we should mark domain-endpoint pair as "limited".

The rate limiting strategy may differ depending on the filtering mode.

  1. VPN mode -- reset the connection immediately once it is received from the TUN.
  2. Proxy + auto mode -- that's tricky, I am not sure yet what's the proper way to handle it.

It's worth noticing, that mixpanel detects DNS-level blocking and does not try to connect to their server:
https://github.com/mixpanel/mixpanel-android/blob/bf7cdd554ae8e2ab8dd2954472e697f8efdc2a22/src/main/java/com/mixpanel/android/util/HttpService.java#L34

@ameshkov ameshkov added this to the 2.9 milestone Apr 20, 2017

@ameshkov ameshkov self-assigned this Apr 20, 2017

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov Apr 20, 2017

Member

Again, mixpanel detects response code 500 and does not make another try (so it will be blocked nicely when https filtering works):
https://github.com/mixpanel/mixpanel-android/blob/bf7cdd554ae8e2ab8dd2954472e697f8efdc2a22/src/main/java/com/mixpanel/android/util/HttpService.java#L147

Member

ameshkov commented Apr 20, 2017

Again, mixpanel detects response code 500 and does not make another try (so it will be blocked nicely when https filtering works):
https://github.com/mixpanel/mixpanel-android/blob/bf7cdd554ae8e2ab8dd2954472e697f8efdc2a22/src/main/java/com/mixpanel/android/util/HttpService.java#L147

ameshkov added a commit to AdguardTeam/AdguardFilters that referenced this issue Apr 20, 2017

ameshkov added a commit to AdguardTeam/AdguardFilters that referenced this issue Apr 20, 2017

@ameshkov ameshkov changed the title from Constant requests to a tacking servers to Constant requests to a tracking server Apr 20, 2017

@ameshkov ameshkov modified the milestones: 2.10, 2.9 Apr 20, 2017

@TPS

This comment has been minimized.

Show comment
Hide comment
@TPS

TPS Apr 22, 2017

Contributor

Are you saying these requests are generated by the app/browser itself or just whatever site that's being hosted? E.g., I regularly run Lastpass as my primary browser, but I don't see these requests in my filtering log. Did I miss something?

Anyway, I'd like to volunteer to test whatever you'd like me to, here. Will my lack of HTTPS filtering be a problem?

Contributor

TPS commented Apr 22, 2017

Are you saying these requests are generated by the app/browser itself or just whatever site that's being hosted? E.g., I regularly run Lastpass as my primary browser, but I don't see these requests in my filtering log. Did I miss something?

Anyway, I'd like to volunteer to test whatever you'd like me to, here. Will my lack of HTTPS filtering be a problem?

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov Apr 23, 2017

Member

Are you saying these requests are generated by the app/browser itself or just whatever site that's being hosted?

Talking about LastPass app only, not the browser. Yeah, if you block api.segment.io, it'll repeat request to it all the time.

Member

ameshkov commented Apr 23, 2017

Are you saying these requests are generated by the app/browser itself or just whatever site that's being hosted?

Talking about LastPass app only, not the browser. Yeah, if you block api.segment.io, it'll repeat request to it all the time.

@ameshkov ameshkov modified the milestones: 2.9 R2, 2.10 May 24, 2017

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov May 24, 2017

Member

Same issue with Kinopoisk app:
#1218

Member

ameshkov commented May 24, 2017

Same issue with Kinopoisk app:
#1218

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov May 24, 2017

Member

Here is what I did:

  1. If HTTPS connection is blocked at the handshake stage, we add the pair of app name and remote endpoint address to the cache of blocked endpoints.
  2. Next time when this app tries to contact that endpoint, we won't spend time on parsing ClientHello and block it right away.
  3. This "blocking" looks different from the side of the app -- instead of SSL error, it now receives a network error (connection reset).
  4. The whole procedure takes almost no time, so it practically solves CPU overuse issue.

Review ID: AFA-CR-10

Tested it with the Kinopoisk app (#1218), apparently, it solves the issue. Didn't test with LastPass yet, though.

Member

ameshkov commented May 24, 2017

Here is what I did:

  1. If HTTPS connection is blocked at the handshake stage, we add the pair of app name and remote endpoint address to the cache of blocked endpoints.
  2. Next time when this app tries to contact that endpoint, we won't spend time on parsing ClientHello and block it right away.
  3. This "blocking" looks different from the side of the app -- instead of SSL error, it now receives a network error (connection reset).
  4. The whole procedure takes almost no time, so it practically solves CPU overuse issue.

Review ID: AFA-CR-10

Tested it with the Kinopoisk app (#1218), apparently, it solves the issue. Didn't test with LastPass yet, though.

@ameshkov ameshkov closed this May 24, 2017

@ameshkov ameshkov changed the title from Constant requests to a tracking server to Constant requests to a tracking server causes CPU overuse May 24, 2017

@TPS

This comment has been minimized.

Show comment
Hide comment
@TPS

TPS May 25, 2017

Contributor

Oddly, for me, DNS blocking seems sufficient, as I rarely see the requests mentioned in the Filtering Log. However, if I'd like to test this via LastPass, how would I proceed?

Contributor

TPS commented May 25, 2017

Oddly, for me, DNS blocking seems sufficient, as I rarely see the requests mentioned in the Filtering Log. However, if I'd like to test this via LastPass, how would I proceed?

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov May 25, 2017

Member

Oddly, for me, DNS blocking seems sufficient, as I rarely see the requests mentioned in the Filtering Log. However, if I'd like to test this via LastPass, how would I proceed?

DNS blocking solves the issue completely, indeed. That solution we need for the case when DNS blocking is not used.

Member

ameshkov commented May 25, 2017

Oddly, for me, DNS blocking seems sufficient, as I rarely see the requests mentioned in the Filtering Log. However, if I'd like to test this via LastPass, how would I proceed?

DNS blocking solves the issue completely, indeed. That solution we need for the case when DNS blocking is not used.

@ameshkov ameshkov reopened this May 25, 2017

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov May 25, 2017

Member

Reopened issue because of a bug -- requests blocked by "cache" aren't recorded to the app stats.

Member

ameshkov commented May 25, 2017

Reopened issue because of a bug -- requests blocked by "cache" aren't recorded to the app stats.

@ameshkov

This comment has been minimized.

Show comment
Hide comment
@ameshkov

ameshkov May 25, 2017

Member

Done

Member

ameshkov commented May 25, 2017

Done

@ameshkov ameshkov closed this May 25, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment