Implement Filtering Log v1

[ameshkov]

View 1 - Filtering Log is stopped

Placeholder
Start button
Warning about performance

Filtering Log Notification

When filtering is started we should show an ongoing notification about this.

View 2 - Filtering Log is started

List of log records.

Show for every log record:
URL
App name
Source domain
Third-Party icon

Log record color indicates if request was blocked or whitelisted.
Red for blocked, green for whitelisted, white bg - for processed.

View 3 - Log Record Details

Tapping on log record opens log record details view.

URL
Type
Source domain
Filter Rule and Filter Subscription name for blocked and whitelisted requests

Buttons:
Open in browser
Block for common requests
Unblock for blocked requests (Unblock creates whitelist rule)
Nothing for requests whitelisted by some filter subsription

View 4 - Filter Rule Creation

Example of such dialog in browser add-on:
http://screencast.com/t/kbttytH1SWj

New rules should be added to user filter.

[Nick-White]

+1 This is the feature I'm waiting the most because there are many ads that go through all the filter subscriptions...

[ameshkov]

For now you can report these to us:
https://github.com/AdguardTeam/ExperimentalFilter/issues

Or you can read AG logs and find out what is not blocked:
https://github.com/AdguardTeam/ExperimentalFilter/wiki/Android

[ameshkov]

@Revertron here is a list of comments for the first filtering log implementation

Filtering log icon

Use this icon for the navigation menu item:
filtering_log_icons.zip

Please let me know what sizes do you need.

Action bar

We need an option to clear the filtering log
Put it in the action bar.
Use the same icon we have in the user filter/whitelist.

"Enabled" should the same size/weigth as action bar title has.

Empty filtering log

Use the same icons and layout as in the case of user filter/whitelist.
Text: Filtering log is empty

Log item

Time and URL color should be of our regular gray color.

[Revertron]

For small icons (as 24dp in UI) we use these sizes: 96, 72, 48, 24, 36 pixels.

[ameshkov]

@Revertron got it, what color do we need for the navigation menu?

[Revertron]

Other icons have $757575 color.

[ameshkov]

Color indication

Change the line background color to indicate request status.

Regular requests: no indication.
Blocked requests: #fff4f2 (pressed state: #ffb6a9)
Whitelisted requests: #defae5 (pressed state: #98ffb2)

[ameshkov]

Log item details fields should depend on the event type. However, there are some common requirements.

Common requirements

1. Tap on a field should automatically copy it's value to the clipboard. We should also show a toast about it.
2. The Status field possible values: Processed, Blocked, Threat, Whitelisted, Redirect. The latter makes sense for Web request type only.
3. Event type field possible values are Web request, DNS request, HTTPS tunnel
4. Application field name depends on the app type and can be App, Browser, System app
5. Filtering rule field is visible in two cases only: if the status is either Blocked or Whitelisted.
6. Possible buttons in the item details footer: BLOCK, UNBLOCK (raised buttons), OPEN (flat button). OPEN button opens request URL in the default browser. OPEN is available for regular web requests with GET method only.
7. Remote Address field format is 127.0.0.1:443 for IPv4 and [2001:4860:4860::8888]:443 for IPv6. Use IpAddressUtils.toString() method to get the proper format for it. Do not show this field if value is empty.

Regular web request fields

• Event type
• Application
• Domain
• Remote Address
• Status (You should use Redirect status whenever redirectUrl field is not empty.)
• Filtering rule
• Request URL
• Source URL (do not show if empty or null)
• Redirect URL (show it if status is Redirect)
• Content type (no need to translate, just use the enum value)
• Start time (format: 18:45.123) (we don't really need date here)
• Elapsed time (format: 123 ms)
• Size (format: ↓123KB ↑5KB)

Please note, that Event type in the case of regular web request also contains HTTP method value.
It should look like:

Value: Web request (GET)
Field name: Event type

DNS request fields

• Event type
• Domain
• Status
• Filtering rule
• Start time
• Elapsed time
• Size

Tunnel request fields

• Event type
• Application
• Domain
• Remote Address
• Status
• Filtering rule
• Start time
• Elapsed time
• Size

[ameshkov]

Filtering log menu item icon:
filtering_log_icons.zip

Fields icons:
filtering_log_fields_icons.zip

[ameshkov]

@Revertron just caught a crash:

22:52:00.037 [main] ERROR c.a.a.c.CustomExceptionHandler - Thread: Thread[main,5,main]; Uncaught exception:

java.lang.UnsupportedOperationException: null
    at java.util.AbstractList.add(AbstractList.java:148) ~[na:0.0]
    at android.widget.ArrayAdapter.insert(ArrayAdapter.java:255) ~[na:0.0]
    at com.adguard.android.ui.FilteringLogActivity$3.run(FilteringLogActivity.java:119) ~[na:0.0]
    at android.os.Handler.handleCallback(Handler.java:751) ~[na:0.0]
    at android.os.Handler.dispatchMessage(Handler.java:95) ~[na:0.0]
    at android.os.Looper.loop(Looper.java:154) ~[na:0.0]
    at android.app.ActivityThread.main(ActivityThread.java:6119) ~[na:0.0]
    at java.lang.reflect.Method.invoke(Native Method) ~[na:0.0]
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:886) ~[na:0.0]
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:776) ~[na:0.0]

[ameshkov]

Rules constructor issues:

1. Does not work properly for DNS requests
2. Patterns should start with ||
3. Add one more option: [x] Apply to the APPNAME requests only. In case if this box is checked, add $app modifier to the rule text.

[ameshkov]

I see that startappexchange.com SDK sometimes sends a GET request with URL length more than 20000. Actually, every time I start ru.sports app, I see this request. We'd better handle long URLs specifically.

[ameshkov]

Add $important modifier support to the rules constructor dialog.

1. Add a checkbox: Give a higher priority to the rule
2. By default, this checkbox should be disabled.
3. However, if you try to unblock a URL blocked with an $important rule, or block a whitelisted URL, make it checked.

[ameshkov]

Handle third-party flag in a smart way

1. If web request is third-party, "Apply to third party requests only" option should be checked.
2. Otherwise, let it be unchecked.

[Revertron]

Done.

[Alex-302]

Issues:

1. Incorrect content type for Web-Socker connections
   wss://ws.hghit.com/ws
Content type = Document
   Add @@||hdrezka.me^$content and go to hdrezka.me;
2. After clearing, filtering log again contains the records;
3. Request sorting is strange - requests from the page are located before request to the page [Moved to Weird records order in the filtering log #1074]

4. Elements, which was removed from HTML are not displayed(like in Adguard for Windows) [Moved to Show removed elements in the Filtering Log #1075]

[ameshkov]

Just fixed point 1.

Regarding point 3, this is by design (at least for now). We should fix it eventually, but we'd better move it to a separate issue.

[Alex-302]

Regarding point 3

done
#1074

[Alex-302]

Added
4. Elements, which was removed from HTML are not displayed(like in Adguard for Windows)

[Revertron]

Fixed point 2.