How to use Apply allowed hosts option when adding a hosts source?

[zgfg]

I want to add a list of allowed hosts, for example:
api.micloud.xiaomi.net
sgpac.account.xiaomi.com
connect.rom.miui.com
update.miui.com
zgwlc.zagreb.hr
magisk-modules-repo.github.io

Instead of manually adding one by one to the Allowed list, or adding them to the adaway-backup.json and to restore from, I have created a file myAllowedList.txt and uploaded that file to my Github project:

In AdAway, Hosts sources I take + and add its URL:
https://github.com/zgfg/Etc/raw/main/myAllowedList.txt

I select Apply allowed hosts because I want to allow those hosts, not to block them:

I apply the change, in Hosts sources it looks ok but given hosts were not added as Allowed:

---

AdAway 5.3.0, root (Magisk Canary 21407) om Xiaomi Mi 9T, Xiaomi.eu 21.1.28 (A11)

But it doesn't matter (phone, Magisk version), the same with no-root mode (Huawei Mate 9, AdAway 5.3.0)

This is actually about the similar problem reported at that time for the local Allowed.List.txt file (Content type) #2185.
Here it's now for the URL type

[PerfectSlayer]

Hi @zgfg

Thanks for the detailed bug report and the test data!
I apply them on the current dev version and, as you could see, it seems to work fine 😉

[zgfg]

Thank you a lot @PerfectSlayer for this new implementation👍👍👍

Do Allowed files also need timestamp (to avoid update problems as with Codeberg domain, see #2570)?

In that case, my example:
https://github.com/zgfg/Etc/blob/main/myAllowedList.txt

can be also tested from the alternative URL:
https://raw.githubusercontent.com/zgfg/etc/master/myAllowedList.txt
(See: #2570 (comment))

Although, it could be valuable if your timestamp fix for githubusercontent.com address could be also applied to the github.com raw form of user addresses

[zgfg]

I also hope @PerfectSlayer that this new implementation will also work for local (on the phone) Allowed files - hence you could also close then #2185

[PerfectSlayer]

Yes, allowed files also need timestamp to define if they need update.
If no timestamp is provide, they will always be downloaded (according the cache control server rules 😉 ).

Using the url: https://raw.githubusercontent.com/zgfg/etc/master/myAllowedList.txt should work well.
As discussed in #2570, I will focus github detection on the final destination of all the github redirection (so raw.githbusercontent.com) as there is too many of them and I am not sure how they evolves through time.

It will also work with the file storage. In fact, it is the same code that check and fetch blocked list and allowed list.
The only code that changes is the part that parses the file content. So same update check, download and read behavior!

[zgfg]

Thanks @PerfectSlayer.
I expected that once you fix Allowed list for remote source files (URL type), that it should also work for the local files (content type) - I just wanted that you don't forget to Close #2185🤓

Regarding to github.com vs githubusercontent.com it must be easier than I originally thought - as analyzied in today's new #2605, Github dynamically redirects itself to githubusercontent.com whenever fetching the file from github.com.

Hence you could IMO just extend your logic for timestamps for files from githubusercontent.com (where it works with no issues) also to files that will be dynamically redirected to githubusercontent.com on each fetch (currently they have issues like 'Codeberg' files as explained in #2605)

. The same logic should work then both for Blocked and Allowed lists (no matter if their URLs are provided in github.com or githubusercontent.com form)

[PerfectSlayer]

Hence you could IMO just extend your logic for timestamps for files from githubusercontent.com (where it works with no issues) also to files that will be dynamically redirected to githubusercontent.com on each fetch (currently they have issues like 'Codeberg' files as explained in #2605)

That is not issue. AdAway downloads hosts source as resource from http servers.
It should not try to understand human readable website and parse it to find a way to get the content.

The github.com domain is for end users and is not a static resource hosting (I am not even sure it is allowed from the GitHub's Terms)
The right solution is to use GitHub pages. In few clicks, you can convert your repository to a static hosting.
It will work as expected: Last-Modified http header is set, GitHub can set a CDN, etc...

raw.githbusercontent.com domain is supported but I am not sure it is the right way to use it.
I developed the workaround as most of the sources are using it but I am not in favor exploiting GitHub API and web site for this usage.

Do you understand this point of view?

[zgfg]

@PerfectSlayer - full screenrecording

Just regularly using Github, regularly taking the link from Raw (long press and Get link address), pasting to AdAway, successfully applying but getting Failed to download (Internet connection problem):

https://mega.nz/file/YxMhlI7D#pKDj0mMDx0aagsAIjvGqNNPBR4y-VRC8AxFAEQCcSSY

[PerfectSlayer]

The internet connection problem when checking for update is fixed for the next release.

But it won't fix the fact the URL you use does not serve a Last-Modified header so the application won't be able check if an update is available (so it will download and apply it every time).

[zgfg]

Let's keep this ticket closed - because the allowed list functionality is fixed.
I've opened another ticket for github.com addresses, but that's definitely less important