Build system: ETP lists dependency is on Amazon S3

[say-yawn]

Why/User Benefit/User Problem

Currently Firefox iOS is generating their own tracking protection lists by grabbing the raw source from shavar-prod-lists and following a similar configuration as shavar-list-creation. However, the logic to generate the tracking protection lists can fall behind (see #5559 and #5968). I suggest that Firefox iOS gets the lists from Shavar services maintained S3 files. With this transition Firefox iOS can deprecate the script to generate their own tracking protection lists and only have to maintain the script to convert the tracking protection JSON files to iOS consumable files all the while always being guaranteed to get the latest tracking protection lists from Shavar.

Acceptance Criteria (how do I know when I’m done?)

• For every new release, download tracking protection files in JSON format from Shavar maintained S3 files
• Use these files to update the tracking protection lists
• Remove the script generating tracking protection lists
• ETP still works

Not included

┆Issue is synchronized with this Jira Task

[say-yawn]

@garvankeeley I still need information on what lists you all were generating so I can make sure those files are available in S3

[garvankeeley]

@garvankeeley I still need information on what lists you all were generating so I can make sure those files are available in S3

These files are specified here:
https://github.com/mozilla-mobile/firefox-ios/blob/master/content-blocker-lib-ios/ContentBlockerGen/Sources/ContentBlockerGen/main.swift#L13

[garvankeeley]

• For every new release, download tracking protection files in JSON format from Shavar maintained S3 files

Dependency management questions: With what URL? And, where is the URL updated?

• Remove the script generating tracking protection lists

iOS requires a specific format that we maintain so full removal is not possible; IIUC the inputs to this processing step are changing; please describe these inputs and from that we can determine if the outputs are still supported.

However, the logic to generate the tracking protection lists can fall behind (see #5559 and #5968).

Those bugs are input changes to the iOS processing script, I would imagine any input changes involving files being added or removed will result in these types of changes. Can you explain?

[data-sync-user]

➤ Laurie Marceau commented:

Closing this since this won’t be fetched from an S3 bucket.