Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove use of google-mapping.json #5968

Closed
1 task
say-yawn opened this issue Dec 31, 2019 · 16 comments
Closed
1 task

Remove use of google-mapping.json #5968

say-yawn opened this issue Dec 31, 2019 · 16 comments
Assignees
Labels
1 Story size Blocked Cannot move forward with the issue QA Verified Issue verified by QA

Comments

@say-yawn
Copy link
Contributor

say-yawn commented Dec 31, 2019

With the introduction of disconnect-mapping.json, google-mapping.json was deprecated and no longer being used to generate tracking protection lists.

Acceptance Criteria

  • google-mapping.json is removed
@garvankeeley
Copy link
Contributor

There is no disconnect-mapping.json in shavar that I see https://github.com/mozilla-services/shavar-prod-lists, is not released yet?

@say-yawn
Copy link
Contributor Author

@garvankeeley
Copy link
Contributor

Why is this file in a different repo? Isn't it to be used in combination with the disconnect list in shavar-prod-lists?

@englehardt
Copy link

Why is this file in a different repo? Isn't it to be used in combination with the disconnect list in shavar-prod-lists?

I suspect there is some legacy reason as to why it's in that repo. It used to be a python file of constats imported by the list creation script.

It could definitely be moved to the shavar-prod-lists repo. Is there a reason you need it moved?

@say-yawn
Copy link
Contributor Author

mozilla-services/shavar-prod-lists#110 PR re-organized all the domains in Disconnect Category to the respective categories and as such disconnect_mapping.json is no longer needed. @garvankeeley can we close this issue now?

@garvankeeley
Copy link
Contributor

I don't think we should close it, there has been no change to the parser, it still uses google-mapping.json: https://github.com/mozilla-mobile/firefox-ios/blob/master/content-blocker-lib-ios/ContentBlockerGen/Sources/ContentBlockerGenLib/ContentBlockerGenLib.swift#L24

@say-yawn
Copy link
Contributor Author

Do you have a timeline on when you expect to remove it?

@say-yawn
Copy link
Contributor Author

Updated the requirements to just remove the google_mapping.json (Disconnect category no longer exists).

@garvankeeley
Copy link
Contributor

I'll try get this bug triaged in our next meeting

@say-yawn
Copy link
Contributor Author

Hi @garvankeeley, just wondering if you have any updates in regards to this? Thank you!

@garvankeeley garvankeeley moved this from Needs Triage to Sprint Backlog in Firefox iOS Development Mar 31, 2020
@garvankeeley garvankeeley added the 2 Story size label Mar 31, 2020
@garvankeeley
Copy link
Contributor

I added it to the sprint backlog, it should get picked up in the next two weeks.

@athomasmoz athomasmoz added 1 Story size and removed 2 Story size labels Apr 14, 2020
@garvankeeley garvankeeley self-assigned this Apr 15, 2020
garvankeeley added a commit to garvankeeley/firefox-ios that referenced this issue Apr 15, 2020
@garvankeeley
Copy link
Contributor

Updated the requirements to just remove the google_mapping.json (Disconnect category no longer exists).

Hi @skim1102, can you clarify the following. I see the changes as being

  1. remove use of google_mapping.json. (One concern I have about this is that the file is not removed from the repo https://github.com/mozilla-services/shavar-prod-lists/blob/master/google_mapping.json)
  2. there is no more 'Disconnect' category in the disconnect-blacklist.json, therefore I must remove this from our parser or it will crash as it expects that category to exist.
  3. possibly I need to add handling for disconnect-mapping.json? I don't know anything about that file, and we aren't handling it currently.
  4. Update our dependency to the current tip of master (I assume). But there is a missing file on master normalized-lists/base-fingerprinting-track.json it was committed to a branch here: mozilla-services/shavar-prod-lists@777688f

@say-yawn
Copy link
Contributor Author

1. remove use of `google_mapping.json`. (One concern I have about this is that the file is not removed from the repo https://github.com/mozilla-services/shavar-prod-lists/blob/master/google_mapping.json)

We will remove this once you can tell me that you are no longer using the file. It will be documented on mozilla-services/shavar-list-creation#118

2. there is no more 'Disconnect' category in the `disconnect-blacklist.json`, therefore I must remove this from our parser or it will crash as it expects that category to exist.

Yes, this will be fine since all the domains that was in the Disconnect category was mapped into the existing categories.

3. possibly I need to add handling for `disconnect-mapping.json`? I don't know anything about that file, and we aren't handling it currently.

No, disconnect-mapping.json has been removed from being used in the config (mozilla-services/shavar-list-creation-config#60) and in the list creation script (mozilla-services/shavar-list-creation#121) and as such you will not need to use this file.

4. Update our dependency to the current tip of master (I assume). But there is a missing file on master `normalized-lists/base-fingerprinting-track.json ` it was committed to a branch here: [mozilla-services/shavar-prod-lists@777688f](https://github.com/mozilla-services/shavar-prod-lists/commit/777688fdf5420c2e00c6cde2689a9d5a21796d4d)

Dependency should be pointing to the latest desktop release version of normalized-lists/base-fingerprinting-track.json current latest release is 75, with 76 becoming the new release on May 5th. Not sure when the new release date is but please ping me if the normalized-lists/base-fingerprinting-track.json is not available when this is being worked on.

Hope this helps!

@garvankeeley
Copy link
Contributor

We will remove this once you can tell me that you are no longer using the file. It will be documented on mozilla-services/shavar-list-creation#118

ok, no need to wait, our dependencies are pinned to a SHA

Dependency should be pointing to the latest desktop release version of normalized-lists/base-fingerprinting-track.json current latest release is 75, with 76 becoming the new release on May 5th. Not sure when the new release date is but please ping me if the normalized-lists/base-fingerprinting-track.json is not available when this is being worked on.

@skim1102 normalized-lists is not on the 75 or 76 branch that I can find, so we can't use those.
Filed mozilla-services/shavar-prod-lists#174

@jevans-mozilla jevans-mozilla added the Blocked Cannot move forward with the issue label Apr 28, 2020
@athomasmoz athomasmoz moved this from Sprint Backlog to Up Next in Firefox iOS Development Apr 29, 2020
@garvankeeley garvankeeley removed the Blocked Cannot move forward with the issue label Apr 30, 2020
@garvankeeley
Copy link
Contributor

garvankeeley commented Apr 30, 2020

Unblocking as my questions are answered, google-mapping.json is in fact redundant on the branch iOS is on, and I can wait on the linked shavar PR until a newer shavar branch is updated and switch iOS to point to that, and land it together with the google-mapping change.

@garvankeeley garvankeeley moved this from Up Next to In Development in Firefox iOS Development Apr 30, 2020
@jevans-mozilla jevans-mozilla added the Blocked Cannot move forward with the issue label May 12, 2020
@garvankeeley garvankeeley moved this from In Development to Sprint Backlog in Firefox iOS Development May 12, 2020
garvankeeley added a commit to garvankeeley/firefox-ios that referenced this issue May 21, 2020
Firefox iOS Development automation moved this from Sprint Backlog to Needs QA May 21, 2020
garvankeeley added a commit that referenced this issue May 21, 2020
…te shavar (#6664)

* Fix #5968: remove use of googlemapping.json and Disconnect cat., update shavar

* update tests
@SimonBasca
Copy link
Contributor

Verified fix on 26.0 (18064).

@SimonBasca SimonBasca added the QA Verified Issue verified by QA label May 26, 2020
@project-bot project-bot bot moved this from Needs QA to Done in Firefox iOS Development May 26, 2020
garvankeeley added a commit that referenced this issue Jul 29, 2020
* Fix #4700: Reader view should switch to a darker theme when using Night mode (#5959)

* Fix #4700: Reader view should switch to a darker theme when using Night Mode

* Fix #4700 - Keep sepia as reader mode theme when switching back from dark to light in case sepia was used in light mode before

* Fix #4700 - Refactor the code from previous commits

* Fix #4700 - Fix the issue with reader mode changing its color after changing font size of font type

* Fix #4976: Make Reader Mode respect accessibility settings (#5983)

* Fix #4976: Make Reader Mode respect accessibility settings

* Fix #4976 - Remove unused variable

* Fix #4976 - Fix selected state for font buttons

* Update after merge from master

* Fix #4700: Code review feedback

* Fix #4700: Remove unused init method

Co-authored-by: Mykola Aleshchenko <maleschenko@homeawaycorp.com>

* No Bug - Issue when launching sync tests due to pipenv error (#6237)

* Fix #6012 - Enhanced Tracking Protection Coversheet (#6228)

* Fix #6012 - Enhanced Tracking Protection Coversheet
- Updated test cases to not show ETP screen randomly
- Added ETP CoverSheet tests

* Fix #6245 - Update uglify to terser js plugin for the webpack step  (#6246)

* Fix #6245 - Update uglify to terser js plugin for the webpack step

* Update autogenerated files

* [nobug, build] update app services to v54 (xcode 11.3.1) (#6247)

* [nobug, build] update app services to v54 (xcode 11.3.1)

* update README

* Release checklist: add check for security advisories (#6179)

* Release checklist: add check for security advisories

* Update release_checklist_template.md

* [nobug, build] Cartfile.resolved protobuf needs to be 1.8.0 (#6248)

App services requires this lib to be v1.8.0

* [nobug, build] resetBookmarksMetadata renamed to resetBookmarkSyncMetadata

* Issue #5901 - fixed placeholders clearing on theem change in customSearchVC (#6236)

user input stays on theme change

address extra lines feedback

* Fix#6134 - Weird Camera View on QR Reader (#6174)

* Fix #6216 - AA Leanplum show onboarding intro test setup (#6257)

* Fix#5873 - Share sheet not in dark mode (#6024)

* Fix#5873 - Share sheet not in dark mode

* #5873: Create theme for shareExtension

* Revert "relaoding tab after scripts were injected into the selected tab’s webview (#6117)" (#6252)

This reverts commit 4c3cf89.

* Add appservices_local_dev.sh script (#6197)

* Fix #6107 - Updated the dark theme icon and locked portrait mode for ETP Coversheet (#6258)

* Fix #6287 - Updated Leanplum SDK to v2.6.4 (#6288)

* Fix #6133: Add tab telemetry (#6281)

* Fix #6133: Add tab telemetry

* Add telemetry docs in tree

* issue #6133: Rename tab count ping

* Fix #5838  for Youtube to load videos in full screen (#6296)

Fix #6272 for whatsapp to load the proper desktop website by using custom user agent

* Bug-1622800 - part 1: Enable taskcluster/taskgraph (#6277)

* Removed needless `Array` initialization (#6301)

* Create ---feature.md (#6129)

This will be used as a template when adding a new feature request.
Note that it will auto-add the `Feature 🎁` label.

* ---bug-report.md (#6128)

This will be used as a template when adding a new bug.
Note that it will auto-add the `Bug 🐞` label.

* Bug-1622800 - part 2: Remove "project" prefix

* Bug-1622800 - part 3: Remove unused `install-sdk.sh`

* Bug-1622800 - part 4: Use `mobile.v2.firefox-ios` instead of `mobile.firefox-ios.v2` (#6303)

* Fix6152 - Home 'Custom URL' placeholder text is barely visible (#6169)

* Fix6152 - Home 'Custom URL' placeholder text is barely visible

* Update DarkTheme.swift

removed white spaces

* QR Denied Torch Remains Open and Background (#6284)

* add fragment support for search engine

* refactor and re indent file

* add test case of fragment usage for search term

* change source of onepassword-app-extension and update

* Update Cartfile

* Update Cartfile.resolved

* Fix UIView.isHidden must be used from main thread only

* Revert "Fix UIView.isHidden must be used from main thread only"

This reverts commit e5ce847.

* FirefoxAccountStateChange post notification wrapped to notify on the main thread.

* fix RustLogins warnings

* migrade deprecated .lproj

* add recommended base localizations

* make background black when camera is denied

* do not allow to open torch when denied camera

Co-authored-by: Garvan Keeley <garvankeeley+github@gmail.com>

* Fix#6146 - Checkmark is above label on tracing protection page. (#6147)

* Fix6151 - The Change Passcode and Require Passcode options are not grayed out when Passcode is OFF (#6250)

* Fix#6292 - Login&Password- SelectAllButton is cropped even if not active (#6293)

* Fix #5034: Autocomplete titles overlap urls with Larger Text (#6270)

* Fix #5264: Bookmarks overlap when using large and bold text (#6265)

* Fix #6233 - Keyboard not dimissed when opening the library panels fro… (#6278)

* Fix #6233 - Keyboard not dimissed when opening the library panels from the home screen

* Tap gesture is removed

* Update app-services version to 0.56.0 (#6317)

* Remove audio UIBackgroundModes (#6316)

App submission was rejected because we are not using this key.

* Update release_checklist_template.md

* Fix #5910 : Long email for fxa is shown with smaller text in Browser Tab Menu  (#6324)

* Fix for issue #5910 for longer emails text resize

* resolved issue with the commit and removed the extra file

* Fix #6154 - Filter logins field has wrong text color in Light theme (#6251)

* Fix #6154 - Filter logins field has wrong text color in Light theme

* Refactoring

* Merge rust-fxa branch to master (#6328)

* First FxA Rust integrations steps (#6077)

* restore use of fxa flow type arg (+4 squashed commits)
Squashed commits:
[c3769e1b9] Push is working!!
[3ea28f833] hookup apns and disconnect account
[a217e9e77] hookup device constellation
[4e65ff129] initial hookup

* hooked up sending tabs using the device list

* Remove using optional RustFirefoxAccounts.shared

* Add stub key migration code

* Update syncUnlockInfo to use getAccessToken(scope: FxAOAuthScope.OldSync)

* RustFirefoxAccounts.shared: guard against re-entrancy

* update app services SHA

* Use NSLock instead of objc_sync_enter

* app-services v0.50.0

* patch cleanup

*  PlacesAPI.resetBookmarksMetadata() is gone, and there is a PlacesAPI.resetBookmarkSyncMetadata.

* update Cartfile.resolved for protobuf 1.8.0

* Remove use of getAccount(), except in the sync case as that is in-progress

* revert changes to Fennec xcscheme

* wip legacy sync

* Fix #6108: populate remote_devices table with new rust fxa component

* hooked up webchannel for fxa login

* Hookup account management. Hookup token migration

* handle push messages for connect and disconnect

* Migration of tokens on updating app

* Show caution icon when migration failed, clear it when sign in to sync is shown

* Hookup webchannel deleteAccount and changePassword. Hookup help viewer in FxAWebView

* Fix migration bug: push notification re-registration is required

* CWTS (#6124)

Co-authored-by: Garvan Keeley <garvankeeley+github@gmail.com>

* Fix sync tabs not working, clients table not populated

* hookup device deleted push notification

* Remove force-unwraps in Profile sync code

* Fix account disconnect when app in foreground. Fix re-login not registering APNS.

* Hookup getTokenServerEndpointURL()

* disable broken tests

* Remove failing fxa push message test

* Use Firefox iOS client ID for FxA

* Set version to 23.1 for releasing a beta

* Handle fxaccounts:logout message.

* Add PushNotificationSetup.swift, and more FirefoxAccount removal

* Setup china server URL for FxA

* Fix #6201 - send to device from extension

* fix broken test

* Handle password change properly (#6205)

* Fix #6182: China sync servers using wrong URL (#6213)

* Fix #6211: app settings for sign-in was not loading the fxawebview (#6217)

* Fix #6212: CWTS options in webview need to be reflected in native settings (#6219)

* Fix #6212: CWTS options in webview need to be reflected in native settings

* remove for loop

* remove duplicate logout() calls

* Fix #6221 and #6229: page dismissed after changing password

* Fix #6207 - dismiss picker after send

* Fix displaying tab sent notification (was accidentally unhooked)

* Fix #6231 - Device picker not updating to latest list (#6238)

* Fix #6231 - Device picker not updating to latest list

* Avoid reloadData if the device list is unchanged

* Use new FxAConfig.china() syntax

* Remove FirefoxAccountSyncAuthState TODO for keychain storage (#6243)

* Remove FxAContentViewController, replaced by FxAWebView

* Add debugging info to push notification messages

* Update the appmenu UI when fxaccount is ready

* Add user profile caching for offline state

* a-s lib to 0.54.1

* fix test breakage

* getTokenServerEndpointURL() is now async

* Add comments to new rust fxa code, split app delegate up for clarity (#6280)

* Add comments to new rust fxa code

* Split up app delegate for push notification handling, and sync sent tab handling

* fix swift lint issues

* address nits

* disconnect was not deleting the cached user profile

Co-authored-by: Edouard Oger <eoger@fastmail.com>

* Revert "Fix #6154 - Filter logins field has wrong text color in Light theme (#6251)" (#6329)

This reverts commit 6cbbc75.

* Nb/6331 fix buddybuild tests (#6332)

* Fix #6331 - Updated useregent tests

* Removed webview user agent test

* Fix #4984: Tab Tray should respect 'Reduce Motion' accessibility setting (#6209)

Co-authored-by: Mykola Aleshchenko <maleschenko@homeawaycorp.com>

* Fix #6154 - Filter login field respects the dark and light theme in the app (#6339)

* [nobug, build] Change sentry DSN env var to remove specific version (#6340)

* Initialize Rust network stack on startup (#6319)

* Upgrade application services to v0.57.0

* Initialize Rust network stack on startup

* Fix #6206: Placeholder text does not follow dynamic font sizing (#6275)

* Fix #6315 - block certain data URL as top-level navigation (#6341)

* Fix #6315 - disallow data URL as top-level navigation

* Allow certain types of data URLs and add test case

* minor fixup

* Fix #6279: Rust fxa Staging server support (#6358)

* For #5958 enable sync ui tests - iOS 13.4 update (#6356)

* Fix #5959 - XCUITests enable syncUI tests and some more iOS 13.4

* fixing more tests

* fix findFromSelection test

* Fix #6224: call deviceConstellation()?.refreshState() only 1x (#6366)

* Fix #6376: migration of pre-rust bookmarks breaking tests (#6377)

For migration of users to Rust bookmarks backend, if users are signed in to sync then next sync will download their bookmarks.
For users not using sync, they will lose their bookmarks, this code extracts their bookmarks from browser.db into places.db.
This code is old at this point, we have no test case for it, and it has to run at a very specifc point in app init, which is hard to ensure given the async startup nature of the new firefox account system.
I think the risk of keeping this outweighs the risk of removing it.

* Fix #6362 - update new joiner build docs in README to mention requirement for Node.js to run bootstrap.sh (#6364)

* Fix #6367 - Take latest EarlGrey that fixes an issue with not being able to build when the project path has spaces (#6371)

* Fix #6184 Clear Recently Closed Tabs (#6185)

* add fragment support for search engine

* refactor and re indent file

* add test case of fragment usage for search term

* change source of onepassword-app-extension and update

* Update Cartfile

* Update Cartfile.resolved

* Fix UIView.isHidden must be used from main thread only

* Revert "Fix UIView.isHidden must be used from main thread only"

This reverts commit e5ce847.

* FirefoxAccountStateChange post notification wrapped to notify on the main thread.

* fix RustLogins warnings

* migrade deprecated .lproj

* add recommended base localizations

* Fix #6184 Clear Recently Closed Tabs

Co-authored-by: Garvan Keeley <garvankeeley+github@gmail.com>

* Fix #6386: fix data URL test case to pass correctly (#6387)

* Bug 1622800 - part 5: Schedule a single screenshot job to Bitrise (#6337)

* Fix #6381: Don't show URLbar lock icon for internal error pages (#6383)

* Fix #6390 and #6376: restore bookmark migration code to support tests (#6393)

We have tests using browser.db for their bookmarks, which need to be
migrated to places.db for them to run. This means we have a test case
for the migration code, which I didn't realize, and not having a test case
for the migration code was a rationale I used to remove it.

* Fix #6392: Move openTabCount telemetry to core ping (#6394)

* [nobugg, build] remove sentry setups being different for release and beta

* Cleanup old fxa code (#6388)

* Remove old FirefoxAccount code

* remove broken/unused tests

* Oops, missed calling disconnect() from Profile.removeAccount()

* Fix #6348: Correctly process engine enablements on first sync. (#6349)

* Bug 1622800 - part 6: Let l10n-screenshotis.sh pass `--test-without-building` to fastlane (#6406)

and output fastlane to bitrise logs.

* Fix issue for new Sumo Page string on iPhone (#6408)

* Fix #6380: crash due to login prompt string format having extra placeholder. (#6412)

* XCUITest - Fix Screenshot test for Long Press Menu Option (#6418)

* Fix #6419: allow video data url navigation (#6422)

* Bug 1631014 - Do no trigger a new set of jobs when PR is edited (#6424)

* Bug 1622800 - part 7: Bubble up bitrise workflow

* Bug 1622800 - part 8: Be more explicit about error handling

* Bug 1622800 - part 9: Introduce build task

* Bug 1622800 - part 10: Output fastlane exit code to be sure of what it returns

* Fix #4986: Settings screen distorted with biggest accessibility font size (#6373)

Co-authored-by: Mykola Aleshchenko <maleschenko@homeawaycorp.com>

* Some cleanups for the sync integration tests. (#6389)

I was attempting to run the sync integration tests on my machine,
and these tweaks to the python code were things I added to help
debug the process. (I didn't successfully get them running but
figured these might be useful nonetheless).

* Fix - Update Pocket more URL (#6425)

* Fix Issue #6078: Make it possible to disable link previews on iOS 13 (#6183)

* Fix #6421: debug setting to change FxA/sync servers (#6426)

* Fix #5798 for handling no network (#6173)

* webview.url was nil which prevents what’s new page from also having the error page

* Fix for issue #6180: [iPad] Long press on refresh menu string cut off (#6370)

* Fix for issue 6180: [iPad] Long press on refresh menu string cut off

* [nobug] Fix spelling mistake in 'LinkPreviews' string key

* Fix #6439: remove quote from downloaded URL (#6440)

* Issue #6439 followup: use webview.replaceLocation

* For #6432-L10nScreenshotsTestUpdateLocalesList (#6437)

* Updated aMozilla Application Services to v0.58.0 (#6450)

* Revert "Fix #5798 for handling no network (#6173)" (#6452)

This reverts commit d975952.

* For #6445 - XCUITest Update Pocket test (#6448)

* Fix #6190 - update local ui after fxa profile updates in the webview (#6365)

* Fix #6190 - update local ui after fxa profile updates in the webview

* Added force refresh to update profile when user updates the display picture or their name

* Fix #6438: Ignore unknown/unused class when de-archiving (#6455)

* Bug #6380: Change L10N string ID to ensure it gets updated (#6456)

This avoids a crash with the wrong number of placeholder items

* Fix #6398: master branch version number set to 0.0.1

* Remove unnecessary use of postMessage()  token (#6467)

* Remove unecessary use of postMessage() sec token

* push up compiled scripts

* Fix #6439: followup, ensure blob URL to download is the requested one (#6466)

* Fix #6439: followup, ensure blob URL to download is the requested one

* npm run build

* Fix #6310: always use Safari UA for desktop (#6468)

* Fix #6310: always use Safari UA for desktop

* Remove testDesktopUserAgent (for now) because of the hard-coded desktop UA

* Fix #6469: wipe the keychain when disconnecting FxA (#6470)

* Test to run snapshots tests faster (#6420)

* Test to run snapshots tests faster

* remove unused references

* rename tests, test suites and screenshots

* fix rename for parallel ex

* Fix #6472: restore logins keychain keys so it can be opened  (#6494)

* Fix #6472: restore logins keychain keys so it can be opened

* Cleanup unused code

* Fix #6474: reconfig FxA in the notification service EVERY time a push arrives (#6496)

* Issue #6253 - add undo button on toast when removing bookmark from photonActionSheet (#6306)

* Fix #6433 - Landing strings for onboarding screen (#6509)

* Fix #6232 - Crashes when trying to add account picture (#6484)

* Fix #6460: reset push registration on not decrypted error.  (#6511)

* Fix #6460: reset push registration on not decrypted error. part1, cleanup old keychain code.

* part2: actual fix, clear apnsToken and get a new one

* Fix #6435: strings for fxa qr code (#6510)

* Fix #6433 - Updated automatic privacy description (#6512)

* Bug 1622800 - part 11: Enable screenshot tests for all locales

* Bug 1622800 - part 12: Use dedicated bitrise workers

* Bug 1622800 - part 13: Create a hook to run the production screenshot jobs

* Bug 1622800 - part 14: Do not fail job on missing bitrise logs

* Bug 1622800 - part 15: Add missing treeherder symbol (#6529)

* Fix #6524: SentryDSN not set properly in app extensions (#6534)

* Fix #6524: followup, fix bad merge

* Adds basic Glean telemetry integration. (#6441)

* Adds basic Glean telemetry integration.  This adds the baseline ping only, along with enabling the Glean debug tools through the custom URL processing.

* Bug #6550: China sync service toggle not calling FxA reconfigure (#6555)

* Fix #6550: Put callback from photo permissions onto main thread (#6556)

* Fix #6483: downloadqueue callbacks should be main thread only (#6558)

* Fix #6480: generateDefaultFaviconImage make thread safe to fix crash (#6557)

* Properly show the device ID when viewing FxA Settings (#6532)

Related to issue 6407

* Fix #6551: update a-s lib to 0.58.3 for sendtab bug with locked device (#6560)

* [nobug] console.app logging for push notification service startup/shutdown

* For #6522 - XCUITests update homepage label (#6523)

* Bug 1622800 - part 16: Stop using jlorenzo's workflows

* Include email in the webchannel status  message (#6565)

* Update release checklist template (#6570)

* update cartfile

application-services needed to be updated to account for new swift version

* Fix #6575: debug setting to toggle china edition of app (#6576)

* Fix #6550: make RustFirefoxAccounts accountManager a Deferred (#6574)

* Fix #6550: make RustFirefoxAccounts accountManager a Deferred
This ensures that there is no race condition of code accessing the accountManager before it
has been initialized (which takes a few ticks to complete).

* Send push notification sentry errors on release, not just beta

* Upgrade Application Services to v0.59.0 (#6578)

* For #6587 - L10n XCUITest fix failing test for right to left writing locales (#6588)

* [nobug, build] remove autogenerated JS files (#6603)

* For #6587 - FollowUp to Fix RTL screenshots tests (#6600)

* For #6587 - FollowUp to Fix RTL screenshots tests

* fix test long press text option

* remove unnecesary comments

* update to account for new api in moz dependency

* Feature #6434 - AB Test New Onboarding (#6601)

Feature #6434 - AB Test New Onboarding

Combined two intro view controllers into one and move some logic from browser view controller to a simple view model

* Bug 1636842 - part 1: Output bitrise artifacts to a different folder

* Bug 1636842 - part 2: Add script to enforce number of screenshots

* Bug 1636842 - part 3: Just log out errors while we get all locales fixed

* Bug 1636849 - Add indexes to l10n jobs (#6605)

* [nobug, build], Bugzila 1638200: Glean in copy carthage deps phase (#6617)

* Fix #6615 master branch, added debug path for onboarding screen (#6616)

Fix #6615 master branch, refactored bvc and added a delay to support showing onboarding screen in case of slow network condition or leanplum server isn't available

Also added simple message to send to sentry when leanplum is unable to fetch A/B test variables

* revert to old code, old XCode

* For #6622 - XCUITest fix custom engine from incorrect template (#6623)

* Added debug menu to show type of leanplum settup (#6632)

* reverted Cartfile.resolved to old version

* Fix #6559: China server switch should not hide when logged-in to FxA (#6633)

* Fix #6559: China server switch should not hide when logged-in to FxA

* add telemetry to tapping the switch

* update path to polyfill

* Rename TabTrayController (#6639)

* fixed homepage

* QR Code Login (#6650)

* New FirefoxAccountSignInViewController for QR Code Sign In and Email Sign In/Sign Up

* Update telemetry to match new requirements.

* Fix issue where email login does not appear to finish (#6659)

* Present qrcode vc modally and push fxa web vc and adjust FxaWebViewController dismissal behavior based on parent vc that presented it

* title on FirefoxAccountSignInViewController, show instructions label in dark mode (#6657)

* Add  title on FirefoxAccountSignInViewController and handle label colors in dark mode

* Updated leanplum to v2.7.2 (#6665)

* Fix telemetry events for qr feature (#6663)

* Add a way to track parent launch page for sign in so that we don't always send app_menu in sign in telemetry events from BrowserViewController

* Fix #6647: Send to device extension needs to wait for FxA to initialize (#6666)

* Fix #6647: Send to device extension needs to wait for FxA to initialize

* disable user interaction

* Fix #5968: remove use of googlemapping.json and Disconnect cat., update shavar (#6664)

* Fix #5968: remove use of googlemapping.json and Disconnect cat., update shavar

* update tests

* For #6652 - XCUITests fix FxA tests after new ui for QR code (#6655)

add screenshot tests and IDs for the new buttons

add comment and use ids

fix sync int tests

* Fix #6668 - Onboarding card should get built in theme on relaunch (#6673)

* Bug 1638073 - Update api to be appropriate for firefoxci (#6610)

* Fix #6668 - Old onboarding card should respect proper theme background (#6685)

* set ThemedTableViewCell textLabel color to themes rowText color (#6678)

Co-authored-by: Tomislav Eric <tomislav.eric@arconsis.com>

* For  #6652 - Follow up fix XCUITests after FxA navigation bar change (#6683)

* For  #6652 - Follow up fix XCUITests after FxA navigation bar change

* fix more tests

* more fixes

* Fix #6687: QR pairing in China uses different url (#6688)

* Fix #6687: QR pairing in China uses different url
Part 1: cleanup getting the china setting

* Part 2: Fix actual bug

* swift lint fixup

* Use getPairingAuthorityURL

* Bug 1636842 - part 2: Make check-screenshots script smarter about the number of locales (#6689)

* Fix #6701: disable QR Pairing for China (#6702)

* Isabelrois/fix screenshots tests (#6704)

* Fix Screenshots tests

* remove print

* Bug 1636842 - part 3: Error out if errors are found while looking at screenshots (#6713)

* Glean delete user data (#6577)

* Set up Glean delete ping and build scripts

* Part 1 #5592 Enable testLongPressOnAddressBar (#6625)

Adding waitForExistence(app.textFields[address])

* Fix #6721: FxAccountManger crash filling deferred 2x, make re-entrancy-safe (#6724)

* Fix #6721: FxAcountManger crash filling deferred 2x, make re-entrancy-safe

* Unrelated: fix warning about this unused code being unsafe

* Update release_checklist_template.md

* Update release_checklist_template.md

* Update README.md (#6728)

* Fix #6611 - Updated sentry log message to include more details for failing to start Leanplum (#6727)

- Added option in debug menu to reset ETP Constants and show cover sheet

* Fix #5592 - Part 2 Enable testCopyPasteComplete (#6725)

* For #6719 - XCUITest fix navigation test (#6720)

* Add docs for glean testing walkthrough (#6732)

* Add docs for glean testing walkthrough

* reduce example tag character count

* Fix #6749 - Fix navigation bar for sync integration tests (#6754)

* Fix #6729 - Firefox for iOS crashes when adding an account picture (#6743)

* Fix #6708 : [Today Widget] Update current widget strings/colours (#6750)

* Fix #6708 : [Today Widget] Update current widget strings/colours

* title of today widget changed
* Update buttons
* Update button colors and strings
* Update widget title string
* Ensure colors change correctly for dark mode
* edited gradient colors names + code styling fixes

* Fix #6464: Search Options: Replace Yandex Market with Hotline for Ukrainian localization (#6681)

* [nobug, cleanup] Rename blacklist to blocklist (#6783)

* [nobug, cleanup] Rename blacklist to blocklist
* rename whitelist to safelist

* Fix #6590: pass deeplink _utm params to fxa webview (#6782)

* Fix #6670: allow empty title for sendtab url (#6781)

* Fix#6737 Incompatible Swift version - framework was built with 5.2.2 (#6738)

update glean and app-services

* update xcode version 11.4 to 11.5

* Update README.md

* [nobug, build] update glean generated file path (#6787)

* [nobug, build] update glean output file name  (#6789)

* [nobug, build] update glean generated file path

* [nobug, build] update glean output file name

* Updated lp event so it doesn't log settings data (#6799)

* Fix #6796 - [XCUITests] TP tests failing after renaming an element ID (#6798)

Co-authored-by: Simon Basca <simonbasca@192-168-0-107.rdsnet.ro>

* Fix #6708 followup, missing display name for Beta builds (#6791)

* [nobug, build] update Glean from 31.0.2 to 31.1.0 (#6784)

* bug 6795: rename whitelist to safelist in tests (#6803)

* use new API UIGraphicsImageRenderer to replace UIGraphicsBeginImageContextWithOptions (#6805)

* Fix searchbar hide issue (#6703)

* remove hideSearchBarWhenScrolling in viewDidAppear

* unfold searchBar on viewDidAppear

Co-authored-by: Tomislav Eric <tomislav.eric@arconsis.com>

* Fix #6802: onReady() not filling the Deferred in the error case (#6804)

* Fix #6692 - Moving most of the refactored leanplum A/B test changes from v26.0 to master (#6736)

* Resolve two warnings that happen in debug or beta builds

* Revert "Resolve two warnings that happen in debug or beta builds"

This reverts commit d872c02.

* Fix #6788: enable QR pairing for china edition (#6811)

* Fix #6723: telemetry lib out-of-space fatal exception (#6813)

Update the lib to v1.1.3 which has fix for this

* [nobug, build] remove duplicate carthage copy-frameworks objcexc. step (#6832)

* Bugzilla 1646756: close last priv tab clears the WKWebViewConfiguration (#6827)

* Bugzilla 1646756: close last priv tab clears the WKWebViewConfiguration

* use prefkey for blockpopups

* Try installing Carthage (#6836)

* Try installing Carthage

modify carthage_command file

add comment

* remove space

Co-authored-by: isabel rios <isabelrios@MBP-de-isabel-2.home>

* Update SnapshotHelper file (#6839)

Co-authored-by: isabel rios <isabelrios@MBP-de-isabel-2.home>

* Fix #4717 - Provide feedback when pinning a site (#6785)

* Fix #4717 - Provide feedback when pinning a site from inside the Library panel

Display toast notification on the Library panel
Display toast notification on the Bookmarks panel
Display toast notification on the Website tab

* Change to remove

Co-authored-by: Vlad Dramaretskyi <vlad@Vlads-MacBook-Pro.local>

* Fixed #6730: open apps preview should hide all screens in private browsing (#6757)

* Fixed #6730: open apps preview should hide all screens in private browsing

* #6730 changed color of backdropContainer and changed it edges to fill whole screen, added it TrayView as well

* Added new color #1D1133 for Ink90

* fixed the cut-off of copied link labels and the contrast of the color for light/dark mode

* bug fixes : widget constraints + color contrasts of labels in light/dark themes + added pdf button designs

* removed commented lines

* styling edits

* edit in constraint for widget button

* removed tab + edits to constraints for buttons

* fixed support for ios 112.4 colors

* removed extra tabs

* code styling

* edits to label color variables definetions

* code styling

* code styling

* code styling

* Push subscription expired handling (#6851)

* Use .afterFirstUnlock on all apnsToken instances

* Re-register push on subscriptionExpired

* Send dev in UA instead of 0.0.1 (#6849)

The durable sync servers 503 if major version == 0 to sidestep
a bug.

* Revert "Fix: #6764 #6763 #6766 #6765 String cutting off in widget after copying link and color contrast of the copied link label" (#6860)

* Revert "code styling"

This reverts commit b1af71b.

* Revert "code styling"

This reverts commit cd8a09b.

* Revert "code styling"

This reverts commit a552d58.

* Revert "edits to label color variables definetions"

This reverts commit 9c5da0a.

* Revert "code styling"

This reverts commit 911f098.

* Revert "removed extra tabs"

This reverts commit 9545675.

* Revert "fixed support for ios 112.4 colors"

This reverts commit 5dd6c0c.

* Revert "removed tab + edits to constraints for buttons"

This reverts commit 33f8d21.

* Revert "edit in constraint for widget button"

This reverts commit 48a429a.

* Revert "styling edits"

This reverts commit dd561ac.

* Revert "removed commented lines"

This reverts commit 4085267.

* Revert "bug fixes : widget constraints + color contrasts of labels in light/dark themes + added pdf button designs"

This reverts commit 981ead9.

* Revert "fixed the cut-off of copied link labels and the contrast of the color for light/dark mode"

This reverts commit 8ec8a75.

* Noorhashem/today widget fixes (#6861)

* fixed the cut-off of copied link labels and the contrast of the color for light/dark mode

* bug fixes : widget constraints + color contrasts of labels in light/dark themes + added pdf button designs

* removed commented lines

* styling edits

* edit in constraint for widget button

* removed tab + edits to constraints for buttons

* fixed support for ios 112.4 colors

* removed extra tabs

* code styling

* edits to label color variables definetions

* code styling

* code styling

* code styling

Co-authored-by: noorhashem <nour.hashem95@gmail.com>

* Fix #6862: UA test broken (#6863)

* added conditional check for triangleIndicies > 2

check the count, sometimes there is no triangleIndicies array provided, presumably because it hasn't changed even if the verticies move?
Probably should update the vertices and texCoords without rebuilding Plane at this point, but for now just ignore it so we don't crash.

* update project file with WEBXR Flags

Added flags for building XRViewerDev and XRViewer conditional code.

* moved webxr code out of AppDelegate into extension

* moved webxr code out of Tab into extension

* added conditionals around webxr code

* For #6865 - UI Test fix Strict TP test (#6866)

* For #6865 - UI Test fix Strict TP test

* update previous test

* Fix #5934: Wrap text in theme settings (#6872)

* Fix #6844 - Condition for when internet ins't available or leanplum is slow wasn't met (#6850) (#6852)

* Rename master branch to main

* rename master branch to main

* Issue 6711: Update colours in ETP shield icon (#6874)

* Issue 6711: Update colours in ETP shield icon

* Fixes

* Fix #6722, fix #6271: Library panels crashing on text size change (#6881)

* Fix #6846 - Add Search button to tabbar in home screen (#6853)

* Add Search button to tabbar in home screen
* Reduce icon flickering and add tests

* Introducing FxAWebViewModel (#6733)

* removing logic from controller

* removing webview within viewModel

* inject LeanPlumClient via init

* cleanup

* cleanup

* applying pr changes

* applying second waves of changes

* Fix #6722: no reload() datasource when app backgrounded (db closed) (#6884)

* Fix #6879: reconfig will crash if FxA init is in-progress (#6880)

* Update .swiftlint.yml

* Bug# 6733: refactor: cleanup and simplify logic flow (#6895)

* Dismiss QR Scanner if it not available (#6870)

* not sure if these should be checked in

* merge failure

this section should have been removed.

* Update UserAgent.swift

fixed user agent to have WebXRViewer in it as needed by the polyfill

* fix up some merge errors

Added XR directory back to project.

Co-authored-by: Mykola Aleshchenko <nick130586@gmail.com>
Co-authored-by: Mykola Aleshchenko <maleschenko@homeawaycorp.com>
Co-authored-by: isabelrios <isabelrios@gmail.com>
Co-authored-by: Nishant Bhasin <nikieme3@gmail.com>
Co-authored-by: Garvan Keeley <garvankeeley+github@gmail.com>
Co-authored-by: Nishant Patel <nishantpatel2718@gmail.com>
Co-authored-by: Yuşa Doğru <yusadogru@gmail.com>
Co-authored-by: Edouard Oger <eoger@fastmail.com>
Co-authored-by: Daniela Arcese <darcese@mozilla.com>
Co-authored-by: Johan Lorenzo <jlorenzo@mozilla.com>
Co-authored-by: Roman Podymov <podymfrombryansk@yandex.ru>
Co-authored-by: Andrei Bodea <bodeaandrei42@gmail.com>
Co-authored-by: KrystynaKruchkovska <37312950+KrystynaKruchkovska@users.noreply.github.com>
Co-authored-by: Mustafa Hastürk <mustafa.hasturk@yandex.com>
Co-authored-by: Vanna Phong <vphong.dev@gmail.com>
Co-authored-by: Noor Hashem <nour.hashem95@gmail.com>
Co-authored-by: Stephen Barker <SteveOfTheStow@users.noreply.github.com>
Co-authored-by: Ryan Kelly <ryan@rfk.id.au>
Co-authored-by: Kayla Galway <kayla.galway@gmail.com>
Co-authored-by: Callie Dunn <q2r5@live.com>
Co-authored-by: Lamour <LamourBt@users.noreply.github.com>
Co-authored-by: Adilet Askerov <adilet.askerov@nu.edu.kz>
Co-authored-by: Garvan Keeley <garvankeeley@gmail.com>
Co-authored-by: Travis Long <tlong@mozilla.com>
Co-authored-by: Vlad Filippov <vlad.filippov@gmail.com>
Co-authored-by: Justin Wood (Callek) <callek@gmail.com>
Co-authored-by: Tomislav Erić <hi@tomislav.mobi>
Co-authored-by: Tomislav Eric <tomislav.eric@arconsis.com>
Co-authored-by: SimonBasca <32130829+SimonBasca@users.noreply.github.com>
Co-authored-by: Haris Zaman <mohammadharis@gmail.com>
Co-authored-by: nishiths23 <nishithsingh23@ymail.com>
Co-authored-by: Simon Basca <simonbasca@192-168-0-107.rdsnet.ro>
Co-authored-by: visitor <625927650@qq.com>
Co-authored-by: Stefan Arentz <stefan@arentz.ca>
Co-authored-by: isabel rios <isabelrios@MBP-de-isabel-2.home>
Co-authored-by: Vlad Dramaretskyi <vlad@Vlads-MacBook-Pro.local>
Co-authored-by: Joe Gasiorek <joeyg@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 Story size Blocked Cannot move forward with the issue QA Verified Issue verified by QA
Projects
No open projects
Development

No branches or pull requests

6 participants