Event logging client for (Modern) Event Platform

[bearloga]

Fixes Phabricator ticket: T228180

Notes

This pull request (a re-do of #3603) adds the iOS version of the Event Platform Client (EPC-iOS) for integration with the modern Event Platform. EPC is an effort to unify algorithms, identifiers, and behaviors across MediaWiki, iOS, and Android.

This initial release supports:

• remote sampling configurations, which puts sampling rates into the centralized stream configuration

For additional background information on instrumentation within the Modern Event Platform, please refer to this page.

Next steps

• While the library has already been heavily reviewed and iterated on in the previous PR, but it wouldn't hurt to review again – especially with respect to thread safety
• Implement StorageManager and NetworkManager to make EPC operational
  • Temporarily remove storage manager facilities (except app install ID & usage data sharing opt-in/out setting retrieval) for initial release until we settle on specification of persistence
• Restore EPC.shared singleton (was temporarily changed until storage & network managers are implemented)
• Resolve open question around per-wiki stream configurations (in BUOD <> Apps sync agenda)
  • Will not have per-wiki stream configurations for apps, any language-specific deviations in sampling settings may need to be handled via the proposed exemptions idea
  • Apps will query 1 MW API (metawiki's) to download stream configs (side note: EventGate uses metawiki's stream configs as its source of truth)
• Make EPC respect user's usage data sharing preference the way Event Logging does
• Q&A functionality with a simple schema & instrument (test schema+stream for analytics events tracked in T259714)
  • Create a schema in this repository: /analytics/test/1.0.0
  • Configure a stream via this file: test.instrumentation and test.instrumentation.sampled
  • Test events verified in database/table (event.test_instrumentation)
• Pilot instrumentation (Edit History Compare feature)
  • Schema (/analytics/mobile_apps/ios_edit_history_compare)
  • Instrumentation (EditHistoryCompareFunnel.swift)
  • Stream (ios.edit_history_compare)
• BLOCKER EventStreamConfig API returns empty configs as [] instead of {} (T259917)
  • Merged
  • Deployed (scheduled for 2020-09-08 MediaWiki deployment train)
  • configURI changed from fake endpoint at pai-test to production endpoint at meta wiki

Future release may support:

• stream cc-ing, which automatically logs events to multiple streams without requiring multiple calls on the instrumentation side

[bearloga]

Just wanted to note that we (Product Infrastructure Data team) decided to put the stream cc-ing feature on the back-burner, as there are still some unresolved questions around how it should really work (implicit mapping as was implemented here or explicit mapping as suggested in T256169#6254835), and we didn't want this feature to be out on iOS from the get-go and potentially never on MediaWiki. All deliberation will happen in T256165.

[tonisevener]

@bearloga PR has been updated with StorageManager and NetworkManager implementations. I also added a couple of (identical) test log calls - one when you pull to refresh on the Explore tab and another when you tap the Settings icon on the Explore tab. I did a little bit of renaming and some more thread safety stuff in the library, hope that's cool. Let me know how this looks on your end, if it seems good we can bring it back to our side for converting a particular feature's logging & final code review.

[bearloga]

Note to self: depending on the outcome of the discussion taking place in the identifiers fragment patch (https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/615559/3/jsonschema/fragment/analytics/identifiers/current.yaml#22), session_id and device_id may need to be renamed to app_session_id and app_install_id

[bearloga]

@joewalsh Thanks for fixing the tests and making those updates!

Status update: We're getting closer to being able to create the pilot schema. There was a lot of discussion and iteration on https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/615559 and we've arrived at a version we're all satisfied with, so that'll be merged tomorrow (Friday) and I'll probably have pilot schema + pilot instrument up for review by EOD/EOW. Exciting!!!

[joewalsh]

Looking great overall! I did my review in the form of proposed changes via a PR on this PR. The review comments/questions are in that PR: #3682

[tonisevener]

I have been reviewing this as a part of the refactor PR and from my perspective these are ready to merge into main. I had one possible cleanup comment here, but client_dt has tripped me up in the past so I could just be missing something. We're happy to discuss the refactor further in our meeting today.

[bearloga]

From the meeting: We decided to move forward with this PR once #3682 is merged into it

[tonisevener]

🎉 🎊 Thanks everyone for their hard work on this! Very excited to get this in.