[Bug] app_opened telemetry event unexpectedly does not run on every app launch

[mcomella]

The app_opened telemetry (docs) says:

A user opened the app

However, this isn't true. For example, this event will not trigger for the following events:

• User clicks the homescreen icon (triggers), hits the Home button, and clicks the homescreen icon (does not trigger)
• User clicks a link from another app (triggers), hits the Home Button, and clicks another link from another app (does not trigger)

This is because the telemetry event is tracked from HomeActivity.onCreate: i.e. it will only be triggered if HomeActivity needs to be created. This probe is roughly equivalent to "a user opened the app for a cold startup" (but will trigger again if HomeActivity is destroyed in the process for any reason).

We should identify the use case for this probe (in the original data review, perhaps?) and correct the behavior accordingly.

Note: matching Android lifecycle events to user perceived app behaviors is very complex: if we're short on time, we may wish to identify a simple implementation-biased use case (e.g. HomeActivity.onStart is called, roughly HomeActivity is displayed) rather than matching against a user behavior (e.g. "has user opened the app?"). If we do implement something more complex, we should consider using a state machine (I had hoped to eventually adapt the StartupTimelineStateMachine for this purpose). Additionally, there are some relevant conclusions about the Activity lifecycle in this investigation I did.

• Fenix version: upstream/master: edc75c3

┆Issue is synchronized with this Jira Task

[liuche]

@vesta0 just to loop you in, the app_opened event is only triggered when Fenix is cold-start opened (like if the user has swipe-closed it, or Android has killed it in the background which a user wouldn't know). So it doesn't track someone reopening the app (so if they have it open for days in the background, this would event would only send once).

Is this how you're understanding app_opened? Perhaps we should re-name this probe and update the description - and if longer term you want to track "opening the app" it sounds like we need to add another probe.

EDIT: deleted an older comment that was inaccurate

[mcomella]

I took a little time to map the current probe to user behavior.

The probe represents cold startup of the homescreen and browser screen (& possibly others but those are the ones I'm familiar with).

These screens will need to be cold started after:

• the device closes Fenix to conserve system resources
• the device is restarted
• the Activity is "finished", which is when:
  • the app is swipe closed (I'd need to double-check this)
  • the user presses the back button to leave those screens (rather then home; we may override this behavior so I'd want to double-check this)
  • when the developer intentionally calls finish() (which I can't think of a use case for but it might happen)
  • perhaps some other cases I'm unaware of

There's a last case for "configuration change" (e.g. rotating the device, plugging in an external monitor) but I think we override the system behavior here so this does not make the app need to cold start.

(note: this is pretty complicated so I had to reference a previous analysis to figure it out. I'll link it because it may be useful to others https://docs.google.com/document/d/13AEfW4k9euEoUBPJgnnRh4CSIVgg5h_eiy5a4kUoivI/edit#heading=h.o46klxjjqror)

[liuche]

I talked to one of the engineers, and I believe this probe is working as intended, that it's "app is cold-started". I can't remember who exactly I talked to, but I think it might have been @boek ?

[boek]

@liuche that is correct. There I can't remember who exactly made the decision but there was a discussion on whether we wanted this versus the foreground/background event we had in Focus. This event was also originally created as a marketing need from Leanplum to maybe trigger an action?

[mcomella]

I think we should update the metrics.md documentation to reflect that this is a cold start only (and cold start to the homescreen/browser, not the app): the current documentation is misleading (e.g. ecsmyth misused it for an analysis they were doing on seeing how often the app is opened and from what type of opening).