Consider reverting tracker to navigator.sendBeacon()

[guigrpa]

In #1160, it was proposed to switch from navigator.sendBeacon() in the tracker to fetch() with keepalive: true. The goal was to avoid some ad blockers from preventing analytics data to go through. This change was implemented in #1163.

I was thinking about the visit duration accuracy issues (#570, #1047) and have found some limitations with the fetch approach, especially if, as suggested by @gnarlex here, we want to send an event when a visitor leaves:

I'm not completely familiar with the analytics script. But couldn't we add a visibilitychange handler in the analytics script and use Navigator.sendBeacon() there to send information to umami once the user leaves a page?

Unfortunately, fetch() even with keepalive: true does not send events reliably on visibilitychange events. I've done some testing with the following code and include my results below:

document.addEventListener('visibilitychange', () => {
  const event_name =
    document.visibilityState === 'visible' ? 'VISIT_START' : 'VISIT_END';
  window.umami.trackEvent(event_name);
});

Case	Chrome	Firefox	Safari	Mobile Safari
Refresh	❌	✅	❌	❌
Switch to ≠ (full-screen) app in same screen	✅	✅	✅	✅
Switch to ≠ tab in same window	✅	✅	✅	✅
Navigate away	❌	❌	❌	❌
Close tab	❌	❌	❌	❌
Close browser	❌	❌	❌	❌

Note that fetch() only sends the event reliably when the web application is still alive. (Browser versions: Chrome v104, Firefox v103, Safari v15.3, Mobile Safari v15.6)

Switching (back) to navigator.sendBeacon() improves the results substantially:

document.addEventListener('visibilitychange', () => {
  const event_name =
    document.visibilityState === 'visible' ? 'VISIT_START' : 'VISIT_END';
  const queued = navigator.sendBeacon(
          'http://xx.xx.xx.xx:3000/api/collect',
          JSON.stringify({ type: 'event', payload: { /* ... */ event_name } })
        );});

Case	Chrome	Firefox	Safari	Mobile Safari
Refresh	✅	✅	✅	✅
Switch to ≠ (full-screen) app in same screen	✅	✅	✅	✅
Switch to ≠ tab in same window	✅	✅	✅	✅
Navigate away	✅	✅	✅	✅
Close tab	✅	✅	✅	❌
Close browser	✅	❌	❌	❌

This is much better: we now have post-mortem events in many cases. Based on this, I propose reverting to navigator.sendBeacon(). In order to avoid the ad blocking problems, we could add fetch as fallback, as @rohandebsarkar suggested in #1160.

What do you think?

[Maxime-J]

Hello, given your testing code,
trackEvent doesn't use the keepalive flag, so indeed, it doesn't give good results in these cases compared to navigator.sendBeacon()

Results would probably be on par if you test with fetch and keepalive

fetch('http://xx.xx.xx.xx:3000/api/collect', {
  method: 'POST',
  body: JSON.stringify({ type: 'event', payload: { /* ... */ event_name } }),
  headers: { 'Content-Type': 'application/json' },
  keepalive: true
})

Would be interesting to see the results 👍

[guigrpa]

Ah, from reading #1160 I was assuming that keepalive was being used. I will then add that case and post the results in this issue!

[guigrpa]

Here's a new summary, including fetch with/without keepalive and navigator.sendBeacon. With fetch() + keepalive, the situation is similar to navigator.sendBeacon(), but we still don't receive VISIT_END in Firefox for some important cases (navigate away, close tab). Currently the best solution seems to be navigator.sendBeacon()…

Case	Chrome	Firefox	Safari	Mobile Safari
1. fetch() (no keepalive)
Refresh	❌	✅	❌	❌
Switch to ≠ (full-screen) app in same screen	✅	✅	✅	✅
Switch to ≠ tab in same window	✅	✅	✅	✅
Navigate away	❌	❌	❌	❌
Close tab	❌	❌	❌	❌
Close browser	❌	❌	❌	❌
2. fetch() (keepalive: true)
Refresh	✅	✅	✅	✅
Switch to ≠ (full-screen) app in same screen	✅	✅	✅	✅
Switch to ≠ tab in same window	✅	✅	✅	✅
Navigate away	✅	❌	✅	✅
Close tab	✅	❌	✅	❌
Close browser	✅	❌	❌	❌
3. navigator.sendBeacon()
Refresh	✅	✅	✅	✅
Switch to ≠ (full-screen) app in same screen	✅	✅	✅	✅
Switch to ≠ tab in same window	✅	✅	✅	✅
Navigate away	✅	✅	✅	✅
Close tab	✅	✅	✅	❌
Close browser	✅	❌	❌	❌

[guigrpa]

Btw, for the new case (2.) I used exactly what @Maxime-J suggested:

fetch('http://xx.xx.xx.xx:3000/api/collect', {
  method: 'POST',
  body: JSON.stringify({ type: 'event', payload: { /* ... */ event_name } }),
  headers: { 'Content-Type': 'application/json' },
  keepalive: true
})

[Maxime-J]

Ah, from reading #1160 I was assuming that keepalive was being used.

And it was for a little while, not in trackEvent() though, only for events attached to links.
It was refactored recently and links handled differently.

But I follow you, tracker needs a way to send events in a persistant way.
Your tests will certainly help them to chose a method for these requests.

[rohandebsarkar]

But this will still not improve the visit duration accuracy. This will only solve the persistence issue faced by the trackEvent() function when the visibility changes.

For improving the accuracy of the visit duration, we would need a way to remember when the user actually leaves the site, this can be done by doing some database changes. I proposed this on #570 (comment).

[guigrpa]

@rohandebsarkar Moving to sendBeacon is not sufficient, but it would be a step in that direction. Umami users could then store events on page leave and use data in their database to manually calculate their visit duration following their (arbitrary) definition.

[guigrpa]

@rohandebsarkar Regarding #1471: Are you also considering the navigator.sendBeacon() (with fetch fallback)?

[mikecao]

The tracker has been updated to no longer use fetch keepalive (doesn't work in Firefox) or navigator.sendBeacon (blocked by all adblockers). Update will be in the next release.

[mikecao]

Fixed in v1.38.0.