Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Support Ping requests to report a better 'time on page' #2041

Open
mattab opened this Issue · 33 comments

14 participants

Matthieu Aubry Anthon Pang Anonymous Piwik user Kevin Locke johsin18 RazvanMihaiu Fredrik Ekelund Jan Grewe Vladimir Asaf Yishai ru108 Ismay IPL01 Glisse1
Matthieu Aubry
Owner

Currently, the last page view is set with a 'time on page' of zero seconds.

Instead, we could regularly ping the Piwik server with a request that notifies Piwik that the user is still on the website.

This request should not record new page view, only increase the visit_total_time counter.

Also, we should only throw this request when the user is actually looking at the page, not doing something else in some other tabs.

The risk with this feature is that instead of improving the report accuracy, we end up inflating numbers: a lot of users leave pages open in other tabs for hours and don't do anything about it. We should heavily test to ensure we never throw a request on a page where the user didn't interact with recently. Some interactions we can maybe detect: mouse click, mouse move, etc.

Of course, we should not make everything slow by adding handlers on mouse move, etc.

Also, once this feature is implemented, a new checkbox should be added to the Advanced section in the 'Tracking Code' admin page. Checking it will generate JS code that enables this feature.

Anthon Pang
Collaborator

(In [3788]) fixes #739 - addEventListeners() on mouseup and mousedown; good catch, Matt!

  • utf8_encode() - required to convert multibyte char to string of bytes; for unit testing, it needs to be a separate function

  • killFrame() - would be called after a trackPageView(); used to prevent one's site from being frame by third-parties; there are tradeoffs (as you point) out, so it is not enabled by default

refs #2041 - setHeartBeatTimer - uses setTimeout, so it happens only once (vs setInterval is recurring); only happens on the landing page; intended to be used to improve bounce rate report; changed it to send ping=1 in the request

Anthon Pang
Collaborator

I reworked setHeartBeatTimer() in [3830]. It now takes an initalPingDelay and recurringHeartBeatDelay as parameters. By default, this is disabled.

Anthon Pang
Collaborator

The initial ping request contains "ping=1". Use this to improve bounce rates and visit time.

Subsequent heartbeat requests (if recurringDelay != 0) contain "hb=1".

I'm going to defer adding event handlers as there are issues with propagating some events like keyclicks.

Mental notes to self:

  • if you only send a beat when there's activity within the last n seconds, then on average, you'll expect the total time to be overstated by n/2 secs
  • to offset/compensate, you can instead send a beat if there's been activity in the last n/2 seconds

  • if there's no activity for n secs, do you stop checking for activity or stop sending beats?

  • what if activity resumes again? how long is too long ago to to resume beats?
Matthieu Aubry
Owner

how do you define 'activity'? Will the ping be sent if user is looking at another tab, another window, or even away from computer?

Anthon Pang
Collaborator

activity: mousemove, click, mouseup, mousedown, mousewheel, scroll, DOMMouseScroll, resize, keypress, keydown, keyup, focus, and blur

  • need this many because of browser differences an the possibility of interference from other eveny handlers
  • handler simply sets the lastActivity timestamp

Yes, ping would be sent. That's why there's a check for activity. If there's no activity, I propose to no longer check.

I'll commit my changes after some more testing.

Anthon Pang
Collaborator

(In [3838]) refs #2041 - setHeartBeatTimer() now takes the minimumVisitLength and the heartBeatPeriod (time between pings) as parameters; this is simpler than initial ping + recurring hb; requests contain "ping=1".

also updated to jslint 2011-02-03

Anthon Pang
Collaborator

(In [3839]) refs #2041 - rollback buggy n/2 logic

Anthon Pang
Collaborator

(In [3840]) refs #2041 - undo previous optimizaton to ping on first page of a new session

Anthon Pang
Collaborator

(In [3841]) refs #2041 - add guard

Anthon Pang
Collaborator

Client-side changes are done, so it would be nice to have the server-side change done in time for 1.2 but we can roll into the next milestone.

Matthieu Aubry
Owner

How often are the activity tests done? every second? Just thinking of keeping JS performance overhead low with this new feature.

I wont be able to do this in time for 1.2, so it will go in 1.3. Is pinging disabled by default? thx

Anthon Pang
Collaborator

Yes, disabled by default; frequency is user-defined.

Matthieu Aubry
Owner

Before enabling this by default I think we could try it on some heavy websites, that are using GA, other libraries (jquery eg.) and doing Javascript heavily. Then we could see if Piwik adds any unexpected performance overhead.

For example it would be nice to say: Piwik will query a few JS variables every second to assess if the visitor is active or not, but will not conflict with any other library or Javascript and should not affect performance at all.

What do you think? for example, have you tried it on a a running piwik loading the dashboard full of widgets.

Matthieu Aubry
Owner

I think, disabled by default, but easy to enable, this would make it a really cool feautre. Also, should be made available via the 'advanced' javascript tag generator see #1845

Anonymous Piwik user

Just wondering where we are on this. I'm in the process of switching several sites from GA to Piwik, and this feature is pretty important to me. Does 1.x mean it's going to be in an indefinite release between now and 2?

Matthieu Aubry
Owner

Polrbear, yes currently not roadmapped for 1.7 but it might change

Anonymous Piwik user

I'm interessed too by this feature. I have activate it on my website, but the presentation in the server side, isn't perfect (the ping is show as a new page visit).

I hope, that it can be made for 1.7.

Thanks.

Matthieu Aubry
Owner

Another bug report: http://forum.piwik.org/read.php?2,83913,page=1#msg-84060

NOTE: This feature does not work currently

Matthieu Aubry
Owner

Proof Reading the upcoming book about Piwik, it's really good.

it gives me an idea related to this feature:

  • we should add a new parameter at the trackPageView function level, that users could use to trigger events/pageviews that will allow to disable the default "lower bounce rate". When "pinging" the page, we don't want to lower bounce rate by default, other users might want to do this as well in some cases.

We could allow for this other addon: send a fake Pageview 15seconds after arrival: http://analytics.blogspot.com/2012/07/tracking-adjusted-bounce-rate-in-google.html - might require the ping query so it doesn't inflate Page view counts, but changes Bounce rates for all other metrics.

Matthieu Aubry
Owner

Note from Anthon:

The original thinking was, fire a ping once the page has been open some minimum amount of time, and keep pinging as long as there's some mouse activity. Basically, the goal would be to report lower bounce rates (ie if the first and only page visited) and more accurate time on site for the last page visited on a site.

The Page Visibility and traceCallback changes means the ping isn't sent until a tab gains focus and has been open the minimum amount of time. Further pings require some activity while the tab is open.

We probably need a way to restart the heartbeat if the tab regains visibility. Also, the inactivity logic doesn't work with passive activities (eg streaming media), but I think could be addressed by adding an API for players. (Are you using event tracking for this?)

This obviously isn't feature complete, so if anyone has ideas on how this should work or could work better, feel free to share your thoughts.

Matthieu Aubry
Owner

I'm very interested in this feature but unfortunately time is missing to implement it for now, unless someone jumps in and produces a patch we can use! Or if you can sponsor the feature please get in touch at sponsors at piwik.org

Kevin Locke

Great feature, looking forward to it. (Just commenting to add myself to the CC list.)

Matthieu Aubry
Owner

See also: #5208 Detect when a user is actively browsing or using the webpage, for more accurate Time on page

johsin18

I would really love to have this feature. There are more and more single-page applications nowadays, aren't there? For them this feature is kind of mandatory.

RazvanMihaiu

Is this feature done? It's almost 4 years old now.

Thank you

Fredrik Ekelund

Just wanted to +1 this, would be a really great feature for using Piwik with SPA's

Jan Grewe

another +1 here ;-)

ru108

Try Using the Page Visibility API?

Feature Chrome (Webkit) Firefox (Gecko) Internet Explorer Opera Safari (WebKit)
Basic support 13 -webkit 10 (10) -moz 10 12.10[*] 7
33 18 (18)

http://caniuse.com/#feat=pagevisibility

Ismay

+1

IPL01

Has this feature been implemented now?

I get a lot of single page views and they all report back as 0s and 100% bounce rates! Not good.

I am not a coder so suggesting doing something myself with the code or using an API isn't really helpful. Is there any chance something can be implemented in an upcoming update?

Many thanks.

Matthieu Aubry mattab modified the milestone: Short term, Mid term
Glisse1

Yay, short-term milestone! +1 :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.