Page Overlay - Analytics Overlay on your site! (Aka in-site analytics)

[mattab]

In order to show analytics data as an overlay, we have to set up a cross-domain communication protocol. The aim of this ticket is to complete a simple version of the feature that provides extensible mechanisms for the communication. Based on this, we can add more things to it.

Page Overlay displays a block around each link showing the % of clicks on this link. the user can still browse the website. On the left a sidebar displays the key statistics for the currently viewed page. It also easily allows the user to open the Page Transitions report and the Row evolution historical report for this particular page.

See #3530 List of Page Overlay Improvements for more info and the current state of the art!

[mattab]

From EZDesign:

I did some research on in-page analytics. Sounds very interesting.

But as it turns out, manipulating the iframe is far from easy to do. I think, I figured out how Google does it and posted it as a reply to a stack overflow question:

http://stackoverflow.com/questions/5210897/how-does-google-analytics-in-page-analytics-work/

I think it's quite problematic, that Google injects something the user doesn't really want, which demonstrates what kind of power Google has. I think this is not really a problem in our case, because we don't inject it, the users do (from their servers which are under their control).

In general, is this feature something we want? It definately is a piece of work, but it's quite a fancy feature. Seeing how much people like to be close to their visitors (i.e. Live), this should be received well!

[mattab]

Timo, in page analytics would be a great feature to have for sure... especially since we can do it with the current data set and don't need to track more data.

The main limitation of this feature I think is that it doesn't differentiate few places linking to the same URL, so you don't know if users clicked mostly on top link or bottom link to "Contact us". But I guess we could somehow fix this issue later on, by storing some kind of unique ID for the DOM node being clicked, and add this field in the log_link_visit_action table ;) this would be hard code but I think possible without changing too many things. Anyway, not for V1!!

We probably could not pre-process the data, so it would be read from the logs directly (still fetched from an API function). Performance wise, we don't have an INDEX on the field idaction_url_ref so GROUP BY on this field will be pretty slow. We could add the INDEX, or maybe just "recommend" users to do it if they find the feature slow to use...? Adding the index for everybody sounds not ideal since it would have a non trivial performance/disk space overhead. But, it might work fine without it for most users anyway!

Also there will be some interesting DOM manipulation to detect all links to another URL (and normalizing URLs in the page transforming ../ etc. in the full URL).

Note that in Piwik we record the URL after removing some URL parameters. You can call getSiteFromId() and then look at excluded_parameters attribute to have the list of all excluded parameters. It is important to remove these from the links detected on the page to ensure you can match the same URLs. Also, common session parameters are excluded (see API SitesManager.getExcludedQueryParametersGlobal()) as well as Piwik campaign parameters (see campaign_var_name and campaign_keyword_var_name in global.ini.php).

Note that I have tried GA in page analytics on 3 different GA accounts but it didn't work in any of them... so I feel like the other guy in stackoverflow "The answer to 'how does it work' is 'it doesnt work'" ;)

Regarding how it works, I haven't looked into it but my first idea would not be to do iframe overlay, but to inject additional DOM elements that would show up on Hover and show the data for each link... Not sure if that would work or if iframe idea is better?

I think that, if we implement this feature, we'll need the "filter" dropdown as well, without it the page would look too busy and full of data maybe..

It would be great & useful & fun feature :)

PS: see also: https://www.google.com/search?q=webtrends+smartview

[timo-bes]

The current working title of the feature is "Insight" or "Page Insight". We recently started using the word "Insight" for intelligent alerts, which would probably be a better way to use it. So we might want to come up with a better name.

Others call the feature "in-site analytics" or "on-site analytics". One suggestion I have is "Piwik Overlay" - but in the menu, it would be shortened as "Overlay". Or maybe "Page Overlay"?

Every comment is appreciated.

[timo-bes]

(In [7223]) refs #2465: Starting the Insight plugin (in-site analytics)

This is the code I wrote about a year ago :)

There are several things that will be changed / removed but it's still good to have it in SVN - that's why I commit it like this.

[timo-bes]

(In [7224]) refs #2465: removed code for resolutions overlay - it turned out to be not that useful. maybe we can rethink it in the future.

[timo-bes]

(In [7225]) refs #2465 svn properties for new files

[timo-bes]

(In [7226]) refs #2465 removed alias domain normalization. it's now consistent with the behavior of transitions which is good for reusing the transitions report.

[timo-bes]

(In [7227]) refs #2465, refs #3332: using Transitions for getting the following pages in Insight

[timo-bes]

(In [7228]) refs #2465 Insight is starting to be cool

• highlight links on hover, displaying a border and the absolute number of clicks. also, the bubble (along with all other bubbles for links to the same url) is highlighted and the url is shown in the status (lower right corner)
• bubbles are updated when links are hidden or added. this is useful for drop-down navigations or ajax content.
• code formatter applied (some whitespace changes)

[mattab]

Just an idea while I think of it:

• it would be great to also show a link to open Transitions popover. I will review in more details later.
• also would be great to see a link to Row Evolution, as it shows historical evolution
  Maybe these 2 could be next to the 2 icons, written in grey, below the list of Metrics, in a new section "More insights on this Page:" ?

[timo-bes]

(In [7261]) refs #2465 improved efficiency of insight tracker extension.

you can review piwik.js now, i don't plan to change it any more.

[timo-bes]

When you open an insight session, you get JS errors like this (at least in Chrome, I haven't checked other browsers):

Unsafe JavaScript attempt to access frame with URL http://piwik.server/index.php?module=CoreHome&action=index&idSite=1&period=week&date=2012-06-24#module=Insight&action=index&idSite=1&period=week&date=2012-06-24 from frame with URL http://web.site/path/. Domains, protocols and ports must match.

It is thrown in piwik.js / getReferrer (line 640) because Piwik is checking the referrer of the parent document, which is the Piwik UI. The same origin policy doesn't allow that.

Who knows how we can avoid that?

[timo-bes]

(In [7262]) refs #2465 using piwikHelper.ajaxCall to build ajax call. it now works for date ranges as well.

[timo-bes]

(In [7263]) refs #2465

• FULL SCREEN MODE!
• smoother handling of the case when there are no pageviews for the current url
• hovering the bubbles highlights the links as well
• safer repositioning of bubbles (timeout instead of interval)
• adding breakpoints to the current url

[mattab]

(In [7306]) Refs #2465
Fixing JSLINT errors + packaging changes in piwik.js in the minified script

Note there are still 2 JSlint errors.

PS: Timo, I will carefully review your work on this ticket after 1.9.1 is out!