Item12552: added tracking of outlinks and downloads

git-svn-id: http://svn.foswiki.org/trunk/PiwikPlugin@16844 0b4bb1d4-4e5a-0410-9cc4-b2b747904278
foswiki · Jul 18, 2013 · e4930e5 · e4930e5
1 parent f9dea39
commit e4930e5
Show file tree

Hide file tree

Showing 7 changed files with 207 additions and 83 deletions.
diff --git a/data/System/PiwikPlugin.txt b/data/System/PiwikPlugin.txt
@@ -4,30 +4,43 @@
 
 %TOC%
 
-<img src="%ATTACHURLPATH%/logo.png" align="right" />
+<img src="%ATTACHURLPATH%/logo.png" class="foswikiRight" />
 
-This plugin lets you track page view using the [[http://piwik.org][Piwik web analytics]] services.
+This plugin lets you track page views using the [[http://piwik.org][Piwik web analytics]] services.
 
 Piwik is the leading self-Hosted, decentralized, Open Source web Analytics
 Platform, used by 460,000 websites in 150 countries. 
 
-In most cases the !JavaScript API is used on the _client side_ to bring Piwik-based web analytics to your site. However browsers tend to
-block these additional third party cookies and services which then results in rather sparse data being collected.
-
-Alternatively <nop>%TOPIC% uses the REST api of Piwik to track page views on the _server side_. That is, your Foswiki server talks
-to a Piwik server informing it about things going on behind the scene. 
-
-There are a couple of advantages and disadvantages comparing client side vs server side page tracking:
-
-   * measurements on the server are more reliable as they don't rely on the client to cooperate with regards to <nop>JavaScript and cookie settings
-   * installing a Foswiki plugin is more standardized compared to injecting additional <nop>JavaScript to your site
-   * page rendering times are captured on the server and propagated to te Piwik analyzer
-   * custom site search tools like [[Foswiki:Extensions/SolrPlugin][SolrPlugin]] can hook into the tracking process on the server in a reliable way
-   * the Piwik API doesn't have to be exposed to the outside which would normally be the case for the browser to be able to contact it
-   * some measures like screen size and browser plugins can't be recorded on the server 
-   * click overlays don't work without piwik-related <nop>JavaScript being added to the site
-
-In short: the bulk of interesting data can be captured server side in a more reliable way while some minor features of Piwik aren't covered yet by data produced by <nop>%TOPIC%.
+In most cases the !JavaScript API is used on the _client side_ to bring
+Piwik-based web analytics to your site. However browsers tend to block these
+additional third party cookies and services which then results in rather sparse
+data being collected.
+
+Alternatively <nop>%TOPIC% uses the REST api of Piwik to track page views on
+the _server side_. That is, your Foswiki server talks to a Piwik server
+informing it about things going on behind the scene. 
+
+There are a couple of advantages and disadvantages comparing client side vs
+server side page tracking:
+
+   * measurements on the server are more reliable as they don't rely on the
+     client to cooperate with regards to <nop>JavaScript and cookie settings
+   * installing a Foswiki plugin is more standardized compared to injecting
+     additional <nop>JavaScript to your site
+   * page rendering times are captured on the server and propagated to te Piwik
+     analyzer
+   * custom site search tools like [[Foswiki:Extensions/SolrPlugin][SolrPlugin]] 
+     can hook into the tracking process on the server in a reliable way
+   * the Piwik API doesn't have to be exposed to the outside which would
+     normally be the case for the browser to be able to contact it
+   * some measures like screen size and browser plugins can't be recorded on
+     the server 
+   * click overlays don't work without piwik-related <nop>JavaScript being
+     added to the site
+
+In short: the bulk of interesting data can be captured server side in a more
+reliable way while some minor features of Piwik aren't covered yet by data
+produced by <nop>%TOPIC%.
 
 ---++ Installation Instructions
 
@@ -37,18 +50,22 @@ To finish installation, you'll have to configure at least:
 
    * the ={ApiUrl}= of your Piwik server, e.g. =http://localhost/piwik/piwik.pgp=
    * The ={TokenAuth}= to access advanced features (see the API menu on your Piwik server).
-   * the ={SiteId}= of your Foswiki server; this is a numeric id of your sites configured in piwik itself
+   * the ={SiteId}= of your Foswiki server; this is a numeric id of your sites
+     configured in piwik itself
    * the ={QueueDir}= to spool tracked page views while being recorded by Foswiki
 
-Note that you must use different =<nop>SiteIds= in each =VirtualHost.cfg= file in case you are using [[Foswiki:Extensions/VirtualHostingContrib]], e.g.
+Note that you must use different =<nop>SiteIds= in each =VirtualHost.cfg= file
+in case you are using [[Foswiki:Extensions/VirtualHostingContrib]], e.g.
 
 <verbatim>
 $VirtualHost{PiwikPlugin}{SiteId} = 1;
 </verbatim>
 
-Page impressions aren't recored to the Piwik server directly for performance reasons. Instead, they are spooled in a queue specified by the ={QueueDir}= 
-configuration setting. Foswiki itself will write to that directory while the interim =piwik_daemon= has to be started which takes over responsibility forwarding
-the records to the actual Piwik server.
+Page impressions aren't recored to the Piwik server directly for performance
+reasons. Instead, they are spooled in a queue specified by the ={QueueDir}=
+configuration setting. Foswiki itself will write to that directory while the
+interim =piwik_daemon= has to be started which takes over responsibility
+forwarding the records to the actual Piwik server.
 
 ---++ Info
 <!--
@@ -61,7 +78,8 @@ the records to the actual Piwik server.
 |  Version: | %$VERSION% |
 |  Release: | %$RELEASE% |
 |  Change History: | <!-- versions below in reverse order -->&nbsp; |
-|  16 Jul 2013 | implemented auto-start feature for the piwik comm server |
+|  18 Jul 2013 | added tracking of outlinks and downloads |
+|  16 Jul 2013 | added auto-start feature for the piwik comm server |
 |  15 Jul 2013 | implemented queue manager for better tracking performance; improved control over pages and actions being tracked |
 |  14 Jul 2013 | initial release |
 |  Dependencies: | %$DEPENDENCIES% |

diff --git a/lib/Foswiki/Plugins/PiwikPlugin.pm b/lib/Foswiki/Plugins/PiwikPlugin.pm
@@ -19,10 +19,11 @@ use strict;
 use warnings;
 
 use Foswiki::Func ();
+use Foswiki::Sandbox ();
 use Error qw(:try);
 
-use version; our $VERSION = version->declare("v1.99.3");
-our $RELEASE = '15 Jul 2013';
+use version; our $VERSION = version->declare("v1.99.4");
+our $RELEASE = '18 Jul 2013';
 our $SHORTDESCRIPTION = 'Server-side page tracking using Piwik';
 our $NO_PREFS_IN_TOPIC = 1;
 our $tracker;
@@ -40,66 +41,20 @@ sub tracker {
 sub initPlugin {
 
   tracker->init;
+  startDaemon() if $Foswiki::cfg{PiwikPlugin}{AutoStartDaemon};
+  addToHead() if $Foswiki::cfg{PiwikPlugin}{TrackOutlinks};
 
-  if ($Foswiki::cfg{PiwikPlugin}{AutoStartDaemon}) {
-    require Foswiki::Sandbox;
-
-    my $request = Foswiki::Func::getRequestObject();
-    my $refresh = $request->param('refresh');
-    $refresh = (defined($refresh) && $refresh =~ /^(on|piwik)$/) ? 1 : 0;
-
-    my $pidFile = $Foswiki::cfg{PiwikPlugin}{PidFile};
-    my $logFile = $Foswiki::cfg{PiwikPlugin}{LogFile};
-
-    if ($pidFile && $logFile) {
-
-      if ($Foswiki::cfg{PiwikPlugin}{Debug}) {
-        print STDERR "PiwikPlugin - pidFile=$pidFile\n";
-        print STDERR "PiwikPlugin - logFile=$logFile\n";
-      }
-
-      my $pid;
-
-      $pid = Foswiki::Sandbox::untaint(
-        Foswiki::Func::readFile($pidFile),
-        sub {
-          my $pid = shift;
-          if ($pid =~ /^\s*(\d+)\s*$/) {
-            return $1;
-          }
-        }
-      ) unless $refresh;
-
-      if ($pid && kill 0, $pid) {
-
-        print STDERR "PiwikPlugin - piwik_daemon already running at $pid\n"
-          if $Foswiki::cfg{PiwikPlugin}{Debug};
-
-      } else {
-
-        my $command = $Foswiki::cfg{PiwikPlugin}{DaemonCmd};
-        $command .= " -restart" if $refresh;
-
-        my ($stdout, $exit) = Foswiki::Sandbox->sysCommand(
-          $command,
-          PIDFILE => $pidFile,
-          LOGFILE => $logFile,
-        );
-
-        print STDERR "PiwikPlugin - started piwik_daemon.\n"
-          if $Foswiki::cfg{PiwikPlugin}{Debug};
-
-        print STDERR "PiwikPlugin - stdout: $stdout\n" if $stdout;
-      }
-
-    } else {
-      print STDERR "PiwikPlugin - Can't auto-start piwik_daemin: no {PidFile} or {LogFile}\n";
-    }
-  }
+  Foswiki::Func::registerRESTHandler('doTrackAction', sub { return tracker->restTrackAction(@_); });
 
   return 1;
 }
 
+sub addToHead {
+  Foswiki::Func::addToZone("script", "PIWIKPLUGIN", <<EOS, "JQUERYPLUGIN");
+<script src='%PUBURLPATH%/%SYSTEMWEB%/PiwikPlugin/jquery.piwik.js'></script>
+EOS
+}
+
 sub completePageHandler {
 
   return unless tracker->isEnabled;
@@ -125,4 +80,61 @@ sub completePageHandler {
   };
 }
 
+################################################################################
+sub startDaemon {
+
+  my $request = Foswiki::Func::getRequestObject();
+  my $refresh = $request->param('refresh');
+  $refresh = (defined($refresh) && $refresh =~ /^(on|piwik)$/) ? 1 : 0;
+
+  my $pidFile = $Foswiki::cfg{PiwikPlugin}{PidFile};
+  my $logFile = $Foswiki::cfg{PiwikPlugin}{LogFile};
+
+  if ($pidFile && $logFile) {
+
+    if ($Foswiki::cfg{PiwikPlugin}{Debug}) {
+      print STDERR "PiwikPlugin - pidFile=$pidFile\n";
+      print STDERR "PiwikPlugin - logFile=$logFile\n";
+    }
+
+    my $pid;
+
+    $pid = Foswiki::Sandbox::untaint(
+      Foswiki::Func::readFile($pidFile),
+      sub {
+        my $pid = shift;
+        if ($pid =~ /^\s*(\d+)\s*$/) {
+          return $1;
+        }
+      }
+    ) unless $refresh;
+
+    if ($pid && kill 0, $pid) {
+
+      print STDERR "PiwikPlugin - piwik_daemon already running at $pid\n"
+        if $Foswiki::cfg{PiwikPlugin}{Debug};
+
+    } else {
+
+      my $command = $Foswiki::cfg{PiwikPlugin}{DaemonCmd};
+      $command .= " -restart" if $refresh;
+
+      my ($stdout, $exit) = Foswiki::Sandbox->sysCommand(
+        $command,
+        PIDFILE => $pidFile,
+        LOGFILE => $logFile,
+      );
+
+      print STDERR "PiwikPlugin - started piwik_daemon.\n"
+        if $Foswiki::cfg{PiwikPlugin}{Debug};
+
+      print STDERR "PiwikPlugin - stdout: $stdout\n" if $stdout;
+    }
+
+  } else {
+    print STDERR "PiwikPlugin - Can't auto-start piwik_daemin: no {PidFile} or {LogFile}\n";
+  }
+}
+
+
 1;
diff --git a/lib/Foswiki/Plugins/PiwikPlugin/Config.spec b/lib/Foswiki/Plugins/PiwikPlugin/Config.spec
@@ -56,6 +56,10 @@ $Foswiki::cfg{PiwikPlugin}{TrackedActions} = 'edit,view,save';
 # Regular expression matched against the web.topic being tracked. A matching topic won't be recorded to Piwik.
 $Foswiki::cfg{PiwikPlugin}{ExcludePattern} = '';
 
+# **BOOLEAN**
+# Boolean flag to trigger tracking of outgoing links and downloads
+$Foswiki::cfg{PiwikPlugin}{TrackOutlinks} = 1;
+
 # **BOOLEAN**
 # Boolean flag to trigger aut-starting the piwik communication daemon.
 $Foswiki::cfg{PiwikPlugin}{AutoStartDaemon} = 1;

diff --git a/lib/Foswiki/Plugins/PiwikPlugin/Daemon.pm b/lib/Foswiki/Plugins/PiwikPlugin/Daemon.pm
@@ -44,7 +44,6 @@ sub new {
     $this->writeDebug("using queueDir $this->{queueDir}");
   }
 
-
   return $this;
 }
 

diff --git a/lib/Foswiki/Plugins/PiwikPlugin/Tracker.pm b/lib/Foswiki/Plugins/PiwikPlugin/Tracker.pm
@@ -117,6 +117,32 @@ sub isEnabled {
   return 1;
 }
 
+################################################################################
+sub restTrackAction {
+  my ($this, $session, $subject, $verb, $response) = @_;
+
+  my $request = Foswiki::Func::getRequestObject;
+
+  my $action = $request->param("action");
+  die "action parameter missing" unless defined $action;
+
+  my $url = $request->param("url");
+  die "url parameter missing" unless defined $url;
+
+  return $this->doTrackAction($url, $action);
+}
+
+################################################################################
+sub doTrackAction {
+  my ($this, $url, $action) = @_;
+
+  die "unknown action '$action'" unless $action =~ /^(download|link)$/;
+
+  writeDebug("doTrackAction($url, $action)");
+
+  return $this->queueRecord($this->createActionRecord($url, $action));
+}
+
 ################################################################################
 sub doTrackPageView {
   my ($this, $web, $topic) = @_;
@@ -157,6 +183,17 @@ sub createSiteSearchRecord {
   return $record;
 }
 
+################################################################################
+sub createActionRecord {
+  my ($this, $url, $action) = @_;
+
+  my $record = $this->createTrackerRecord;
+  $record->{$action} = $url;
+  $record->{url} = $url;
+
+  return $record;
+}
+
 ################################################################################
 sub createPageViewRecord {
   my ($this, $pageTitle) = @_;

diff --git a/pub/System/PiwikPlugin/Makefile b/pub/System/PiwikPlugin/Makefile
@@ -0,0 +1,5 @@
+FOSWIKI_ROOT?=~/foswiki/trunk/core
+TARGET=jquery.piwik.js 
+
+-include $(FOSWIKI_ROOT)/pub/System/JQueryPlugin/Makefile.include
+
diff --git a/pub/System/PiwikPlugin/jquery.piwik.uncompressed.js b/pub/System/PiwikPlugin/jquery.piwik.uncompressed.js
@@ -0,0 +1,49 @@
+jQuery(function($) {
+  var baseUrl = location.protocol + "//" + location.hostname + (location.port && ":" + location.port),
+      pubUrl = foswiki.getPreference("PUBURL"),
+      pubUrlPath = foswiki.getPreference("PUBURLPATH"),
+      trackActionUrl = foswiki.getPreference("SCRIPTURL")+"/rest/PiwikPlugin/doTrackAction";
+
+  $(document).on("click", "a", function() {
+    var $this = $(this),
+        href = $this.attr("href");
+
+    if (typeof(href) !== 'undefined') {
+
+      // is it an outgoing link?
+      if (href.indexOf("http") == 0 && href.indexOf(baseUrl) != 0) {
+
+        //console.log("external url clicked",href);
+
+        // record it
+        $.ajax({
+          url: trackActionUrl, 
+          async: false,
+          data: {
+            "action": "link",
+            "url": href
+          }
+        }); 
+      }
+
+      // is it a link to an attachment?
+      if (href.indexOf(pubUrl) == 0 || href.indexOf(pubUrlPath) == 0) {
+
+        if (href.indexOf("/") == 0) {
+          href = baseUrl+href;
+        }
+        //console.log("attachment clicked", href);
+
+        // record it
+        $.ajax({
+          url: trackActionUrl, 
+          async: false,
+          data: {
+            "action": "download",
+            "url": href
+          }
+        }); 
+      }
+    }
+  });
+});