New Plugin: Demography reporting

[sgiehl]

I've developed a plugin (based on current trunk [2319]), which allows to track additional information about community users. It generates reports about:

• usertype (user/guest) including evolution
• users age-group
• evolution of average age
• gender (male, female, unknown) including evolution
• user activity (list of the most active user)

It uses the setCustomData of the JS Tracker and takes currently three parameters:

• userid - Unique identifier of an user (might be id or nickname) up to 100 chars
• gender - Gender of user (might be 'F', 'M' or empty for unknown)
• birthdate - Birthdate of user (in format YYYY-MM-DD)

Tracking the users information would work like this:

<script type="text/javascript">
try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", {$IDSITE});
var user = { 
      'userid' : 'ID-or-Nickname',
      'gender' : 'F',
      'birthdate': '1985-05-13'
};
piwikTracker.setCustomData( user );
piwikTracker.trackPageView();
} catch( err ) {}
</script>

If one of those parameters is tracked it will be saved or even updated in an later request within the same visit. For that the plugin extends the 'log_visit'-table within installation. Its adds three columns at the end of the table.

The plugin uses the translations and is available in english and german.

I'll attach some screenshots and the sourcecode.

Feel free to test, review and comment.
Suggestions for improvements or additional features are welcome ;)

Keywords: plugin, community

[sgiehl]

Attachment: plugin source code
Community.zip

[sgiehl]

Attachment: screenshot of age report
age.jpg

[sgiehl]

Attachment: screenshot of age report
age2.jpg

[sgiehl]

Attachment: screenshot of gender report
gender.jpg

[sgiehl]

Attachment: screenshot of gender report
gender2.jpg

[sgiehl]

Attachment: screenshot of useractivity report
useractivity.jpg

[sgiehl]

Attachment: screenshot of usertype report
usertype.jpg

[mattab]

This looks very interesting! This is a very interesting third party plugin.

The code is great, I have nothing to say (which is rare ;-))

One performance improvement could be to have the UPDATE done by the main piwik UPDATE (on log_visit table). If you want to submit a patch to have a hookable UPDATE please do so, I'll commit it to trunk.

If you are interested in contribute more to Piwik, you are very welcome! Great developers are not easy to find ;)

[sgiehl]

Thanks ;-)
If you want you can also integrate the plugin to core.

I will have a look at the mentioned improvement.

[sgiehl]

I created a patch to make update hookable (see #1440). Already changed my plugin. Will upload changes here as soon as the patch is commited to trunk.

[mattab]

Patch #1440 is now committed to trunk.

[sgiehl]

Just added the new version. It now uses the hookable update and includes some small fixes.
Known "Bug":

• Displaying the average age in a period other than day doesn't work.
  I'm looking for a way to calculate the average value for a given period. There isn't such a feature in the archive processing at the moment.

[sgiehl]

Attachment: new version using the hookable update function
Community-0.2.zip

[anonymous-matomo-user]

Vers. Piwik 6.4 / Community 0.2

Ich habe einige Websites und nur 1x ist der Code fr das Plugin im Trackingcode enthalten.
Wenn das Plugin aktiv ist wird nur noch die Seite in Piwik ausgewertet
die den var(user) code enthlt.
Gewertet wird aber auch erst ab dem Moment in dem dort in den enthaltenen Variablen
Daten drin stehen.
Ist das ein Fehler bei mir, oder allgemein so?

I have some websides and only 1x the code is included for the Plugin in the Trackingcode.
If the Plugin is active only the side is evaluated in Piwik
var (user) code includes.
But it is evaluated only from the moment in the contained variable
Data are in it.
Is this normaly?

Normaly Trackingcode

try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 1);
piwikTracker.setDocumentTitle(document.title.substring(0,document.title.length-88));
piwikTracker.trackPageView();
piwikTracker.enableLinkTracking();
} catch( err ) 


Community Trackingcode

try {
var piwikTracker = Piwik.getTracker(pkBaseURL + "piwik.php", 10);
piwikTracker.setDocumentTitle(document.title.substring(0,document.title.length-69));
var user = {  'userid' : '{$this->user->username}',  'gender' : ge,  'birthdate': '{$this->user->userOption12}' };
piwikTracker.setCustomData( user );
piwikTracker.trackPageView();
piwikTracker.enableLinkTracking();
} catch( err ) { }

[sgiehl]

Dass die Community Daten nur ab dem Zeitpunkt ausgewertet werden knnen, ab dem der entsprechende Trackingcode eingebunden ist, ist nicht anders mglich, da zuvor ja auch keine Daten dafr gespeichert werden konnten. Was genau meinst du damit dass die Seiten auf denen noch der normale Trackingcode eingebunden ist nicht mehr ausgewertet werden? Zeigt nur das Community-Plugin keine Daten an, oder funktioniert die komplette Auswertung aller Daten bei dir nicht mehr?

It is only possible to display values after the extended tracking code has been included. Before no additional data has been saved. What do you mean with only the site where the extended tracking code is included evaluates? Does only the community plugin show no data for the rest, or is even the hole tracking not working?

[anonymous-matomo-user]

Entschuldigung, war nicht ganz klar ausgedrckt
Sobald das Plugin aktiv ist werden die anderen
Seiten von PIWIK nicht mehr ausgewertet
Das das Plugin erst auswerten kann sobald Daten
vorhanden sind ist klar

Sorry, was not expressed clearly
As soon as the Plugin is active the others
Sides of PIWIK no more evaluated
The Plugin can only evaluate as soon as data are available is clear

[sgiehl]

Do you have access to the error logs. Are there any errors occuring?

[anonymous-matomo-user]

There are no entries in the errorlog

[sgiehl]

hm... are the visitors still tracked? eg. are there new entries generated in the vistor_log table?

[anonymous-matomo-user]

No, I have in log_visit no new entry with aktiv Community Plugin.
Because my Goals don't work any more since 6.4 it can be also a Piwikproblem what can be repaired in 6.5.
(With aktiv Goals the Rest of Piwik is working)

Unfortunately, I have installed your Plugin only with 6.4 and have no possibilities of comparison

I forgot to tell you: Great Plugin

[sgiehl]

Attachment: v 0.3 - fixed problem with averge age
Community-0.3.zip

[anonymous-matomo-user]

Now the Plugin works (Piwik 0.9 / 0.3)

At first I had a installerror, DB-Fields without Null by userid and gender, and than I found
as soon as I have an entry in birtdate Piwik stops Tracking.
I have in my Community Mysqlfields with char(10) and date, both with yyyy-mm-dd at least
I tested your example code.
Do you have an idea what I'm doing wrong?

P.S.
Could it bee that Action and Max Action has twisted data?

[anonymous-matomo-user]

Now I have tested the Plugin with new installed older Piwikversions (without other Third Party Piwik Plugins).
With version 6.4 the Plugin working fine and from version 7 the mistake appears.

[sgiehl]

Attachment: v 0.3 - fixed fatal occuring since 0.6.4; implemented getReportMetadata to make reports available for pdf plugin
Community-0.4.zip

[sgiehl]

@bastelix: would you please try this new version, it should fix the problems.

[anonymous-matomo-user]

I have a problem...

If I have xampp unsecure, this plugin works perfectly, but.... If I have xampp secure all tracking doesnt work... :(

Anyone can help me????

[robocoder]

What is "xampp secure"?

[sgiehl]

Do you mean xampp running on https?

Well, could you please give some detailed information?
Which operating system are you running? And which version of xampp and piwik do you use?
Are any errors occuring in the error log?

[sgiehl]

Plugin might not work with newer versions of piwik. As custom variables are now implemented in piwik, plugin is no longer needed...

[anonymous-matomo-user]

I have created workround in my piwik server , so we can "support" the Community plugin graphic Api :

1. Add the "old" 3 fields (userid,gender,birthdate) to the piwik_log_visit table , by installing the community plugin .

2. 3 custom variables were used to store the value of this plugin :

custom_var_k1 ='userid'
custom_var_k2='gender'
custom_var_k3='birthdate'

request example will look something like that :
http://my_server/piwik/piwik.php?idsite=1&rec=1&_cvar={"1":["userid","myuser"],"2":["gender","F"],"3":["birthdate","2000-01-25"]}

3)To display the graphic presentation of the previous plugin (use the community folder) , i'm copying the date into the "OLD" data columns , using the following trigger :

create trigger Community_trigger
before insert on piwik_log_visit
for each row
begin
IF NEW.custom_var_k1='userid' THEN SET NEW.userid = NEW.custom_var_v1 ;
END IF;
IF NEW.custom_var_k2='gender' THEN SET NEW.gender = NEW.custom_var_v2 ;
END IF;
IF NEW.custom_var_k3='birthdate' THEN SET NEW.birthdate = NEW.custom_var_v3 ;
END IF;
end$$

So , we can use the grapgic API as before .