Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Dashboard for multiple sites (potentially hundreds of websites) #389

Closed
mattab opened this Issue · 35 comments

6 participants

@mattab
Owner

Some Piwik users add a lot of websites in Piwik. It is also common that users of a given Piwik installation can access more than 10 websites. Currently they have to go to each of these websites to check their reports.

They need quick & easy way to get an overview of what’s happening in all their websites at once, and a quick access to more details.

Mockup

[[Image(http://issues.piwik.org/attachments/389/multi%20site%20report.PNG?format=raw)]]

Overview

  • This page is space efficient but clear to display data about several websites
  • gives an overview of what happened on all your websites, so you can choose which ones need investigation.
  • display raw numbers
  • and show the evolution of the metrics over the last days/weeks/month. via sparkline + red/green indicator
  • this page links to each website report by a click on the website name
  • possible to change the period of the dashboard: Day/Week/Month/Year
  • each website is a simple row on the page
  • for each website, print the
  • website name
  • nb of unique visitors
  • nb of pages
  • (keep space for nb of goal conversions – coming)
  • a sparklines of the last 30 days/week/month/year for unique visitors
  • when hovering the website name, the “drag icon” appears on the left of the website name. This icon makes draggable the list of websites so that each user can reorder their list of websites. By default website are ordered alphabetically. The website order in the list are saved for each user in the database (like the dashboard is), except for anonymous for whom the order is only stored in session (alternatively the feature can be disabled for anonymous user). The drag n drop feature would use Jquery UI draggable that we already use to reorder widgets in the dashboard.
  • This dashboard can be accessed by a new entry “All websites” that always appears on the top of the “Website selector” on the top right of the page.

Speed When loading 300 websites, we will lookup numerical values, for the period selected.

To show UU + visits + pages + conversions we need to load the 4 related numeric values for 300 websites. that’s 1 SELECT query to return 1200 values.

to show green/red evolution indicator we need to select the compared numeric value from yesterday/last week/last month/last year. that’s one SELECT to return 300 values.

loading 300 sparklines will take a loong time, as the browser will issue limited http request on the piwik domain. loading 300 sparklines will certainly take more than 20 seconds. is that an issue?

Future releases

  • adding sorting feature so that user can sort table by unique visitors, conversions, etc.
  • adding search feature so that user can see only websites with names matching a string. the search box would have auto suggest feature.
  • add possibility to choose the default page to arrive on – either the dashboard with all websites or the first available website in the list

Estimate

between 4 and 6 days of work Keywords: multi-sites

@mattab
Owner

Attachment: [multi site report.PNG](http://issues.piwik.org/attachments/389/multi_site_report.PNG)

@zawadzinski
Collaborator

Current version in the attachment.

Now working on core to optimize Archive/Array/*.php

@anonymous-piwik-user

I’ve installed and activated this plugin but there is no new entry in the “Website selector” on the top right of the page. Only my sites are listed, but there still isn’t an extra one for All websites. I installed and activated it on two different servers, and neither shows the new selection in the drop-down.

@anonymous-piwik-user

Aha! I figured this one out. The documentation is incorrect. The plugin does work, just not as it states above. Rather than placing a new entry for All sites in the website selector in the top right of the page, it instead places an extra menu item in the main blue horizontal menu at the end after Dashboard, Visitors, Actions, Referrers. The new item is called Sites Summary.

@anonymous-piwik-user

I originally reported this problem on the forums because trac was down for me, but now I’ve come here. Here’s a summary of the bug below, but a larger version of the error is at the forum thread located at http://forum.piwik.org/index.php?showtopic=831. Neither this report nor the one at the forum includes the entire error because it is incredibly huge.

Since I upgraded to version 0.2.33, the multisites plugin has produced an error – the FIRST time I run it each day. After the first time, it’s fine (until the next day). Here’s an edited version of the error. The real version was ENORMOUS. I’ve also edited my domain info to keep it generic.

Notice: Undefined property: Piwik_DataTable_Simple::$rows in /home/myfolder/public_html/piwik/core/DataTable.php on line 457 Backtrace —>#0 Piwik_ErrorHandler(8, Undefined property: Piwik_DataTable_Simple::$rows, /home/myfolder/public_html/piwik/core/DataTable.php, 457, Array ()) called at [/home/myfolder/public_html/piwik/core/DataTable.php:457]…………

@mattab
Owner

see also: add a line with total for all websites: http://www.karlkopp.com/blog/2009/4/30/piwik-updated-multisitesplugin

@mattab
Owner

Please update to the new MultiSitesPlugin.tar for compatibility with 0.2.35.

@anonymous-piwik-user

This pops up with piwik 0.2.35 (using the updated version above):

Fatal error: Unsupported operand types in /var/www/apps/content/piwik/plugins/MultiSitesPlugin/Controller.php on line 204

@zawadzinski
Collaborator

Current version of MultiSitesPlugin. It still needs a lot of improvements and refactoring.

I am concerning rewriting the plugin to use directly piwik_archive_numeric_* tables in order to speed up the plugin (it is still very slow when even having "just" hundreds of sites).

If you got < 500 sites you can use this version (but remember to setup auto archiving http://piwik.org/docs/setup-auto-archiving/ - piwik won't be capable of archiving so many sites via web request!)

@zawadzinski
Collaborator

The Problem.
There is no way to use SQL query result limiting (LIMIT) and sorting (ORDER BY) using Piwik API. This is very important for MultiSitesPlugin, because when having thousands of sites we do not want to fetch all results, and sort & limit them in PHP.

Propsed solution.
We can build MultiSitesPlugin_API that will query directly SQL fetching data from piwik_archive_numeric_* tables. We do not want to check if data for specific site is archived, so we assume that all data is archived using cron auto-archiving (users will be noticed about it when installing MultiSitesPlugin).

This solution will significantly increase MultiSitesPlugin performance.

@zawadzinski
Collaborator

Attachment:
MultiSitesPlugin.zip

@mattab
Owner

good stuff!
here is my feedback:

about the UI

  • the sparklines html includes could be loaded directly, no need to show the rolling image (ie. we know how the call looks like); currently it does one http request to request each image HTML but jquery could write directly the right html.
  • the numbers and text could be centered / aligned middle in the cell, and cells could be less high
  • 1 - 12 z 12 is in polish?
  • the CSS could reuse the existing data table CSS for UI consistency

thanks! it is getting real close to be included in core ;)

PS: we can include in core before the optimization happen on the php side, so please feel free to do the UI updates and then work on the php side if you wish to push this in core.

@anonymous-piwik-user

Is this placement normal ?

[[Image(http://img229.imageshack.us/img229/4594/20062009083816.jpg)]]

If anyone know what to change in the main CSS file...

@zawadzinski
Collaborator
@zawadzinski
Collaborator

New version in the attachment.

We use now Piwik's CSS styles, and plugin Piwik coding standards are followed.

@anonymous-piwik-user

I'm sorry to say, the "button" in the menu is still "misplaced" :

(Firefox 3.0.11)

[[Image(http://img526.imageshack.us/img526/3176/09072009100750.jpg)]]

@zawadzinski
Collaborator

Its caused by translation (that is too long and does not fit in the button), it is not this plugin related issue.

@anonymous-piwik-user

Switching to english i still get this :

[[Image(http://img200.imageshack.us/img200/1931/10072009151203.jpg)]]

The submenu is always present no matter what language.

@mattab
Owner

the location in the menu is temporary and will be fixed before submitting the plugin to the core - most likely it will appear as "All Websites" next to Dashboard, top left of screen.

@robocoder
Collaborator
  • For consistency, I'd suggest dropping the "Plugin" suffix from the plugin name.
  • Given the autoloader, the require_once's can be removed from Controller.php and MultiSitesPlugin.php.
@anonymous-piwik-user

A few issues I have on this plugin. Firstly, it forces the php process to use 100% of the CPU for a long period of time (around 60 seconds - which would match the max process time in php.ini). It also throws a javascript error:

Webpage error details

User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 4.0.20506; Tablet PC 2.0;)
Timestamp: Wed, 15 Jul 2009 00:33:50 UTC

Message: Unknown runtime error
Line: 124348672
Char: 3
Code: 0
URI: http://example.com/piwik/index.php?module=CoreHome&action=index&idSite=2&period=day&date=2009-07-15

Message: Unknown runtime error
Line: 124348672
Char: 3
Code: 0
URI: http://example.com/piwik/index.php?module=CoreHome&action=index&idSite=2&period=day&date=2009-07-15

@anonymous-piwik-user

this plugin can't work properly on my IE browser(can't list any website),on chrome and firefox it works well,is it a bug or I've done something wrong?

@mattab
Owner

Once the few bugs listed here are fixed, we would like to commit it to trunk asap after Piwik 0.6. Maciej, would you mind taking a look at the reported issues? thanks!

@pebosi

Attachment:
MultiSites.zip

@pebosi

removed the suffix and the require_once calls. added a link to topbar (see included patch).

changed some stuff in index.tpl, but i was just testing, so there's a problem when changing date or period.

regards pebosi

@mattab
Owner

Pebosi, can you please clarify what the issues are? What is missing for the plugin to be included in 0.5.1 in a stable state?
Thank you!

@pebosi

When changing the date the parameter "date" in url is not replace but it's appended to the url without an ampersand. so there's an error...
i tried using no ajax so i think there's a problem with the broadcast.

regards

@pebosi

Attachment:
MultiSites.2.zip

@robocoder
Collaborator

In [1678], refs #389, commit MultiSites plugin

@pebosi

Hi,

bugs in the last commit [1678]:

  • Site selector is not hidden
  • date change is not affected

regards pebosi

@mattab
Owner

(In [1692]) refs #389 Fixing date selection, fixing column sorting, few styles modifications

@mattab
Owner

(In [1696]) MultiSites plugin enabled by default refs #389

@mattab
Owner

(In [1701]) refs #389 using the jquery magic so that IE works OK

@mattab
Owner

Included and enabled by default in 0.5.3

Next steps are performance improvements, see #1077

@JulienMoumne
Collaborator

(In [6478]) fixes #2708

refs #5491

  • PDFReports major refactoring. Any plugin can now add new kinds of reports. Required for #2708 and #3118.
  • test report functionality ($idReport == 0) dropped in Piwik_PDFReports_API->generateReport()
  • All Websites report shows 3 more metrics: Goal Conversions, eCommerce Conversions and eCommerce Revenue. Can be removed if asked to.
  • Piwik_PDFReports_API->sendEmailReport renamed to sendReport
  • All Piwik_PDFReports_API method signatures updated to support generic report parameters refs #389
  • new API method to retrieve only one Piwik site : Piwik_MultiSites_API->getOne()
  • per #2708 description, Piwik_MultiSites_API methods now support a new parameter named enhanced. When activated, Goal Conversions, eCommerce Conversions and eCommerce Revenue along with their evolution will be included in the API output.
  • API metrics refactored in (@ignored)Piwik_MultiSites_API->getApiMetrics()
  • Metadata now returns 12 metrics : nb_visits, visits_evolution, nb_actions, actions_evolution, revenu, revenue_evolution, nb_conversions, nb_conversions_evolution, orders, orders_evolution, ecommerce_revenue, ecommerce_revenue_evolution refs #3118
  • ReportPublisher plugin could now easily be implemented commits merged
  • r6243
  • r6422 (#3012) TODO
  • the MobileMessaging settings page may need some embellishment
  • @review annotations need some attention
  • test if the MultiSites API evolutions have some impact on Piwik Mobile and other client code
@mattab mattab added this to the Piwik 0.5.2 milestone
@mattab mattab self-assigned this
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.