Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

New plugin: define Custom Alerts and receive email when triggered #1486

Closed
mattab opened this Issue · 52 comments

4 participants

Matthieu Aubry Fabian Becker Anonymous Piwik user Thomas Steur
Matthieu Aubry
Owner

UPDATE January 26th: Custom Alerts plugin is now available in the Marketplace! Test it by using the latest Piwik beta version, and then installing the Custom Alerts plugin in one click: go to Piwik, click Settings then click Marketplace and click Install button.


Create custom alerts:

  • define the alert name ("Big drop in purchases")
  • select a website (or list of websites) on which the Alert is defined
  • receive alert by email (email will contain alert description + link to Piwik dashboard URL for the given website ID and period).
  • select the Alert period: should it be daily, weekly or monthly?
  • select the report (Websites, Keywords, Countries, general stats)
  • define Metrics (visits, page view, avg. visit duration, Goal 1 conversions, total goal conversions, etc.)
  • define the Alert: when Visits decrease 50%, when purchases are more than 50 per day, etc.

See attached screenshot for preview.

Note: Autocomplete the dimensions will probably not be in the V1 of the script. Autocomplete would be a nice feature as it helps users not making mistakes (eg. is it Germany or DE or germany?).

This plugin requires each Piwik plugin to define their API metadata (list of reports callable, and list of metrics + translations for each metric). Covered in #1485

Matthieu Aubry
Owner

Feedback

  • This applies to: group reports by category within the SELECT
  • Pre-set useful alert: Alert me when Average generation time is more than 2 seconds
  • History of triggered alerts is long list. Maybe we can use AngularJs to display this list with paging?
Matthieu Aubry
Owner
Matthieu Aubry
Owner

Note: Custom Alert plugin UI html/js/css controller that allows to select one or several websites will be reused for sure in the PDF plugin.

It would be great if this bit of code was made reusable, thanks!

Matthieu Aubry
Owner

Thought dump about unit tests

// test alert on one website
test_oneWebsite_visitIncrease
// test alert setup on two websites
test_severalWebsite_visitIncrease
// test alert setup on two websites, but user doesn't have access to one of these anymore
test_severalWebsite_userDoesNotHaveViewAccessForOneWebsite

// test all metrics matches/contains/etc. conditions
test_metrics_matchesExpression
test_metrics_containsExpression_withChineseCharacters

// test different periods
// test all increase/decrease/below/above conditions
test_metrics_IncreasePercent_LastDay
test_metrics_increaseValue_lastWeek
test_metrics_specificGoalConversion_isMoreThan_lastMonth

// tests searched string matches several rows (ie. keywords matching 'test')
// test with bounce rate: bounce rate being a processed metric, all matching rows should be summed before processing the bounce rate
test_metrics_matchSeveralRows_bounceRate
// same test, but simply with Visits evolution
test_metrics_matchSeveralRows_visits

// + test Goal metrics for one website, General goal metric alert..

For each of these tests, or maybe once for all tests, you could generate tracking hits using the code from Main.test.php

For example you would generate 10 or 15 visits to try and match all your use cases, then you would run in one test method all sub tests. (otherwise, generating visits for each test case would make tests run very slow).

In UI, important to test alert name has ', " for sql, and check chinese characters.

Matthieu Aubry
Owner

(In [2585]) * Small modifications to UI

  • created a Filter/AddColumnsProcessedMetrics.php that can be reused in the Custom Alerts plugin refs #1486 - it processes the 3 default processed variables (actions per visit, avg time on site, bounce rate) for any datatable that has a 'dimension' in the returned metadata :)
Matthieu Aubry
Owner

(In [2594]) Refs #1485

  • Fixing wrong metadata for all plugins, cleaning up columns and ensure consistency in API (eg. VisitsSummary.get now returns conversion rate with %)
  • Adding a generic filter Truncate, updated doc at: http://dev.piwik.org/trac/wiki/API/Reference filter_truncate ; if set, will truncate the table after $filter_truncate rows. The last row will be named 'Others' (localized in the requested language) and the columns will be an aggregate of statistics of all truncated rows.
  • Adding common helpers to get a "pretty" value from a given metric and column name
  • Adding new time export format that will be used in PDF reports 12:34:56 format

Refs #1486

  • a new filter, AddColumnsProcessedMetricsGoal, can be reused to easily add processed goal metrics for a given datatable. For example: $dataTable->filter('AddColumnsProcessedMetricsGoal', array($enable=true, Piwik_DataTable_Filter_AddColumnsProcessedMetricsGoal::GOALS_FULL_TABLE)); will process all goal metrics (and per goal conv rate, rev per visit, conversions). When possible, only reports necessary should be loaded using GOALS_MINIMAL_REPORT or GOALS_OVERVIEW or a given idGoal
Matthieu Aubry
Owner

(In [2701]) Refs #1486
Patch by halfdan required by Alerts plugin

  • Invert pattern match
  • Date.addPeriod and Date.subPeriod
Fabian Becker
Collaborator

I'm uploading a current status here - not quite working, more for code review and feedback. I'm half-way through migrating to the new UI that was introduced with PDFReports.

Integration.test is testing addAlert/editAlert (success!) and processing of alerts (little buggy).

I'll be travelling for some days and will finish Alerts when I'm back.

Fabian Becker
Collaborator

Attachment: Current status - not for production
Alerts.zip

Matthieu Aubry
Owner

delaying for now, even though I hope it makes the cut :)

Matthieu Aubry
Owner

Halfdan, do you think the plugin can be put into trunk at some point? This would be a great feature and it is more than half done for sure :)

Matthieu Aubry
Owner

@halfdan Have you had a chance to make progress on the feature? Even if not complete we'd be very interested to see what you've done and if we can reuse it. Thanks!

Anonymous Piwik user

Hi, I would be very interested to help or to continue this project. @halfdan, it would be very nice if you could report what is working and what not, and which kind of problems you encountered.

Fabian Becker
Collaborator

Hi @Codalibre,
that is very kind of you. I am however still working on this plugin and hope to have it finished by the end of the year. I have not responded to matt's request since I am in direct contact with him.

I will let you know if I need any help.

Matthieu Aubry
Owner

If you are interested in this feature, please pledge now! We are running a campaign for Custom Alerts: http://crowdfunding.piwik.org/custom-alerts-plugin/

It will be awesome, but we need your support!

Anonymous Piwik user

I'd love to see another feature.
Piwik could become a helpful tool in order to be alerted when it comes to problems with the website.

There are a lot of things that can go wrong after updating your website. You have a lot of log files and tests but nevertheless sometimes there goes something wrong on the website (e.g. browser problems with js, layout problems, ...).

Sure, you could have an alert for higher bounce rates on critical pages (e.g. product pages).

It would be great if there were alerts that are a bit more complex.
My scenario:
Alert if the ratio of visitors that view a product page, add a product to cart and checkout decreases.
This means the checkout funnel should alert me if there is a problem in one of the steps.

Matthieu Aubry
Owner

@mohelten Piwik does not yet support conversion funnel, so this is not possible, but it's a good idea for the future.

Matthieu Aubry
Owner

I got a few emails in the last few weeks from funders asking about Custom Alerts.

We have some news: while unfortunately Fabian @halfdan is right now in hospital recovering from accident but will be out soon. AND he has a version of the plugin for beta which we'll release as part of Piwik 2.0 as a core plugin, as announced. Stay tuned & halfdan hope you get well!

Matthieu Aubry
Owner

@Fabian any update for our community of funders eagerly waiting to hear an update? hope you are well!

Matthieu Aubry
Owner

hey @Fabian any update?

Matthieu Aubry
Owner

No update yet, so likely won't make it to 2.0 (sorry for such long wait to all funders!)

Thomas Steur
Owner

In f9ff741: refs #1486 made previous alerts plugin compatible with Piwik 2.0 and added the possibility to actually list, create and edit alerts

Thomas Steur
Owner

In 1e6cece: refs #1486 code cleanup, implemented some comparisons, bugfixes

Thomas Steur
Owner

In 3bd9c0b: refs #1486 we do not want to have sql statements in controller, api and plugin class. Put them in a model, makes it also better testable

Thomas Steur
Owner

In 450e861: refs #1486 some more cleanup, bugfixes and moved defined metrics/group conditions to the processor as only this class can know which ones it supports

Thomas Steur
Owner

In 853b18c: refs #1486 more refactoring, bugfixing, display report name is list of all alerts

Matthieu Aubry
Owner

In 8d02c09: Deleting custom alerts plugin for 2.0.2 release refs #1486

Matthieu Aubry
Owner

@tsteur

PHP Strict standards: Declaration of Piwik\Plugins\CustomAlerts\tests\CustomProcessor::getMetricFromTable() should be compatible with Piwik\Plugins\CustomAlerts\Processor::getMetricFromTable($dataTable, $metric, $filterCond = '', $filterValue = '') in /home/travis/build/piwik/piwik/plugins/CustomAlerts/tests/ProcessorTest.php on line 15
Thomas Steur
Owner

I stopped using refs once we created the CustomAlerts repository. We have a nice history just for this plugin there and the refs are not being displayed here anyway. I think only for Piwik repository. We changed this recently with the Git problems and I think we even did not process all repositories before.

Matthieu Aubry
Owner

The git problems should now be fixed, now that git was upgraded on the server.

What would it take to process all repositories again and have trac Refs work on any github.com/piwik/* repository?

  • This would be nice as it brings openness and awareness of our work. Also it will help a lot users who come here from the changelog and like to see detailed list of changes.
Thomas Steur
Owner

Trac config ini needs to be adjusted for this. To work for all repositories automatically won't work

Thomas Steur
Owner

In c2d8069: refs #1486 make it possible to reuse only phoneNumbers and emails without the html for graph type

Thomas Steur
Owner

In 8769612: refs #1486 fix link to MobileMessage settings works only if current action is index

Thomas Steur
Owner

In 8b76960: refs #1486 move script tag into td to make sure nth-child(2n) css selector works. Otherwise sometimes two rows have same backgroundcolor in an entitytable

Thomas Steur
Owner

In e13682c: refs #1486 fix selected phone numbers are not recoginzed

Thomas Steur
Owner

In c18171d: refs #1486 trigger an event when a phone number is deleted

Thomas Steur
Owner

In 5773706: refs #1486 added some methods needed in CustomAlerts plugin to validate report and metric is still valid

Thomas Steur
Owner

In 329369d: refs #1486 fix website id is not updated but website name if someone switches the website in website selector

Thomas Steur
Owner

In 32f40ab: refs #1486 added method to detect whether current year is a leap year

Thomas Steur
Owner

In a9d8fc6: refs #1486 added method to create a scheduled time depending on the websites timezone

Thomas Steur
Owner

In 66bc3b1: refs #1486 if AllWebsites was selected in the site selector, the value was of the site selector was never updated. Also do not toggle the name of the current selected website with the clicked website as it is kinda confusing and leads to website names appearing multiple times

Thomas Steur
Owner

In 3664f48: refs #1486 added an anchor so we can directly link to a specific plugin

Thomas Steur
Owner

In 15853da: refs #1486 execute applyQueuedFilters only if not disabled, prevents an issue when filter_truncate and flat=1 is used in Actions.getPageUrls: Trying to add two strings values in DataTable\Row::sumRowArray

Thomas Steur
Owner

In 04ac332: refs #1486 updated submodule where we now store the reportUniqueId instead of module.action to support all reports (goals etc)

Thomas Steur
Owner

In 4b10f39: refs #1486 load fixtures from plugins when running UI tests

Thomas Steur
Owner

In 961f0e9: refs #1486 do not apply queued filters when they are disabled

Thomas Steur
Owner

In 9dbe600: refs #1486 in case we do not apply the filters directly, keep them in the new data table. Otherwise we cannot apply them later in the customAlerts plugin manually

Matthieu Aubry
Owner
Thomas Steur
Owner

In 711456d: refs #1486 this should fix the scheduled alerts for month or week run one day too late under circumstances.

Thomas Steur
Owner

In 416a911: refs #1486 fix tests

Thomas Steur
Owner

Closing this ticket for now. Please create new tickets for changes

Matthieu Aubry mattab added this to the 2.2.0 - Piwik 2.2.0 milestone
Thomas Steur tsteur was assigned by mattab
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 made previous alerts plugin compatible with Piwik 2.0 and …
…added the possibility to actually list, create and edit alerts
f9ff741
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 we do not want to have sql statements in controller, api a…
…nd plugin class. Put them in a model, makes it also better testable
3bd9c0b
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 some more cleanup, bugfixes and moved defined metrics/grou…
…p conditions to the processor as only this class can know which ones it supports
450e861
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 more refactoring, bugfixing, display report name is list o…
…f all alerts
853b18c
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 make it possible to reuse only phoneNumbers and emails wit…
…hout the html for graph type
c2d8069
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 fix link to MobileMessage settings works only if current a…
…ction is index
8769612
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 move script tag into td to make sure nth-child(2n) css sel…
…ector works. Otherwise sometimes two rows have same backgroundcolor in an entitytable
8b76960
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 fix selected phone numbers are not recoginzed e13682c
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 added some methods needed in CustomAlerts plugin to valida…
…te report and metric is still valid
5773706
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 fix website id is not updated but website name if someone …
…switches the website in website selector
329369d
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 added method to create a scheduled time depending on the w…
…ebsites timezone
a9d8fc6
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 if AllWebsites was selected in the site selector, the valu…
…e was of the site selector was never updated. Also do not toggle the name of the current selected website with the clicked website as it is kinda confusing and leads to website names appearing multiple times
66bc3b1
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 execute applyQueuedFilters only if not disabled, prevents …
…an issue when filter_truncate and flat=1 is used in Actions.getPageUrls: Trying to add two strings values in DataTable\Row::sumRowArray
15853da
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 updated submodule where we now store the reportUniqueId in…
…stead of module.action to support all reports (goals etc)
04ac332
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 in case we do not apply the filters directly, keep them in…
… the new data table. Otherwise we cannot apply them later in the customAlerts plugin manually
9dbe600
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 this should fix the scheduled alerts for month or week run…
… one day too late under circumstances.
711456d
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #1486 fix tests 416a911
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.