Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

New plugin: define Custom Alerts and receive email when triggered #1486

Closed
mattab opened this Issue · 52 comments

4 participants

@mattab
Owner

UPDATE January 26th: Custom Alerts plugin is now available in the Marketplace! Test it by using the latest Piwik beta version, and then installing the Custom Alerts plugin in one click: go to Piwik, click Settings then click Marketplace and click Install button.


Create custom alerts:

  • define the alert name ("Big drop in purchases")
  • select a website (or list of websites) on which the Alert is defined
  • receive alert by email (email will contain alert description + link to Piwik dashboard URL for the given website ID and period).
  • select the Alert period: should it be daily, weekly or monthly?
  • select the report (Websites, Keywords, Countries, general stats)
  • define Metrics (visits, page view, avg. visit duration, Goal 1 conversions, total goal conversions, etc.)
  • define the Alert: when Visits decrease 50%, when purchases are more than 50 per day, etc.

See attached screenshot for preview.

Note: Autocomplete the dimensions will probably not be in the V1 of the script. Autocomplete would be a nice feature as it helps users not making mistakes (eg. is it Germany or DE or germany?).

This plugin requires each Piwik plugin to define their API metadata (list of reports callable, and list of metrics + translations for each metric). Covered in #1485

@mattab
Owner

Feedback

  • This applies to: group reports by category within the SELECT
  • Pre-set useful alert: Alert me when Average generation time is more than 2 seconds
  • History of triggered alerts is long list. Maybe we can use AngularJs to display this list with paging?
@mattab
Owner
@mattab
Owner

Note: Custom Alert plugin UI html/js/css controller that allows to select one or several websites will be reused for sure in the PDF plugin.

It would be great if this bit of code was made reusable, thanks!

@mattab
Owner

Thought dump about unit tests

// test alert on one website
test_oneWebsite_visitIncrease
// test alert setup on two websites
test_severalWebsite_visitIncrease
// test alert setup on two websites, but user doesn't have access to one of these anymore
test_severalWebsite_userDoesNotHaveViewAccessForOneWebsite

// test all metrics matches/contains/etc. conditions
test_metrics_matchesExpression
test_metrics_containsExpression_withChineseCharacters

// test different periods
// test all increase/decrease/below/above conditions
test_metrics_IncreasePercent_LastDay
test_metrics_increaseValue_lastWeek
test_metrics_specificGoalConversion_isMoreThan_lastMonth

// tests searched string matches several rows (ie. keywords matching 'test')
// test with bounce rate: bounce rate being a processed metric, all matching rows should be summed before processing the bounce rate
test_metrics_matchSeveralRows_bounceRate
// same test, but simply with Visits evolution
test_metrics_matchSeveralRows_visits

// + test Goal metrics for one website, General goal metric alert..

For each of these tests, or maybe once for all tests, you could generate tracking hits using the code from Main.test.php

For example you would generate 10 or 15 visits to try and match all your use cases, then you would run in one test method all sub tests. (otherwise, generating visits for each test case would make tests run very slow).

In UI, important to test alert name has ', " for sql, and check chinese characters.

@mattab
Owner

(In [2585]) * Small modifications to UI

  • created a Filter/AddColumnsProcessedMetrics.php that can be reused in the Custom Alerts plugin refs #1486 - it processes the 3 default processed variables (actions per visit, avg time on site, bounce rate) for any datatable that has a 'dimension' in the returned metadata :)
@mattab
Owner

(In [2594]) Refs #1485

  • Fixing wrong metadata for all plugins, cleaning up columns and ensure consistency in API (eg. VisitsSummary.get now returns conversion rate with %)
  • Adding a generic filter Truncate, updated doc at: http://dev.piwik.org/trac/wiki/API/Reference filter_truncate ; if set, will truncate the table after $filter_truncate rows. The last row will be named 'Others' (localized in the requested language) and the columns will be an aggregate of statistics of all truncated rows.
  • Adding common helpers to get a "pretty" value from a given metric and column name
  • Adding new time export format that will be used in PDF reports 12:34:56 format

Refs #1486

  • a new filter, AddColumnsProcessedMetricsGoal, can be reused to easily add processed goal metrics for a given datatable. For example: $dataTable->filter('AddColumnsProcessedMetricsGoal', array($enable=true, Piwik_DataTable_Filter_AddColumnsProcessedMetricsGoal::GOALS_FULL_TABLE)); will process all goal metrics (and per goal conv rate, rev per visit, conversions). When possible, only reports necessary should be loaded using GOALS_MINIMAL_REPORT or GOALS_OVERVIEW or a given idGoal
@mattab
Owner

(In [2701]) Refs #1486
Patch by halfdan required by Alerts plugin

  • Invert pattern match
  • Date.addPeriod and Date.subPeriod
@halfdan
Collaborator

I'm uploading a current status here - not quite working, more for code review and feedback. I'm half-way through migrating to the new UI that was introduced with PDFReports.

Integration.test is testing addAlert/editAlert (success!) and processing of alerts (little buggy).

I'll be travelling for some days and will finish Alerts when I'm back.

@halfdan
Collaborator

Attachment: Current status - not for production
Alerts.zip

@mattab
Owner

delaying for now, even though I hope it makes the cut :)

@mattab
Owner

Halfdan, do you think the plugin can be put into trunk at some point? This would be a great feature and it is more than half done for sure :)

@mattab
Owner

@halfdan Have you had a chance to make progress on the feature? Even if not complete we'd be very interested to see what you've done and if we can reuse it. Thanks!

@anonymous-piwik-user

Hi, I would be very interested to help or to continue this project. @halfdan, it would be very nice if you could report what is working and what not, and which kind of problems you encountered.

@halfdan
Collaborator

Hi @Codalibre,
that is very kind of you. I am however still working on this plugin and hope to have it finished by the end of the year. I have not responded to matt's request since I am in direct contact with him.

I will let you know if I need any help.

@mattab
Owner

If you are interested in this feature, please pledge now! We are running a campaign for Custom Alerts: http://crowdfunding.piwik.org/custom-alerts-plugin/

It will be awesome, but we need your support!

@anonymous-piwik-user

I'd love to see another feature.
Piwik could become a helpful tool in order to be alerted when it comes to problems with the website.

There are a lot of things that can go wrong after updating your website. You have a lot of log files and tests but nevertheless sometimes there goes something wrong on the website (e.g. browser problems with js, layout problems, ...).

Sure, you could have an alert for higher bounce rates on critical pages (e.g. product pages).

It would be great if there were alerts that are a bit more complex.
My scenario:
Alert if the ratio of visitors that view a product page, add a product to cart and checkout decreases.
This means the checkout funnel should alert me if there is a problem in one of the steps.

@mattab
Owner

@mohelten Piwik does not yet support conversion funnel, so this is not possible, but it's a good idea for the future.

@mattab
Owner

I got a few emails in the last few weeks from funders asking about Custom Alerts.

We have some news: while unfortunately Fabian @halfdan is right now in hospital recovering from accident but will be out soon. AND he has a version of the plugin for beta which we'll release as part of Piwik 2.0 as a core plugin, as announced. Stay tuned & halfdan hope you get well!

@mattab
Owner

@Fabian any update for our community of funders eagerly waiting to hear an update? hope you are well!

@mattab
Owner

hey @Fabian any update?

@mattab
Owner

No update yet, so likely won't make it to 2.0 (sorry for such long wait to all funders!)

@tsteur
Owner

In f9ff741: refs #1486 made previous alerts plugin compatible with Piwik 2.0 and added the possibility to actually list, create and edit alerts

@tsteur
Owner

In 1e6cece: refs #1486 code cleanup, implemented some comparisons, bugfixes

@tsteur
Owner

In 3bd9c0b: refs #1486 we do not want to have sql statements in controller, api and plugin class. Put them in a model, makes it also better testable

@tsteur
Owner

In 450e861: refs #1486 some more cleanup, bugfixes and moved defined metrics/group conditions to the processor as only this class can know which ones it supports

@tsteur
Owner

In 853b18c: refs #1486 more refactoring, bugfixing, display report name is list of all alerts

@mattab
Owner

In 8d02c09: Deleting custom alerts plugin for 2.0.2 release refs #1486

@mattab
Owner

@tsteur

PHP Strict standards: Declaration of Piwik\Plugins\CustomAlerts\tests\CustomProcessor::getMetricFromTable() should be compatible with Piwik\Plugins\CustomAlerts\Processor::getMetricFromTable($dataTable, $metric, $filterCond = '', $filterValue = '') in /home/travis/build/piwik/piwik/plugins/CustomAlerts/tests/ProcessorTest.php on line 15
@tsteur
Owner

I stopped using refs once we created the CustomAlerts repository. We have a nice history just for this plugin there and the refs are not being displayed here anyway. I think only for Piwik repository. We changed this recently with the Git problems and I think we even did not process all repositories before.

@mattab
Owner

The git problems should now be fixed, now that git was upgraded on the server.

What would it take to process all repositories again and have trac Refs work on any github.com/piwik/* repository?

  • This would be nice as it brings openness and awareness of our work. Also it will help a lot users who come here from the changelog and like to see detailed list of changes.
@tsteur
Owner

Trac config ini needs to be adjusted for this. To work for all repositories automatically won't work

@tsteur
Owner

In c2d8069: refs #1486 make it possible to reuse only phoneNumbers and emails without the html for graph type

@tsteur
Owner

In 8769612: refs #1486 fix link to MobileMessage settings works only if current action is index

@tsteur
Owner

In 8b76960: refs #1486 move script tag into td to make sure nth-child(2n) css selector works. Otherwise sometimes two rows have same backgroundcolor in an entitytable

@tsteur
Owner

In e13682c: refs #1486 fix selected phone numbers are not recoginzed

@tsteur
Owner

In c18171d: refs #1486 trigger an event when a phone number is deleted

@tsteur
Owner

In 5773706: refs #1486 added some methods needed in CustomAlerts plugin to validate report and metric is still valid

@tsteur
Owner

In 329369d: refs #1486 fix website id is not updated but website name if someone switches the website in website selector

@tsteur
Owner

In 32f40ab: refs #1486 added method to detect whether current year is a leap year

@tsteur
Owner

In a9d8fc6: refs #1486 added method to create a scheduled time depending on the websites timezone

@tsteur
Owner

In 66bc3b1: refs #1486 if AllWebsites was selected in the site selector, the value was of the site selector was never updated. Also do not toggle the name of the current selected website with the clicked website as it is kinda confusing and leads to website names appearing multiple times

@tsteur
Owner

In 3664f48: refs #1486 added an anchor so we can directly link to a specific plugin

@tsteur
Owner

In 15853da: refs #1486 execute applyQueuedFilters only if not disabled, prevents an issue when filter_truncate and flat=1 is used in Actions.getPageUrls: Trying to add two strings values in DataTable\Row::sumRowArray

@tsteur
Owner

In 04ac332: refs #1486 updated submodule where we now store the reportUniqueId instead of module.action to support all reports (goals etc)

@tsteur
Owner

In 4b10f39: refs #1486 load fixtures from plugins when running UI tests

@tsteur
Owner

In 961f0e9: refs #1486 do not apply queued filters when they are disabled

@tsteur
Owner

In 9dbe600: refs #1486 in case we do not apply the filters directly, keep them in the new data table. Otherwise we cannot apply them later in the customAlerts plugin manually

@mattab
Owner
@tsteur
Owner

In 711456d: refs #1486 this should fix the scheduled alerts for month or week run one day too late under circumstances.

@tsteur
Owner

In 416a911: refs #1486 fix tests

@tsteur
Owner

Closing this ticket for now. Please create new tickets for changes

@mattab mattab added this to the 2.2.0 - Piwik 2.2.0 milestone
@tsteur tsteur was assigned by mattab
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 made previous alerts plugin compatible with Piwik 2.0 and …
…added the possibility to actually list, create and edit alerts
f9ff741
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 we do not want to have sql statements in controller, api a…
…nd plugin class. Put them in a model, makes it also better testable
3bd9c0b
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 some more cleanup, bugfixes and moved defined metrics/grou…
…p conditions to the processor as only this class can know which ones it supports
450e861
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 more refactoring, bugfixing, display report name is list o…
…f all alerts
853b18c
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 make it possible to reuse only phoneNumbers and emails wit…
…hout the html for graph type
c2d8069
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 fix link to MobileMessage settings works only if current a…
…ction is index
8769612
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 move script tag into td to make sure nth-child(2n) css sel…
…ector works. Otherwise sometimes two rows have same backgroundcolor in an entitytable
8b76960
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 fix selected phone numbers are not recoginzed e13682c
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 added some methods needed in CustomAlerts plugin to valida…
…te report and metric is still valid
5773706
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 fix website id is not updated but website name if someone …
…switches the website in website selector
329369d
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 added method to create a scheduled time depending on the w…
…ebsites timezone
a9d8fc6
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 if AllWebsites was selected in the site selector, the valu…
…e was of the site selector was never updated. Also do not toggle the name of the current selected website with the clicked website as it is kinda confusing and leads to website names appearing multiple times
66bc3b1
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 execute applyQueuedFilters only if not disabled, prevents …
…an issue when filter_truncate and flat=1 is used in Actions.getPageUrls: Trying to add two strings values in DataTable\Row::sumRowArray
15853da
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 updated submodule where we now store the reportUniqueId in…
…stead of module.action to support all reports (goals etc)
04ac332
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 in case we do not apply the filters directly, keep them in…
… the new data table. Otherwise we cannot apply them later in the customAlerts plugin manually
9dbe600
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 this should fix the scheduled alerts for month or week run…
… one day too late under circumstances.
711456d
@sabl0r sabl0r referenced this issue from a commit in sabl0r/piwik
@tsteur tsteur refs #1486 fix tests 416a911
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.