VisitFrequency API: read pre 2.0 data

[mattab]

in VisitFrequency API in 2.0 we started processing data for a segment:

const RETURNING_VISITOR_SEGMENT = "visitorType%3D%3Dreturning%2CvisitorType%3D%3DreturningCustomer";

In pre 2.0 it used to be recorded as metrics in the VisitFrequency plugin archives.

Let's add a few lines of code to read the data from pre-2.0 format, when the $date is earlier than Dec 17th (date of 2.0 release)

This is a follow up of the change in a701d4a

[mattab]

Attachment: I think incorrect fix :(
Fixes_#4878_Read_the_old_data_first__This_is_only_a_partial_fix.patch

[mattab]

The complicated thing is to deal with cases where the data spans across multiple days or weeks or months. In this case the API must both select the old data and the new one and then merge these two datasets into the returning datatable.

[diosmosis]

In 6db8dea: Fixes #4878, use pre-piwik 2 data in VisitFrequency API if post-piwik 2 data cannot be found. Commit includes commented out tests for changes.

[mattab]

build is failing, reopening

[mattab]

I have tested as follows:

• install 1.12
• generate data, and archive
• check that there are returning visits in Visitors>Engagement
• Delete all logs
• upgrade the DB to 2.1.0
• open Visitors>Engagement

What I see:

• Metrics are displayed below the graph
• The graph has no value
• The sparklines are flat

See attached screenshot.

can you reproduce?

[mattab]

Attachment:
sparklines and graph flat, but metrics work.png

[mattab]

In 99319ec: Refs #4878 typos @diosmosis please add integration test triggering this code path

[mattab]

https://demo.piwik.org/index.php?module=API&method=API.get&format=XML&idSite=7&period=day&date=2014-02-24,2014-03-25&expanded=1&segment=visitCount%3E=5,actions%3E=10;customVariableName3==Forum%20status;customVariableValue3==LoggedIn%20user

Call to undefined method Piwik\Plugins\VisitFrequency\API::addPrePiwik2DataIfMissing()

[mattab]

In 1ff6595: Refs #4878 Fix error addPrePiwik2DataIfMissing not defined

[diosmosis]

In 8dfafd8: Refs #4878, fix bug in VisitFrequency.addPrePiwik2DataIfNewDataAbsent and add test for multiple periods w/ pre Piwik 2 VisitFrequency data.

[diosmosis]

Tested w/ updated piwik 1.12 install, works. Closing.

[diosmosis]

In daf970f: Fixes #4878, re-introduce VisitFrequency Archiver in order to successfully combine pre-2.0 VisitFrequency data with new data.

[diosmosis]

In 775663f: Refs #4878, fix nb_uniq_visitors metric issue in VisitFrequency archiving and fix errors in build.

[diosmosis]

In ccf4b4b: Refs #4878, fix strange bug in computing unique returning visitors for visit frequency api that caused fixture to fail when running UI tests.

[diosmosis]

In 2acf544: Refs #4878, add another test.