Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Allow Email Reports to be sent in HTML #2318

Closed
JulienMoumne opened this Issue · 52 comments

3 participants

Julien Moumné Anthon Pang Matthieu Aubry
Julien Moumné
Collaborator

See /trac/ticket/151#comment:11

Actions :

  • Refactor PDFRenderer as an implementation of an abstract report renderer class (Piwik_ReportRenderer)

  • Move PDFRenderer to core/ReportRenderer and rename it to Piwik_ReportRenderer_Pdf)

  • Add an HTML implementation to ReportRenderer (Piwik_ReportRenderer_Html)

  • Update PDFReport UI & database to allow sending reports in HTML

Julien Moumné
Collaborator

I have almost finished working on this ticket.

Images (Piwik logo, browser logos, ..) in emails have to link to a Piwik instance so they can be downloaded by email clients.

Should we add a config option with the URL of the piwik instance to use ? Or should we use

$_SERVER['SERVER_NAME']

or some sort?

Julien Moumné
Collaborator

I understand changing file names of a plugin is virtually impossible. (PDFReports -> EmailReports).

However, Is it ok to change

'name' => 'Pdf Export Plugin',

to

'name' => 'Email Report Plugin',

?

Same goes for

'PDFReports_PluginDescription' => 'Create and download your custom PDF reports, and have them emailed daily, weekly or monthly.',

'PDFReports_DefaultPDFContainingAllReports' => 'Default PDF containing all available reports.',

and many others

?

What about translations ?

Anthon Pang
Collaborator

Replying to JulienM:

Should we add a config option with the URL of the piwik instance to use ? Or should we use

$_SERVER['SERVER_NAME']

or some sort?

See Piwik_Url.

Anthon Pang
Collaborator

re: comment:2 it's easy to rename a plugin but a bit messy for the update script, and a PITA for translators.

That said, since pdf reports can also be downloaded, I would prefer simply "Reports".

Julien Moumné
Collaborator

Piwik_Url is what I was looking for thanks.

"Reports" is more correct indeed.

For translations, how about removing the translation for other languages so it appears as "not translated" in the translation ui ?

Julien Moumné
Collaborator

Attachment:
gmail-piwik-logo.jpg

Julien Moumné
Collaborator

Attachment:
gmail-browsers.jpg

Julien Moumné
Collaborator

Attachment:
msn-piwik-logo.jpg

Julien Moumné
Collaborator

Attachment:
msn-browsers.jpg

Julien Moumné
Collaborator
Julien Moumné
Collaborator
Julien Moumné
Collaborator
Julien Moumné
Collaborator
Julien Moumné
Collaborator
Julien Moumné
Collaborator
Julien Moumné
Collaborator

I've read the code managing database updates. I understand there is a way for plugins to define database updates the same way core does.

I have to add 'format' column to the pdf table. Should this go in core or plugin DB updates ?

Anthon Pang
Collaborator

core/Updates/

Matthieu Aubry
Owner

Excellent work mate...!

I tested report for komodo island and it worked well :)

Some feedback:

  • could logos have padding/margin right in outlook?
  • more padding in the first row
  • reuse colors / fonts / font-sizes of current Piwik UI so it looks as close as possible to current HTML
  • we could also modify PDF so they look the same as well
  • OK to modify translations for sure, but when you modify translations just change the Index of the translation, otherwise other 'outdated' translated strings will be reused. For example 'PDFReports_PluginDescription' could become 'PDFReports_PluginDescriptionReports' eg.
  • while "Reports" is maybe more correct, "Email reports" is better because it advertises the main purpose of the feature, which is to email reports. Downloading reports is rarely useful and mostly there for debugging
  • OK to put Updates in core/Updates rather than plugins/ as this is how we do it for all core plugins (when we don't it's either to show how it works, or by mistake) I also vote for including in 1.4 as 1.3 is now final and will be released anytime.
Julien Moumné
Collaborator

Now the ground work is done it's very easy to add new formats. I'm going to implement Piwik_ReportRenderer_Odt (using http://www.odtphp.com/) if that's ok.

Julien Moumné
Collaborator
  • logo padding ok for all clients
  • TH padding ok for all clients
  • new scheme applied to html and pdf as well
  • added core/Updates/1.4-rc1.php
Anthon Pang
Collaborator

It looks like the ODTPHP is license incompatible with the GPLv3. Also, development on the library seems to have tapered off -- what version of the Open Document Format does it support?

In the meantime, feel free to add support for odt, but check for the presence of the library in the event we can't include the library in the distribution.

Anthon Pang
Collaborator

phpdocwriter is LGPL (ok), but also hasn't been updated in a while.

If odtphp is a better library, you could try asking the devs to update the license (eg from "GPLv2" to "GPLv2 or above").

Julien Moumné
Collaborator

phpdocwriter generates .sxw files [http://phpdocwriter.sourceforge.net/phpdocwriter/examples/ex6.php] so I'm going to send an email to the developers of ODTPHP

Julien Moumné
Collaborator

On this page [they link to http://www.gnu.org/copyleft/gpl.html.

In [they also link to http://www.gnu.org/copyleft/gpl.html.

[http://www.gnu.org/copyleft/gpl.html] seems to be v3.

Do they only need to update the text in [http://www.odtphp.com/index.php?i=credits] from

OdtPHP is released under the terms of the GNU Public License. To get more informations about the GPL license, click here. 

to

OdtPHP is released under the terms of the GNU Public License. To get more informations about the GPLv3 license, click here. 

?

Anthon Pang
Collaborator

It must have been the sleep deprivation -- I swear I saw "v2" somewhere. Thanks for double checking. Go head.

Julien Moumné
Collaborator

I'm running into a bit of trouble with that library.

To zip/unzip odt data they use the PclZip library.

With the latest packaged version (1.0.1) it is not possible to override the temp directory used by PclZip.

Looking at the latest revision http://odtphp.svn.sourceforge.net/viewvc/odtphp?view=revision&revision=51 second point:

- Option to choose tmp directory by appplication works correctly now

shows they have fixed the issue.

However, it has not been packaged yet.

Should I update the file myself or drop the idea of having odt export for lack of quality libraries?

Anthon Pang
Collaborator

grab the latest from their svn

Julien Moumné
Collaborator

I should have investigated before. This library doesn't allow table generation with a variable number of columns.

They suggest to use http://pear.php.net/package/OpenDocument/

Julien Moumné
Collaborator

No table support in this library.

Matthieu Aubry
Owner

As per discussion I think safer not to implement ODT at this point.

Julien Moumné
Collaborator

Updates:

  • generate report as inlined content (outputType=0) is dropped. This feature is not that useful, not future proof (can any format be inlined in the browser?) and necessitate a Null Renderer so the ResponseBuilder doesn't append the success message at the end of the content.
  • toc is kept for HTML but removed for PDF (bookmarks are good enough)
Julien Moumné
Collaborator
Julien Moumné
Collaborator

Here is the proposed patch

  • @Review markers are remaining questions
  • includes r4415, r4505, r4506 (PDFRenderer & API).
  • styles are inlined because gmail doesn't support other means
  • fixes #2320
Matthieu Aubry
Owner

Feedback

  • @Review, should this be abstracted in a Piwik_EmailReport_x ? looks good as is
  • rename _epilogue to _footer for consistency?
  • function Piwik_Smarty() should be function __construct()
  • great work on the Piwik_ReportRenderer refactoring!
  • the comment " * To render a report as a pdf file, simply do: [...]" is not useful, because users will have to use the API to get any report, not use the low level classes. Only core will use these, and the code is clean & self documented as it is
    • I propose to change $outputFilename = PIWIK_INCLUDE_PATH . '/tmp/' . $filename; to use /tmp/assets instead. there have been permission problems with deleting files in tmp/ which I think will work better in tmp/assets since this directory is created by the webserver user, therefore there will be better umask there (or am I wrong...?)
  • @Review : Code inspired from TCPDF Plugin Here I wouldn't put all the caching headers etc. just keep it simple (by design, none of the API is cached at present). There is currently one piece of code that allows download of a file, it is in Csv.php > renderHeader() which is a lot simpler than TCPDF code (and similar header to Csv.php is probably enough for serving HTML content)
    • don't forget to change to // $smarty->assign("currentHost", "http://".Piwik_Url::getCurrentHost()); ;)
  • are both _epilogue.tpl and _header.tpl or could it just be _header.tpl?
  • when I generate an HTML report, only the Logo/intro and TOC are displayed, no reports are displayed. But maybe this is because the patch didn't apply properly or something..?
  • in the UI, when report is set to HTML, can you display a HTML logo, for example this one looks nice.
  • also would it be possible to capitalize HTML and PDF in the UI
  • and show the icons in the SELECT box as well as before the Download button?

After these modifications, OK to commit. Very good code, great work Julien!! :)

TODO post commit

  • add integration tests to test HTML output
    • when 1 report: no TOC
    • with all reports that are metadata compatible

(I can do it when this is committed)

Julien Moumné
Collaborator

(In [4512]) fixes #2318 #2320

Anthon Pang
Collaborator

(In [4516]) refs #2318

Matthieu Aubry
Owner

(In [4517]) refs #2318 Prevent "Notice undefined index" for existing (PDF) reports

Matthieu Aubry
Owner

(In [4518]) refs #2318 xss protection + working if piwik installed in sub dir (eg. in http://localhost/piwik)

Matthieu Aubry
Owner

(In [4519]) Adding link on logo. refs #2318
I think this feature is done, kuddos Julien!
I will just try and add integratio tests on the html output

Matthieu Aubry
Owner

Reopening, to not forget: HTML wont work correctly when sent from cron, since the hostname can't be read most of the time. We need to record the piwik hostname+path in a flag in the DB, that would be used in place in
$smarty->assign("currentPath", Piwik_Url::getCurrentUrlWithoutFileName() );

Matthieu Aubry
Owner

(In [4552]) Fixes #2318, #2325 - URL will be checked for in each piwik request to UI and API (not tracker), and will be set if not already set.
then, we use this URL for the HTML email images & link to the piwik reports

Matthieu Aubry
Owner

(In [4568]) Fixing Notice:</strong> <i>Undefined variable: reportFont</i> in <b>/home/www/dev5.piwik.org/core/ReportRenderer/Pdf.php</b> on line <b>95</b> Refs #2318

Matthieu Aubry
Owner

Regression in PDF: in Greek for example, letters now appear as ???? after 1.4-rc2 upgrade (reproducible on localhost). Before, it used to display OK. Maybe a refactoring miss out?

Julien Moumné
Collaborator

(In [4577]) fixes #2318 refs #2320 : revert font to the one used before trying to make style uniform

Julien Moumné
Collaborator

(In [4619]) refs #2318 fixing font-size incorrectly interpreted in gmail

Julien Moumné
Collaborator

(In [4659]) refs #2318 font & size enhancements

Julien Moumné
Collaborator

(In [5313]) refs #2318 fix for r4518, xss protection broke URLs in HTML Reports, applying same protection as in datatable_cell.tpl

Julien Moumné
Collaborator

(In [5415]) fixes #2706

  • refs #2318, #71 : Graphs now supported
  • refs #2670, #898 : Restoring VisitsSummary report metadata before eCommerce
  • refs #1721 : Additional security, filename fixes and new internal outputType
Julien Moumné
Collaborator

(In [5582]) * fixes #2706, #2828, #2704, refs #1721, #2637, #2711, #2318, #71 : horizontal static graph implemented

  • fixes #2788, refs #2670, #1721, #2637, #2711 : default graph type logic moved to ImageGraph API, improved date/period logic, new parameter graphs_default_period_to_plot_when_period_range
  • fixes #2704, #2804, refs #1721 : pChart updated to 2.1.3, pChart code removed from Piwik code, OOP refactoring, support for unifont.ttf if present in ImageGraph/fonts, testAllSizes now uses report metadata ImageGraphUrl
  • refs #5491 : space between report title and report table reduced to avoid page overflow
  • refs #2829 : TODO display percentages
  • r5544, r5547, r5549 merged
Julien Moumné
Collaborator

(In [6849]) refs #3323 #3088 #2708 #71 #2318

  • generate and compare HTML, PDF & SMS reports in Test_Piwik_Integration_EcommerceOrderWithItems & Test_Piwik_Integration_TwoVisitors_TwoWebsites_DifferentDays
  • report content as return value of PDFReports->generateReport() with new output type OUTPUT_RETURN
Julien Moumné
Collaborator

(In [6938])

refs #3227

  • include back <ImageGraphUrl>

refs #2318

  • reset smarty cycle between each report
Julien Moumné JulienMoumne added this to the 1.4 - Piwik 1.4 milestone
Julien Moumné JulienMoumne self-assigned this
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.