ResourcesBot: Implement better reporting

[holybiber]

The ResourcesBot should generate some nice report of what it did. Like when it runs daily, I'd like to get an email with a summary after the run saying something like:

• 5 new worksheets added, 3 worksheets updated, 2 new PDFs
• updated 6 language reports
• detected 25 inconsistencies

Details:

• Spanish: 1 new worksheet, 1 new PDF
• Tamil: 1 new worksheet
• ...
• ExportPDF: ...
• ExportHTML: ...

Not sure how to do that best. Probably add some data structure / function to the module registration (#113) / the abstract base classes for modules in post_processing.py
Not sure if it's possible to find some structured way, probably just have a string. Maybe two:

• one short overview
• one more detailled report

I think it's best to use mediawiki markup for this string to add links etc. Because we may want to write this report to some internal page directly into the system for reference and with one mediawiki API query it's possible to convert that into HTML and send out an HTML email e.g.

[holybiber]

Some more specific thoughts:

• Introduce resourcesbot/reporting.py with a class ModuleReport which holds the report of running one module with one specific language
• class ModuleReport should have some methods:
  • boolean was there anything to report?
  • int count_incidents
  • warnings (return string with gathered warning logs from the module run)
  • summary (e.g. "Updated list of available training resources" or empty string if there was nothing to report)
• LanguagePostProcessor.run returns such a ModuleReport
• within one full resourcesbot run for each of the 7 modules and for each of the ~50 languages we'll have 350 ModuleReports that can be combined in different ways to return meaningful reports for the people involved

New abstract static method LanguagePostProcessor.report_summary(Dict[str, ModuleReport] reports) -> str that each module needs to implement and writes one summary line from the ~50 reports from its runs for each language, e.g.

• module WriteList will say Updated the list of available training resources in 4 languages
• module ConsistencyCheck will say Detected 67 inconsistencies (looking at ModuleReport.count_incidents)

Also generate language-specific more detailed reports, like for every language one line (ModuleReport.summary) for each module + warnings if there have been any in that language. That should probably another static function in reporting.py

General thoughts:

• generate mediawiki (e.g. for links) and if necessary convert that to HTML
• in the end send out HTML mails: one with an overall overview, in the future also individual language-specific mails
• warnings (from the logger, already existing) + reports should be included. Other logging information (logging.INFO...) should not be included. If that's needed to debug something we can look into resources_info.log

Next step would probably be to break that down into some smaller sub-tasks.

[GabrielSousa02]

One thought to add:
Within that class, we should have specific methods that would return specific formatted text output, e.g.:

• report.generate_markdown()
• report.generate_mediawiki()
• report.generate_html()
• report.generate_text()

This would facilitate the entire reporting process, as we could generate emails, files or any other format we want, using those pre-built functions to create the text in the desired formatting.

[JoshKowi]

What if we have one general ModuleReport class implementing the basics and then have sub-classes for each module, like ExportHtmlReport and WriteListsReport ?
Then it would be possible to have a classmethod ModuleReport.report_summary(Dict[str, ModuleReport] reports) -> str with which the corresponding summary for one module could be created independently of the LanguagePostProcessor-Object, directly out of the reports of one type.
I feel like this could be a cleaner solution, what do you think?

[holybiber]

@JoshKowi yes that sounds like a good idea, go ahead!

[JoshKowi]

I pushed a first prototype on my local fork - maybe have a look at the architecture and if the result is what you need. (At the moment a report is just being printed to the command line at the end of the resourcesbot run.
Next steps would be:

• Add export option for more Detailed report per module (what was done for each language) in reporting.py
• Add different export options in reporting.py like generate_html etc
• Add report for resourcesbot
• Add report for GlobalPostProcessor write_summary
• Gather more information, like warnings and errors.

Many of the LanguageReport subclasses are quite similar and differ only by a few lines, but I'm not sure if there is a better architecture that still provides the possibility to gather customized Information and create a customized summary for each module. If you have good ideas, please let me know.