Stats: reporting and visualization over time

[ntarocco]

Reports

It would be very useful to generate reports. Details to be defined.

Visualization over time

View/download stats should be visualized in a period of time. Technical implementation to be discussed.

[lnielsen]

Here's a couple of draft ideas for how a Community Metrics page could look like for further discussion:

[ntarocco]

Feedback from users: it would be also very useful to see the view/download stats in the record landing page and community by geolocation, so that users can easily visualize this at global or local scale.

[monotasker]

I've been working on a stats-dashboard module for Knowledge Commons works that took these mockups as a starting point. It's nearing completion as a stand-alone extension package. I'll share more details here next week, but a few notes:

• It works for an entire instance (as an instance stats page) as well as for individual communities
• This version is themed for our sites, but obviously fonts, colours, etc. could have different defaults
• Layout and components are configurable via a config variable (dictionary), so it can be extended easily
• Includes back-end pre-aggregation of daily time-series data with sub-counts based on a variety of metadata fields (resource type, funder, language, etc.)
• Exposes the pre-aggregated stats via the stats API
• The box widgets have configurable dynamic view switching between bar chart, ring chart, and test list. (In the config dictionary one can set the default view and which view options are exposed for each widget)

Here are some screenshots of the current iteration (using test data). Some things still need significant work, and there are bugs. But you can see what we've built so far.

[slint]

This looks amazing :)

You mention that this could work for communities as well. One of the prerequisites for this is having the community IDs indexed in the stats indices (at least the aggregations), similarly to how we include the resource type and other metadata.

Is this something you managed to address? Otherwise it's something we plan to work on in the Zenodo team in early August (I've updated #188 to reflect that)

[monotasker]

Thanks! And yes, that's actually one of the last pieces that I'm working on this week. I'm adding community ids and a few key pieces of record metadata to those view and download event documents. I'm creating a service that will update those index templates and migrate the monthly indices for past events, augmenting them with the extra data. That's a process that will have to run for some time as a background task on big instances, but I think it will be manageable. I'm also including event builders that use the updated index template mapping and can be applied via the STATS_EVENTS configs.

[monotasker]

The traffic tab (again, this layout is all configurable)

[monotasker]

The usage tab:

[monotasker]

The contributions tab (currently the least built-out):

[monotasker]

Feedback is most welcome, and if I can adapt the package to meet the needs of InvenioRDM generally, I'll be happy to make changes or collaborate with others on it.

At the moment I'm doing some significant reworking of the back-end usage stats aggregation, which has been tricky to do at scale. But I'll share more next week.

[mfenner]

I look forward to the presentation and discussion next week. For my instance I have a self-hosted Plausible Analytics instance that generates some of these dashboards https://plausible.rogue-scholar.org/rogue-scholar.org. I have also tested the umami web analytics platform https://umami.is, which has easier requirements but is not as powerful.