Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove metrics computation #273

Merged
merged 7 commits into from Nov 20, 2023
Merged

Remove metrics computation #273

merged 7 commits into from Nov 20, 2023

Conversation

abulte
Copy link
Contributor

@abulte abulte commented Sep 8, 2022

This is to prepare for the release of a new udata-metrics that handles metrics computation from log files, instead of from Matomo.

--

Up until this version, metrics were tentatively computed from Matomo for pretty much every object of a udata instance. We found this method to be pretty inaccurate, since Matomo limits the number of records stored to roughly the most visited 1000 for a given "namespace". There's a way around that by using the data from the Live module of Matomo but it's not enabled on our instance for privacy reasons. We also needed to count hits that Matomo could not know of: resources (files) downloads made directly on our server.

Thus, this module now only does three things:

  • inject the Matomo tracking code in the templates
  • process events for tracking the API usage in Matomo
  • process events for tracking some Matomo goals

Metrics computation is now handled by https://github.com/opendatateam/udata-metrics and does not use Matomo as a source.

⚠️ Jobs to be unscheduled:

  • piwik-update-metrics
  • piwik-current-metrics
  • piwik-yesterday-metrics

@abulte abulte marked this pull request as draft September 8, 2022 15:03
@abulte abulte marked this pull request as ready for review September 12, 2022 14:54
@abulte
Copy link
Contributor Author

abulte commented Sep 12, 2022

(Wait for the metrics pipeline refactoring is complete to ship this)

Copy link
Contributor

@quaxsze quaxsze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such nettoyage

@maudetes
Copy link
Contributor

(Related to datagouv/data.gouv.fr#915)

Copy link
Contributor

@maudetes maudetes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

version: '3'
services:
piwik:
image: udata/piwik-test
Copy link
Contributor

@maudetes maudetes Sep 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want to keep a piwik/matomo image for testing and dev purposes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted for total removal because this image is quite a complex hack that was done specifically to debug metric computations (I should know, I did it ;-)). I don't think it's necessary anymore but YMMV.

@maudetes maudetes merged commit 48d4c33 into master Nov 20, 2023
1 check passed
@maudetes maudetes deleted the prepare-v2 branch November 20, 2023 15:25
@abulte
Copy link
Contributor Author

abulte commented Nov 20, 2023

Yeah! 🚀 ⏲️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants