DB space usage: Purge temporary archives more often #2369

mattab opened this Issue Apr 28, 2011 · 2 comments

1 participant

Piwik Open Source Analytics member

Currently, we purge out of date archives, only that are more than 24 hours (sometimes, up to more than 48 hours old).
If the archiving runs very frequently (eg. the API is used for something all the time), then Piwik will create many archives that are quickly out of date.

we should delete out of date more aggressively.

This will have a side effect:

  • if archiving in browser in browser is disabled
  • AND timeout set to N seconds (eg. N = 3600)
  • AND cron running archive.sh runs less often than N seconds
  • THEN the latest cron run will delete outdated archives (that are older than N + safety margin of 1 hour)
  • THEN if user requests reports from the browser, the out of date archives that used to display fine (because we used to purge them only once they were 24 hours old), will not display anything anymore... so users will have to fix their cron so it runs more frequently, or increase the Timeout in General Settings to match the cron frequency

The above explanation is in preparation of forum posts "there is no data but it used to work" -> Ask "do you use cron, what is timeout in General Settings, check cron runs as often..." I'll probably add it to the FAQ if it's asked often

Piwik Open Source Analytics member

(In [4587]) Fixes #2369

Piwik Open Source Analytics member

To produce the most efficient database space usage with current algorithm, it is necessary to archive 'today' (clean up this monthly archive tables), and archive 'this year' (clean up january archive table containing all yearly archives).

For example for one forum user, it went from 1.5Gb to.... 65Mb ;)

@mattab mattab added this to the 1.4 - Piwik 1.4 milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment