New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Piwik reports 0 visits for single days ni some rare cases #2548

Closed
peterbo opened this Issue Jul 6, 2011 · 12 comments

Comments

Projects
None yet
4 participants
@peterbo
Copy link
Contributor

peterbo commented Jul 6, 2011

See http://forum.piwik.org/read.php?2,78544,78566, and http://forum.piwik.org/read.php?5,78506.

In my case, archiving is triggered by the browser. No cron job is running.

MySQL: 5.0.51a-24+lenny5
PHP: 5.2.6-1+lenny10

@peterbo

This comment has been minimized.

Copy link
Contributor Author

peterbo commented Jul 6, 2011

Attachment:
piwik-bug.JPG

@mattab

This comment has been minimized.

Copy link
Member

mattab commented Jul 7, 2011

I have seen this bug occur maybe once every 6 months on demo.piwik.org as well. I suspect this comes from edge case when archiving is running twice at the same time, or maybe in the case of failing archiving, but I'm not sure.

Fixing the data is done by deleting the piwik_archive_xx table for the month containing the "zero data" reports, and then re-processing usually works fine.

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Jul 31, 2011

See #2597 and #2574

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Jul 31, 2011

(In [5052]) fixes #2602, fixes #2574, refs #2548, refs #2597

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Jul 31, 2011

So, my thought on this is to use MySQL's advisory lock in launchArchiving()

        public function launchArchiving()
        {
                $this->get_lock();
                $this->initCompute();
                $this->compute();
                $this->postCompute();
                // we execute again the isArchived that does some initialization work
                $this->idArchive = $this->isArchived();
                $this->release_lock();
        }

Note: Lock names are system-wide. We could use something like:

$lockname = "piwik.$dbname.$tables_prefix.$idsite";
@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Jul 31, 2011

How long should the timeout be? And should it keep retrying indefinitely?

@mattab

This comment has been minimized.

Copy link
Member

mattab commented Jul 31, 2011

The lock is I think a good idea!

the lock should also contain the $date as we want to protect records for a given site on a given day to run concurrently.

The potential problem is for times that the archiving will crash (memory exhausted etc.) after the lock was acquired. Then, would we have to release it manually, or how would this work?

This has been a long standing bug this one, which would be nice to fix if it doesn't adds complexity etc. :)

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Jul 31, 2011

By date, do you mean dateStart, dateEnd, and period?

If the mysql client connection is terminated, the lock is automatically released. There is a note somewhere which mentions a bug in older mysql versions: if the connection terminated, the lock wouldn't be released right away (i.e., it would still wait until the timeout expired).

I'm thinking a timeout of 30 seconds with no retry.

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Aug 1, 2011

(In [5065]) fixes #2548, refs #2597 - use advisory locks

@DaSchTour

This comment has been minimized.

Copy link
Contributor

DaSchTour commented Sep 3, 2011

Maybe this should be release in 1.5.2? They way till 1.6 is long and I don't want to manipulate my database to have the stats repaired

@DaSchTour

This comment has been minimized.

Copy link
Contributor

DaSchTour commented Oct 30, 2011

how is this fixed now? will Piwik recognise the broken days? Will I still have to delete something in the database? Only fixing that this will not happen again is only the half-solution, repairing the result of this bug should be done for a real fix

@robocoder

This comment has been minimized.

Copy link
Contributor

robocoder commented Oct 31, 2011

It should be fixed for new data.

The Piwik update doesn't automatically repair previously archived data because it could potentially time-out (i.e., time intensive). It's also non-trivial to analyze previously archived data to see if an archived 0 really was a 0.

You're welcome to drop your piwik_archive_* tables, and run the archive script manually (or via cron).

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment