Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when trust_visitors_cookies = 1 is used, replaying logs cause discrepencies #6664

Closed
mattab opened this issue Nov 13, 2014 · 0 comments
Closed
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Nov 13, 2014

Set as Critical priority as it's blocking some work for Enterprise client. there are data discrepencies between the Live instance and the instance built with replayed logs.

Explanation
in the log importer we do not properly put all the requests for a given visitor into the same bulk requests, and maybe requests by same visitor ends up being inserted at the same time in two or more bulk requests. The logic for creating queues is here: https://github.com/piwik/piwik/blob/master/misc/log-analytics/import_logs.py#L1161-L1166 - it may be fragile.

This is causing an issue when

  • trust_visitors_cookies = 1 in the config.
  • --replay-tracking is used in the log importer

Proposed solution

Currently requests are grouped by IP address.

New proposed logic:

  • if uid is set, use it,
  • else if cid is set, use it,
  • else if _id is set, use it,
  • else if cip is set, use it,
  • else use IP address.

This should then solve the problem with trust_visitors_cookies = 1 as in all cases (even when the setting is 0) the requests will be grouped by Visitor ID or Visitor IP (with priority to Visitor ID).

@mattab mattab added Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority. labels Nov 13, 2014
@mattab mattab added this to the Piwik 2.10.0 milestone Nov 13, 2014
@diosmosis diosmosis self-assigned this Nov 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. Critical Indicates the severity of an issue is very critical and the issue has a very high priority.
Projects
None yet
Development

No branches or pull requests

2 participants