Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

In archiving process, truncate datatables in-place everywhere this can result in speed gains #3484

Closed
diosmosis opened this issue Oct 27, 2012 · 6 comments
Closed

In archiving process, truncate datatables in-place everywhere this can result in speed gains #3484

diosmosis opened this issue Oct 27, 2012 · 6 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone
Long term

Comments

@diosmosis
Copy link
Member

Most of Piwik when archiving will generate DataTables and then apply a sort and truncation filter before storing them as records. This is highly inefficient when dealing w/ DataTables w/ thousands of rows, since every one of those rows will have to be first allocated & constructed then sorted.

As done in the Actions plugin it is possible to apply ordering through the SQL and then use new DataTable functions to limit the number of rows in a DataTable w/o having to create and sort every possible one. This technique should be used everywhere possible.
@mattab
Copy link
Member

mattab commented Dec 15, 2012

It should be used for all reports that are bigger than normal: Custom Variables, Geoip reports, Actions, Referrers.

@anonymous-matomo-user
Copy link

When will you start on this? Is there some more documentation to possibly patch this ourselves?

@mattab
Copy link
Member

mattab commented Feb 11, 2013

Not planned at this stage as we lack resources. You are welcome to sponsor this work if you want to get it done, or send a patch....

@anonymous-matomo-user
Copy link

What would the sponsorship cost?

@mattab
Copy link
Member

mattab commented Feb 14, 2013

Get in touch for more info at matt@piwik.org or http://piwik.org/consulting/

@diosmosis diosmosis added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the P: normal label Aug 3, 2014
@mattab
Copy link
Member

mattab commented Mar 30, 2015

This issue was fixed in 2.12.0!

@mattab mattab closed this as completed Mar 30, 2015
@mattab mattab added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Mar 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Performance For when we could improve the performance / speed of Matomo. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Milestone
Long term
Development

No branches or pull requests
3 participants
@diosmosis @mattab @anonymous-matomo-user