Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Segment Editor: request should timeout when it takes too long to generate auto suggested values #4253

Closed
gaumondp opened this Issue · 14 comments

3 participants

gaumondp Matthieu Aubry Thomas Steur
gaumondp

When you got a website with more than 10 000 pages, using the Segment editor with "Page URL" makes it erratic.

The ajax call to gets that many URLs takes more than a minute and produce a "Page unresponsive" with Chrome. And I'm on a dedicated PHP/Piwik machine with MySQL on another.

Maybe there should be a check by Piwik to see if there's more than 200 urls to not try to fill the area and let us fill it manually?
Keywords: Segment editor

Matthieu Aubry
Owner

it works for us on piwik.org but we do not have 10,000 pages indeed. what is the error in your server error logs ?

gaumondp

Technically there is no Piwik error on the server Matt.

It's the web browser that gets impatient and send a "Page unresponsive" after waiting too long.

A quick fix (I can't provide since I'm no coder, sadly) would be to add a "no ajax query in segment editor".

BTW, we also get the infamous "Page unresponsive" when trying to use Segment Editor with IP filtering because Piwik tries to help us by filling the field from many thousands of IP that went at our door...

Small websites will not have such trouble but Government' ones (lots of different users and more than 1000 pages) will always do.

Matthieu Aubry
Owner

It was reported in #4509 as well:

Running v2.0.2
Database is approx 10gb.
When I go to create a new segment using the page url filter, it takes an incredibly long time to pre-fill the url list (Avg 5-10mins), effectively locking the table, resulting in locked transactions for the live stats data that is coming in. We're getting on average 5 hit per second during peak times. We don't want to go over 500 max connections on our database, which we max every time i try to add a new segment. Is there a way to not have the segment editor query the table for the prefilled url list so i can just specify the url i want to use? Thanks!

Matthieu Aubry
Owner

If you need to work around the issue, you can use the latest beta version, disable the "Live" plugin, and then the segment editor will work.

Matthieu Aubry
Owner

Realistically, this is clearly a bug: It crashes and prevent users from creating segment.

We should find a solution to this issue, some ideas of possible solutions:

  • Setting to disable auto-suggest feature and/or set it to lower number of visits.

  • Read values from archives (call plugin APIs) instead of reading from logs (from Live API)

    • To know the raw value to segment with, for each plugin API we need to know the mapping from Archived label <> original value from logs.
gaumondp

I don't speak fluent ajax but maybe introduce some kind of timout "if we reach 10 seconds to get back data, disable autocomplete".

Or a manual "stop autocomplete" at the right side of any field in segment editor (IP , Page name and URL are those common).

Anything to make Segment Editor usable in large websites.

Dali

Thomas Steur
Owner

In e81d7e1: refs #4253 as suggested by dalidev I specified a timeout for the autocomplete request (20seconds). If it is not loaded after 20 seconds

Thomas Steur
Owner

In ea91d38: refs #4253 slightly decrease the limit as there will be only 30 values kept anyway

Thomas Steur
Owner

@dalidev: Do you still experience "Page unresponsive" issues with the latest 2.2.1 release? I had a look at the API which returns max 30 results. This is implemented already for a longer time so there "shouldn't" be any "page unresponsive" issues.

Otherwise I decreased the limit a little bit which should make it slightly faster on the server side and added a request timeout of 20seconds on the client side.

gaumondp

Thomas, my server doesn't see the internet, auto-update isn't possible. Since I got +50k pageviews a day I don't upgrade too often. I'm at 2.2.0 at the moment.

So I'm limited in quick feedback on recent fixes. Pretty sorry about that.

But looks good on paper. ;)

Dali

Matthieu Aubry
Owner

Ok we experience the same issue on demo.piwik.org. The request for auto-suggest takes 2 minutes or so, and if I click on "Save & Apply" before the end of that request, then the "SegmentEditor.add" request is queued and waiting for auto-suggest response to come back.

Here is proposed solution:

  • When "Save & Apply" is clicked, cancel all running ajax requests
Matthieu Aubry
Owner

See #5121 New config setting to disable segment auto complete

gaumondp

Thanks for the new feature for sites with 10 000 URLs...

This is appreciated.

Dali

Thomas Steur
Owner

So we can close this ticket?

gaumondp gaumondp added this to the 2.3.0 - Piwik 2.3.0 milestone
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #4253 as suggested by dalidev I specified a timeout for the auto…
…complete request (20seconds). If it is not loaded after 20 seconds
e81d7e1
Philip Taffner sabl0r referenced this issue from a commit in sabl0r/piwik
Thomas Steur tsteur refs #4253 slightly decrease the limit as there will be only 30 value…
…s kept anyway
ea91d38
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.