New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate per query statistics #263

Closed
jvshahid opened this Issue Feb 19, 2014 · 5 comments

Comments

Projects
None yet
7 participants
@jvshahid
Contributor

jvshahid commented Feb 19, 2014

More specifically, for a given select query:

  1. Time took to run the query (possibly slow queries can make it to a slow log)
  2. Memory consumed by a given query (probably just an estimate)
  3. Shards hit by a given query
  4. Local aggregation flag
  5. Number of points read from each shard
  6. Number of servers hit
  7. Number of points returned from each server
@elcct

This comment has been minimized.

Contributor

elcct commented Mar 9, 2014

I have implemented EXPLAIN command that works with any SELECT queries

Example:

explain select value from testing limit 20

Will return:

explain

shard_local says if query was executed on a local server or through remote call
points_read says how many points query processor has received from datastore
points_written says how many points query processor has written
run_time milliseconds spent on executing the query

Other example:

explain select count(value) from testing group by time(1m)

Result:

agg2

It can already give some interesting data - like how come empty shard has appeared or that LIMIT clause works per shard...

To an extent this depends on work I did for #294

At the moment it is collecting data from QueryEngine, but I am not sure if data for other engines that participate in SELECT query could be useful

Do you think I should continue with this?

@pauldix

This comment has been minimized.

Member

pauldix commented Mar 9, 2014

Yeah, looks great. Not sure we need the other engines to produce stats. As long as we see the same metrics regardless of query, which I think it will because at some point it goes through the engine?

Can you push to a branch and submit a PR so we can have a look?

@Dieterbe

This comment has been minimized.

Contributor

Dieterbe commented Sep 4, 2014

want!!

@jvshahid jvshahid added feature and removed engine labels Oct 9, 2014

@toddboom toddboom added the idea label Nov 25, 2014

@IngaFeick

This comment has been minimized.

IngaFeick commented Feb 24, 2015

+1
Would also be cool if this worked for "list series" commands aswell.

@beckettsean

This comment has been minimized.

Contributor

beckettsean commented Sep 17, 2015

closing in favor of #2044

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment