Analytics - allow parameter parsing

[shawnjohnson]

This might be a bit optimistic of a request, but it would be great if we could parse out the parameters and their values from requests to add more value to the analytics reports. For example, we include "documentId" as a parameter, and it might be nice to aggregate based on that value to illustrate a popular document.

[GUI]

Thanks for the suggestion. We actually are collecting the underlying data that would allow us to easily aggregate on this, we just don't have it exposed in the UI. Do you have any ideas on what sort of output you'd find useful for this type of report? Or would something as simple as selecting a URL query parameter and then viewing a list of the top 100 items and their associated hit counts work?

If something like that simple list would work, I'd be happy to pull together those numbers for you now and send them your way. Just let me know the date range and any other criteria you're looking for. If we can get you the information you want manually, then that might also help us figure out what type of UI and report would work best for this.

For reference, all GET query parameters are being indexed as nested fields under request_query. Do you could perform advanced filters for queries like request_query.documentId:WHATEVER, but that obviously only filters and doesn't aggregate in the way you're looking for. I guess I never documented the request_query parameter, so I'll at least get that documented.

This is also an area where Kibana could be interesting to explore again to provide a better analytics interface for these types of arbitrary aggregations. I'm still not convinced it's a great solution for what we're trying to do in our admin (I'm not sure how we would integrate it with our new admin permissions where you can only view some subset of the stats, and last time I checked you could perform arbitrary, resource-hogging queries), but it's probably worth looking into again with our upgrade in elasticsearch and the new kibana 4 beta.

[shawnjohnson]

Starting with a manual export might be a good start. I was thinking along
the lines of Kibana, which we are using with our logs, as well as Google
Analytics. Currently we are sending a GA event which includes key
parameters. So, one example might be to see the top XX most requested
documentId's. Same could be say for our /docket and /comment endpoints.
If those could plot in time-based charts, similar to the current charts -
that expands to the capability to something like top 10-most popular
documents by day.

On Thu Oct 30 2014 at 12:57:56 PM Nick Muerdter notifications@github.com
wrote:

Thanks for the suggestion. We actually are collecting the underlying data
that would allow us to easily aggregate on this, we just don't have it
exposed in the UI. Do you have any ideas on what sort of output you'd find
useful for this type of report? Or would something as simple as selecting a
URL query parameter and then viewing a list of the top 100 items and their
associated hit counts work?

If something like that simple list would work, I'd be happy to pull
together those numbers for you now and send them your way. Just let me know
the date range and any other criteria you're looking for. If we can get you
the information you want manually, then that might also help us figure out
what type of UI and report would work best for this.

For reference, all GET query parameters are being indexed as nested fields
under request_query. Do you could perform advanced filters for queries
like request_query.documentId:WHATEVER, but that obviously only filters
and doesn't aggregate in the way you're looking for. I guess I never
documented the request_query parameter, so I'll at least get that
documented.

This is also an area where Kibana
http://www.elasticsearch.org/overview/kibana/ could be interesting to
explore again to provide a better analytics interface for these types of
arbitrary aggregations. I'm still not convinced it's a great solution for
what we're trying to do in our admin (I'm not sure how we would integrate
it with our new admin permissions where you can only view some subset of
the stats, and last time I checked you could perform arbitrary,
resource-hogging queries), but it's probably worth looking into again with
our upgrade in elasticsearch and the new kibana 4 beta.

—
Reply to this email directly or view it on GitHub
#136 (comment).

[mgwalker]

Closing as stale, on the assumption that the need will resurface if it still exists.