Add API Requests By User

[dfarr]

This adds a report to collect the top 20 API users each day, and
log the total number of requests each hour. Reports this info in
a simple table, this is handy for identifying users who are
approaching, or over the API rate limit.

An example of the API Rate Limit page:

[larsxschneider]

👋 @dfarr ! I have a few questions:

(1) Was there a specific reason to print the numbers per hour? I am asking because the finest granularity for all other numbers in Hubble is a "day" and I wonder if we should stick to that.

(2) Would it make sense to move your "Top API Users" into the API requests pages?

Thanks for this PR, this looks really useful 👍 😊

[dfarr]

I think it would make sense to do top API users - does regenerating this every day make sense? If so it would be a pretty simple tweak to the script and simply moving the table to a different page :)

[toddocon]

I think generating this report every day makes sense. I have some very heavy API (and polling) users and would like to make this information more visible.

Question, does this report still generate if rate limiting is not enabled? I assume this only makes sense if rate limiting is on. There are other logs and methods to pull out the heavy polling users. That's what I use now.

[larsxschneider]

Yeah, top API users per day makes total sense. @dfarr Do you want to tackle this?

@toddocon AFAIK this should work with and without rate limiting enabled.

[dfarr]

Will do 👍

From my understanding, the api-requests-by-user.sh script will extract the data regardless of whether or not rate limiting is enabled. Rate limiting information does surface in some fields of the unicorn logs, but these fields are ignored by this script.

[dfarr]

Updated! Here is a screenshot of the updated housekeeping/api-requests page:

[toddocon]

I like it! Can't wait to get this in my version.

[pluehne]

@dfarr: Thank you very much. That’s a very nice addition to Hubble, and your code is well-written! I just have two minor remarks below that you could address 🙂.

[pluehne]

Could you add a short description of this report as a comment, as we have with the other reports?

[pluehne]

I’d drop the “Top” in the title, because it’s common for our charts only to show the top entries (for instance, in the Git traffic table).

[larsxschneider]

@dfarr Thank you very much 😃 👍

I rebased your PR and made a few minor modifications. Please take a look: f14daab