Add more Prometheus metrics

[yuvipanda]

Now that #3490 has been merged, we should add more prometheus metrics to the notebook server!

Some ideas for metrics to add...

1. Number of kernels running (labeled by type)
2. Number of sessions open (not sure if this is useful?)
3. Number of terminals open
4. Mirror of the activity tracking metrics
5. Kernel start / stop latency metrics
6. 0mq metrics

Am sure there's more that I don't know of!

[yuvipanda]

Can someone with rights mark this as 'good first issue'? :)

[dhirschfeld]

I'm interested in RAM - per kernel and total. Not sure if that's already available though?

[rgbkrk]

There's some light amount of collaboration around RAM on a kernel level in jupyter/jupyter#264, though it's on a spec level (especially since the actual kernel make be several child processes deep). @ivanov would probably enjoy having a collaborator on the Python side -- I'm looking forward to the UI portion of using it. The notebook server can use it as well since it has access to the messages as it transports them from ZeroMQ to WebSocket.

[yuvipanda]

@dhirschfeld @rgbkrk we could possibly also do it from the default Kernel Manager, since it is just spawning local processes and knows how to collect metrics for them (and their children). This lets other kernel managers report their own metrics as they wish, and works across all kernels without any extra work. It would be complimentary to jupyter/jupyter#264.

[GoelJatin]

Hi @yuvipanda ,

If no one is working on this, then I would like to take it up.

I would like to start with the very first one for now.

But have a few doubts.

Do we want the number of kernels running at the time when the API is called / keep on collecting them over the complete period of time since the notebook server was started?

Please let me know accordingly.

CC @Madhu94

[GoelJatin]

Guys,

Any update on this?

[manuhortet]

Hey @GoelJatin, I'm up to work on this with you.

I'm interested in RAM - per kernel and total. Not sure if that's already available though?

As I think this will be the most useful metric to add, may I take it? :)

Will try to do it from the default Kernel Manager, as mentioned by @yuvipanda

[GoelJatin]

Hey @manuhortet , sure go ahead.

No concerns from my end. :)

[LiryChen]

Hey, I am a first-timer looking for tasks to do too. I found this issue pretty interesting. Anything I can help with?

[konnermacias]

@manuhortet Have you been able to make any progress on adding that metric? I'm a first-timer as well and would love to help out!

[manuhortet]

Hey! Sincerely I've been delaying some OS contributions in order to gain time for personal projects. I'm sorry I delayed you two too doing that! You can take this issue if you want to. In fact, feel free to ask me if you face any problems. Good luck! @konnermacias @LiryChen