Document internal pieces of the Prometheus server and how they fit together

[juliusv]

The Prometheus server is a big beast with many subcomponents, which can be hard for new contributors to get into. I would like to see documentation that zooms in to the Prometheus piece of https://prometheus.io/assets/architecture.svg and explains for developers how all the different pieces fit together:

• config
• SD
• scraping
• TSDB
• rules
• alert notifications
• etc.

This should include a diagram of how the pieces interconnect, what information they exchange, and then a natural language explanation of each piece and its assumptions. Where necessary, it could link to more detailed documentation about a specific piece, like https://github.com/prometheus/prometheus/blob/master/discovery/README.md.

[krasi-georgiev]

👍
Is this going to dive in actual code or just more detailed view about each piece in the architecture?

I am asking as I have noticed that any code references and components get outdated very quickly.

[juliusv]

@krasi-georgiev I was thinking it might link to actual code and maybe mention some of the used types or interfaces where appropriate, but it shouldn't go into too much detail itself. Still, it might get outdated, so links should go to a particular Prometheus revision and the doc should state that the explanations are based on that, so that people can also see how updated or outdated the doc is.

[krasi-georgiev]

yep that should work!
ping me when you have the google doc so I can follow and nag you with questions 😜
Maybe I can also help with some parts.

Here is a deep dive in the TSDB , started by @gouthamve and than I also started adding different bits.
https://docs.google.com/document/d/1t5eSLlH36oK1Ex3PWwuc4R58kx4PC7Ago90TRR_7nmo/edit#heading=h.9fse7yo9k0j6

[juliusv]

@krasi-georgiev Thanks! Oh yeah, if you could help flesh out / finalize / website-ize the TSDB documentation, that would be great. That probably makes sense as a separate item with a lot of detail, that this main overview can then link to.

[RichiH]

Sounds sane; if you need to point at docs that would be good to have, I think we should simply create placeholders for that to flesh out later. If that means the official docs are interspersed with a few TODO for a short time, I think that's OK. We should have a link in there to point out how people can help, but else...

[lucperkins]

@RichiH @juliusv I have this on my radar. No ETA just yet but I'll be sure to prioritize it.

[juliusv]

@lucperkins My plan so far was to write this one myself, or do you think it's better if someone new to the project does it? Of course it'd be great to get a review in any case.

[lucperkins]

@juliusv @RichiH I've got this on my TODO list.

@krasi-georgiev One possibility might be something like this "Code Organization" doc that I put together for the Apache Heron documentation. Basically a high-level intro to the repo that points out important subdirectories and places in the code. Since it's mostly Go, links to the GoDoc could be applied liberally as well. Would you find that useful?

[juliusv]

@lucperkins Oh ok! So currently I have a work item open with the CNCF for me to do this and with much more detail planned than in https://apache.github.io/incubator-heron/docs/contributors/codebase/. It would dive into more detail about how each component works, how it connects to other components, and what assumptions there are (which might be difficult to derive from just reading code). Do you feel strongly on taking this on though?

[lucperkins]

@juliusv I'm happy to leave that up to you, of course. One possibility could be that you give me a rough "braindump"-style Google doc and I can spin that into a more full-fledged doc plus the "drilldown" diagram that you mention. Also happy, of course, to stay out of your lane and provide a review!

[juliusv]

@lucperkins Sounds good to me, let's collaborate on it! I hope to start working on this by next week, maybe even before. What do you usually use for diagrams? We have been using https://www.draw.io/ so far, which maybe isn't as shiny as some desktop tools, but has the advantage that everyone can update diagrams no matter what their Desktop OS is, and without needing to install expensive software.

[lucperkins]

@juliusv Awesome. I look forward to it. I've used a ton of things for diagrams (OmniGraffle, LucidChart, Sketch, etc.). I think draw.io should be just fine for something like this.

[krasi-georgiev]

@lucperkins when I am studing the tsdb code I had a big problem to understand what each piece means and how they interact between each other.

for example
what a block is?
what is a chunk
what is the head
.....
the list is long.

I don't think that linking to code is good strategy as it changes so often that it will get outdated very quickly. OTOH what each piece does and how they interact remains constant.

I am hoping to have just a glossary of what each component means with a very brief description as well as a Architecture overview similar to what Prometheus has. That has been quite useful and it is valid even now after so many code changes.

The document I linked is an attempt for this but it goes into more details than we need.
I am still kind of updating from week to week and eventually should have something easy to understand Main problem is that the more I understand the code the less often I need to go back to this document to get it updated, but maybe with your help we can complete it.

[lucperkins]

@krasi-georgiev Agreed re: linking directly to code. GoDoc links are more sustainable, as the names for important structs/interfaces/constants/etc seem to be largely unchanging in the Prometheus code.

[juliusv]

Good idea, will keep that in mind for the public interfaces. Sometimes you can't avoid linking to code (a specific revision then) when it's about unexported internals that are important for devs.

@lucperkins It'd be great to get your help to get the TSDB doc that @krasi-georgiev linked into shape!

[lucperkins]

@juliusv @krasi-georgiev Let’s get it done 👍🏻

[krasi-georgiev]

please ping me next week for the TSDB

[henridf]

@juliusv if you need more cycles i'd be happy to contribute, whether it's reading/editing, or writing portions relative to the pieces i'm more familiar with (which would currently be remote storage and promql/).

(It sounds like you already have a couple cooks, so totally ok if you'd rather not spoil the broth!)

[juliusv]

@henridf Yep thanks! Let me get the initial version out first, and then everyone is free to add their broth ingredients to it :)

[juliusv]

@lucperkins @brian-brazil @RichiH here's a WIP for this: #4295

Interested in general feedback for now, I will still add more details to the doc and polish it later.

[juliusv]

This was merged.