Skip to content

Commit

Permalink
[docs] Share Cloudprober's journey through an About page. (#439)
Browse files Browse the repository at this point in the history
* Add blog-meta shortcode to add blog-like metadata for such pages.
  • Loading branch information
manugarg committed Sep 5, 2023
1 parent 811adcf commit a07e5f7
Show file tree
Hide file tree
Showing 4 changed files with 104 additions and 2 deletions.
4 changes: 4 additions & 0 deletions docs/assets/scss/common/_custom.scss
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,7 @@ a.docs-link {
.doks-navbar {
background-color: #f0f0f0;
}

.nav .nav-item {
text-transform: uppercase;
}
10 changes: 8 additions & 2 deletions docs/config/_default/menus/menus.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,23 @@
url = "/docs/config/overview"

[[main]]
name = "How-Tos"
name = "How-To"
weight = 60
identifier = "how-to"
url = "/docs/how-to/external-probe"

[[main]]
name = "Exporting Metrics (Surfacers)"
name = "Surfacers"
weight = 70
identifier = "surfacers"
url = "/docs/surfacers/overview"

[[main]]
name = "About"
weight = 80
identifier = "about"
url = "/docs/about/"

[[social]]
name = "GitHub"
pre = "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"20\" height=\"20\" viewBox=\"0 0 24 24\" fill=\"none\" stroke=\"currentColor\" stroke-width=\"2\" stroke-linecap=\"round\" stroke-linejoin=\"round\" class=\"feather feather-github\"><path d=\"M9 19c-5 1.5-5-2.5-7-3m14 6v-3.87a3.37 3.37 0 0 0-.94-2.61c3.14-.35 6.44-1.54 6.44-7A5.44 5.44 0 0 0 20 4.77 5.07 5.07 0 0 0 19.91 1S18.73.65 16 2.48a13.38 13.38 0 0 0-7 0C6.27.65 5.09 1 5.09 1A5.07 5.07 0 0 0 5 4.77a5.44 5.44 0 0 0-1.5 3.78c0 5.42 3.3 6.61 6.44 7A3.37 3.37 0 0 0 9 18.13V22\"></path></svg>"
Expand Down
91 changes: 91 additions & 0 deletions docs/content/docs/about/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: "About Cloudprober"
lead: ""
draft: false
images: []
weight: 300
date: 2023-08-20
lastmod: 2023-08-20
contributors: ["Manu Garg"]
---

{{% blog-meta author="Manu Garg" publishDate="Sep 05, 2023" readingTime="3 min" %}}

## Origin

I started working on Cloudprober in 2016. I was leading the Cloud Networking SRE
team at Google at that time. As Google Cloud started to grow big time, we
started running into observability issues all over. Customers were finding
problems before us, resulting in a lot of time being spent in troubleshooting
and communicating back and forth[^1].

[^1]:
A customer-reported infrastructure issue is much harder to debug than an
issue discovered by your own observability stack.

Problem was that Google's existing observability tools didn't work for the
external Cloud. We needed to build things from ground up. As probers[^2] are
pretty much the foundation of the reliable observability at Google, we decided
to prioritize the development of a prober for Cloud. That's how the journey of
Cloudprober began.

[^2]:
Almost all of Google's systems rely on probers to detect customer facing
problems.

Even though the primary goal of Cloudprober at that time was to discover and
alert on Cloud Networking availability and performance problems, we decided to
develop it as a generic prober that could be used to monitor a wide variety of
systems and services. We also decided to make Cloudprober open source so that a
wider community could trust it, contribute to it, and run it on their systems.

Keeping Cloudprober generic and ready for open-source paid off. More and more
Cloud teams started using it internally, and our open-source readiness
commitment made sure we kept our interfaces clean.

## Built for Scale

Cloudprober was built to probe 100s of 1000s of endpoints (IPs and HTTP URLs),
while keeping resources, and more importantly, management overhead very very
low. That's the reason Cloudprober tries to be frugal with the resources,
maximizes resources utilization relying heavily on Go concurrency, supports
probing large number of targets in parallel at a high frequency (in
milliseconds), minimizes the need of frequent rollouts by supporting dynamic
targets discovery, has native implementations for common probe types, and so on.

## Beyond Google and Open-Source

During its early days, our priorities for Cloudprober were scalability,
reliability, and ease of management. Things began to change as more and more
Cloud products started using it, and users started asking for more features.
However, the real shift to features happened after we open-sourced Cloudprober
in 2017.

We added a wealth of features over time, such as first-class Kubernetes support,
a built-in probe status UI, PostgreSQL and Cloudwatch surfacers, OAuth support,
Validators, and most recently, built-in alerting capability. We used the same
codebase for the internal and open-source versions, which created a huge
advantage -- we had an extensive deployment internally which provided a
continuous testing platform for Cloudprober, particularly for its scalability
and performance aspects, while it was going through all the big changes.

## Move away from Google Github

I left Google in Nov 2021. To keep working on Cloudprober independently, I moved
Cloudprober's Github repository from
<a href="https://github.com/google/cloudprober">github.com/google/cloudprober<a>
to
<a href="https://github.com/cloudprober/cloudprober">github.com/cloudprober/cloudprober</a>.
This was a disruptive move (and we lost a lot of Github stars in the process
:smiley:), but it had to be done one day anyway, in order for Cloudprober to
become an independent entity and grow even faster. While I can't say this
authoritatively now, I believe Google still uses Cloudprober, possibly even more
widely now, based on Googlers' interactions with the project.

## Growth and stability

Throughout its journey, Cloudprober has continuously adapted and expanded to
meet the evolving needs of its users -- an essential trait for any software.
Without this attribute, software inevitably withers over time. To ensure
Cloudprober thrives and evolves robustly, we've been very diligent that it grows
in a structured way, a commitment we'll uphold in future as well.
1 change: 1 addition & 0 deletions docs/layouts/shortcodes/blog-meta.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<p><small>Posted&nbsp;on&nbsp;{{ .Get "publishDate" }} by {{ .Get "author" }}&nbsp;&hyphen;&nbsp;<strong>{{ .Get "readingTime" -}}&nbsp;read</strong></small><p>

0 comments on commit a07e5f7

Please sign in to comment.