Welcome to o11y weekly (observability weekly!).
Here is the latest news / feedbacks about observability.
I wanted to share my experiment since I am using observability tools for more than a decade! I started with graphite in 2013 thank to Arnaud Porterie (IIRC, he was leaving France to join Docker in the docker engine team).
A friend suggested me to write blog posts about my own experiments, hands-on but also news in the observability.
This week, for the very first post, lets talk about controversal savings dilemma in Observability I was facing.
This post introduces the concept of the o11y/observability weekly and a serie that will come about using efficient solutions and/or scale effective one.
To me, making savings is part of our IT business in search of Efficiency.
So my suggestion for observability in big companies is to focus first on scalable effective solution by secretly using efficient one if you are experienced in and does not impact timing/costs.
Depending the context, making an Efficient solution by making savings from Effective one seems to be a bad idea !? It should trigger a WTF; well, wait a moment and bare with me to look it in details.
The power of efficiency is the next level of effectiveness.
It is important to tackle first effectiveness before having efficient tools/solutions.
I worked on start-up / scale-up / small and big companies including Snapchat where scalability / stability is really important.
A good general rule is to make it effective then efficient to scale at lower cost and reduce for instance the cost per user when you are working on a free social network.
In social network, you measure important KPIs like DAU Daily Active Users DNU (Daily New Users) MAU (Monthly Active Users), then to support high growth you need to have efficient solution to make the cost per user stable and lower.
In such companies where the value is the density and cost of the network to make money later, it is very important to keep costs low per user and that is why Efficiency is really important.
In my experience, I worked for a social network company and independant hotel industries where such properties are really important to make business.
I had recently a discussion with a friend experienced about making business and stated:
"You do not scale your business by making savings but by finding fund and investors!"
This is a fairly good point! I have to admit that scaling effective solution just works! If costs are not a problem, then there is no need to be efficient since, it can be hard and expensive.
This is why making platform cost savings (cloud / observability) is an unpopular/critized topic for big companies or simply company that can scale cost easily.
In big companies, a team having a big budget generally hates savings because if they make it, they will have less budget for the next year.
This idea seems counterintuive first so this is why it is important to avoid the idea of savings but talking about performance and usage !
When usage is high and relative to the business, it is really easy to scale the platform.
The platform should scale easily: scaling from 5 to 50 servers should reduce response time and the usage should be balanced over 50 servers.
Lets imagine that the platform is using 100% 5 servers but not using the 45. Well, there is a scalability issue where the load/data cannot spread easily.
So having effective solutions cannot really helps to scale and finding hotspot is really important.
So lets scale the effective solution!
That is why in big companies, they focus on stability and effectiveness over efficiency, but, the thruth is that efficient solutions are better because they scale really easily and at lower cost.
So my suggestion for observability in big companies is to focus first on scalable effective solution by secretly using efficient one if you are experienced in and does not impact timing/costs.
Lets start a serie of posts and hands-on about scalable effective and/or efficient solutions.
Next planned week topics :
- What is not an o11y/observability solutions (QoS/SLA) ?
- How to monitor costs on your Grafana Cloud instance ?
- ...