-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Metricbeat] Add cluster identifier to Kubernetes metadata #17467
Comments
Pinging @elastic/integrations-platforms (Team:Platforms) |
I totally agree we need a field like this. This has been lightly discussed in the past, the biggest issue is that Kubernetes clusters don't have a name you can access through the API (afaik, this may have changed). We need to figure out what would be a good alternative that provides a unique value per cluster that is recognizable by users. One possible option would be to use the API Server URL. Do you folks know a better alternative? this may require some research |
Yeah I do not believe kubernetes itself really knows. Maybe the cloud metadata processor could add it as it is specific per cloud: eg. GKE |
Sorry for the long comment, I wanted to provide some context on previous discussions and current options and it got a bit out of hand 😬 I remember discussing about this in the past, also about other services. Similar problem appears with almost any other service (mysql, kafka...), you may have several clusters of the same service reporting to the same ES cluster. In most of the cases I would say that services don't have something like a cluster name. Historically the only solution for that has been to use custom fields, that can be set at a global level or at a module level. After adding these fields you can take them into account by adding filters when querying the data or when visualizing it in Kibana. When migrating to ECS we thought about this problem and we added the For kubernetes we would need something else in any case, because we may have a mysql cluster, deployed in a kubernetes cluster, so we cannot use the
I don't think clusters have a name by themselves, but I have just discovered that there is a
From the point of view of beats deployed in Kubernetes the API server URL is always going to be the same, isn't it? I think that whatever we do we are going to need to expose this through the config so users can set any value they want (as we do with custom fields or with |
This config map is created by kubeadm, and it doesn't exist in GKE clusters, so I guess we cannot rely on it by now https://github.com/kubernetes/community/blob/master/contributors/design-proposals/cluster-lifecycle/bootstrap-discovery.md#new-kube-public-namespace |
Thank you for all the background explanation @jsoriano! It's good to see that we can find a human readable name in some cases. WDYT about something like this?:
The UI must always use |
Field names LGTM, but we would still have to see how to obtain a unique URL in the different deployment methods of Kubernetes. If this field is going to be the expected identifier in UI we should still provide a way to set it up manually for cases that are not automatically supported. Regarding other orchestrators, do you think we should find a common solution? or we decide on them when/if we have UIs or dashboards where we need a convention. |
What about defining those fields in an ECS format so they are same no matter the orchestrator as @jsoriano mentioned above? Kubernetes
Cloud Foundry
Nomad
|
that sounds like a good idea to me! I think that should be part of a broader conversation about how we want to expose the different orchestrators and if we can map their constructs into common fields |
This came again today, I think that adding Once we reach the orchestrator layer in the inventory schema discussions we can revisit how this should look for all of them. |
Is there any update about this feature. We have the same scenarios where we have multiple Kubernetes clusters that we want to monitor centrally and having this metadata available will be very beneficial. Thanks! |
FYI a standardisation of orchestrator fields is in progress on ECS, see elastic/ecs#1230 |
Now that we have this in ECS we plan to move ahead and add these fields, @ChrsMark I'm assigning you for now 🙏 |
Today there's no way in beats to distinguish between multiple Kubernetes clusters. We can monitor and visualize pods, containers, but there's no way to see the full picture:
Kubernetes cluster 1 -> Nodes -> Pods -> Containers
Kubernetes cluster 2 -> Nodes -> Pods -> Containers
As a DevOps engineer I want to be able to easily identify which cluster my nodes and pods belong to so that I could have a holistic view of all my Kubernetes deployments irrespective of where they're deployed.
cc: @exekias, @jsoriano, @ChrsMark
The text was updated successfully, but these errors were encountered: