implement enhance-k8s-metadata plugin with label reading #33
Conversation
| def filter(tag, time, record) | ||
| decorate_record(record) |
There was a problem hiding this comment.
time not used here? Or it is inside record?
There was a problem hiding this comment.
it's inside the record. time here is only for showing relative message in fluentd log
| log.debug "Record doesn't have [#{@in_namespace_path}] field" | ||
| elsif pod_name.nil? | ||
| log.debug "Record doesn't have [#{@in_pod_path}] field" | ||
| else |
There was a problem hiding this comment.
So we will only get label for pod, but not for other objects?
| # parameters for read/write record | ||
| config_param :in_namespace_path, :string, default: '$.namespace' | ||
| config_param :in_pod_path, :string, default: '$.pod' | ||
| config_param :out_root, :string, default: 'kubernetes' |
There was a problem hiding this comment.
Is this part even configurable?
There was a problem hiding this comment.
we probably need it's configurable for reuse it on metrics/log. I don't think it's changeable if we want application work
| @kubernetes_url = "https://#{env_host}:#{env_port}/api" unless env_host.nil? || env_port.nil? | ||
| end | ||
| log.info "Kubernetes URL: '#{@kubernetes_url}'" | ||
|
|
There was a problem hiding this comment.
Could you explain a bit how authentication works? Where does customer get certificates or bearer token?
There was a problem hiding this comment.
These files/tokens/env are injected by k8s when containers are launched so the processes inside of containers can talk to k8s-apiserver, k8s-dns, services, etc.
In a typical cluster, customer don't need to setup any of them, they are automatically generated.
To look into it, a simplest way is running into the containers:
kubectl exec -it <pod-name> /bin/sh
| def init_cache | ||
| @all_caches = { | ||
| CACHE_TYPE_POD_LABELS => LruRedux::TTL::ThreadSafeCache.new(@cache_size, @cache_ttl), | ||
| CACHE_TYPE_OWNER_REFS => LruRedux::TTL::ThreadSafeCache.new(@cache_size, @cache_ttl) |
There was a problem hiding this comment.
What does owner refs do? Don't see any usage here?
There was a problem hiding this comment.
Planning to use it later for deployment lookup. I can remove it for now
| cache = @all_caches[CACHE_TYPE_POD_LABELS] | ||
| labels = cache[key] | ||
| if labels.nil? | ||
| labels = fetch_pod_labels(namespace_name, pod_name) |
There was a problem hiding this comment.
A general question: it's a bit hard to follow this kind of function call in another module. Is it by convention people ignore the class reference? Would be much easier to follow if we know where doesn't that function come from. Also what if required modules overload a method, which one would be used?
There was a problem hiding this comment.
You can consider the module is another approach to do something like writing util functions in trait in Scala but more flexible. (e.g. duck typing) It's more natural way to reuse code in Ruby rather than using inheritance.
For connivence, if you install Ruby plugin in VS code (and some dependency gems following the instruction), it can support Cmd+Click to jump between functions from different modules
| module SumoLogic | ||
| module Kubernetes | ||
| # module for connecting to Kubernetes cluster | ||
| module Connector |
There was a problem hiding this comment.
is it possible to re-use it in the events plugin?
There was a problem hiding this comment.
It's possible, but need to split the code into a mixin so different plugins can refer to.
Or you can just copy the code into your plugin first with same module names and later we can merge and change the dependencies
@lei-sumo @samjsong
FYI - @maimaisie @yuting-liu @rvmiller89