Network transform: Align kubernetes transform with kube-enricher

[eranra]

Align the network transform functionality of flowlogs2metrics with https://github.com/netobserv/goflow2-kube-enricher so that the information and fields added by the network transform phase are equivalent. This will allow smooth integration with NOO.

[jotak]

I had a look at FL2M code, I don't think we necessarily want to map exactly FL2M fields to the ones from goflow-kube, we can do some adaptation in the loki queries and console plugin, for instance some hard-coded stuff like the suffix "_Name", "_Type" etc. are different but we can keep the FL2M version. There's still some additions to bring, here's what I can list:

• add something like "_Owner" and "_OwnerType". They would be filled only for Pod types. The logic is here, basically it needs a new ReplicaSet informer.
• add something like "_HostIP" or "_NodeIP" for Pod types (not relevant for other types).
• about the kube labels, this is certainly interesting, but I guess it could also potentially impact a lot on storage, don't you think? Because we don't control how many labels people will have on their workloads. Maybe it is wise to make it configurable (enable/disable) ?

[jotak]

Just thinking about another gap with goflow-kube. In gfk, for Loki export, we index on the Src/DstWorkload fields, which almost maps to "*Owner" field in my suggestion above. Except that, for Services (and Nodes), we won't have any index, whereas in gfk services were written in the same Workload field, hence was included in the index.

So, thinking out loud for a workaround in fl2m: we could always fill that Owner field:

• For Pods with an owner, it's that owner (or super-owner for replicasets)
• For Pods without owner => copy pod name as owner
• For Service => copy service name as owner
• For Node => copy node name as owner
• For unmatched IP => only case without an owner

Maybe this is cheating a little bit with the concept of "owner", but I think it's important to have this "reasonably aggregated" index in Loki

[eranra]

@jotak I added #88 #89 #90 to capture the above gaps.

With regards to your comment above (#76 (comment)) --- can this be front-end work and not back-end. I do not think that adding to Loki information with is not accurate provides any value. I think that the other way around, we will get search results that are not very reasonable

[eranra]

@jotak I ended up implementing the logic for Owner here: https://github.com/netobserv/flowlogs2metrics/pull/91/files#diff-6d907b530ec4b15f1e32b17d5ba122ec547cc5fb3bcf4d709a3d21a7864cfc3bR108

Should be similar to what you had in mind :-)

[eranra]

completed :-)