Repository navigation
Release v0.3.179
Summary
Outside Kubernetes the network stream folded every container's traffic onto the single node entity, so no per-container entity existed and per-workload attribution was impossible. This keys entities by container ID instead — collapsing onto the node only for traffic that is not a container's — and creates the entity on first sight where nothing announces containers. The Kubernetes path is unchanged.
Overview
handleNetworkEvent and handleDnsEvent decide which stream entity a connection belongs to. Today the fallback onto the node entity fires whenever ns.k8sObjectCache == nil:
if entityId == "" || ns.k8sObjectCache == nil {
entityId = ns.nodeName
}That cache is a Kubernetes API cache. In this repo's binary it is always present, but embedders that run the sensor outside a cluster construct the stream with a nil cache — and there every event, including every container's, is folded onto the single node entity. No per-container entity is ever created, so every container on the machine is indistinguishable in the stream and per-workload attribution downstream is impossible.
After this change the collapse happens only for traffic that is not a container's: an empty container ID, or the "host" sentinel that procfs.go assigns to processes outside every tracked mount namespace. Everything else keys by container ID.
Keying alone is not enough, because an event for an entity that does not exist is dropped, and outside Kubernetes nothing subscribes this package's ContainerCallback — so no container entity is ever announced. When kubernetesMode is off, the entity is therefore created on the container's first event, and pruned once it goes an interval without traffic, so a machine with container churn does not accumulate dead ones.
In Kubernetes it keeps dropping, and that is deliberate. A missing entity in a cluster does not mean "never announced"; it means the container was removed or is ignored. Removal is not quiet, either: the container collection publishes EventTypeRemoveContainer before the container leaves the collection and then holds it in a 2-second cache for enrichers, while the watcher dispatches this callback onto a worker pool. So events for a container that is already gone keep arriving for a while afterwards. Creating entities for those would put a container carrying no pod or workload identity into the payload, for a pod that no longer exists, on every pod termination.
Two smaller fixes ride along:
- Traffic recorded before a container is announced is no longer thrown away.
containerCallbacksubmits its subscribers to a worker pool while events flow straight through, so a container's first outbound connection routinely lands before itsAddContainerreaches this package. The announcement now fills identity in over the event maps already there instead of replacing them. The node entity is excluded — it is not announced, so it has no such window, and excluding it keeps that path byte-identical. - The
k8sInventorylookups inbuildNetworkEventare nil-guarded. The inventory is only built whenkubernetesModeis set, so a pod- or service-kind endpoint arriving anywhere else would dereference a nil interface.
Additional Information
Effect on Kubernetes: none. The removed k8sObjectCache == nil disjunct never fired there, and every new behaviour is either behind !cfg.KubernetesMode or scoped away from the node entity. TestHandleNetworkEvent_KeyingIgnoresK8sObjectCache and TestHandleNetworkEvent_KubernetesStillDropsUnknownEntities pin that.
One known gap is deliberately not addressed here. ContainerCallback deletes a stopped container's entity along with any traffic it had not yet flushed, losing up to one interval — which matters most for a short-lived container that makes a single outbound connection. That behaviour is pre-existing and untouched by this PR: it is neither introduced nor widened here. But per-container entities make it matter more, so I will follow up with a separate PR that retires the entity at the next flush instead of deleting it. I am keeping it separate because that one is a Kubernetes behaviour change and deserves its own review rather than riding inside a change whose point is that Kubernetes is unaffected.
Docs: adds docs/features/network-stream-entity-keying.md. Also drops hostNetworkSensorEnabled and hostMalwareSensorEnabled from docs/CONFIGURATION.md — neither key exists anywhere in the codebase, while hostMonitoringEnabled, which does, was missing from the table.
How to Test
This package does not build on macOS, since inspektor-gadget is Linux-only. On a Mac:
docker run --rm -v "$PWD":/src -w /src -v "$(go env GOMODCACHE)":/go/pkg/mod \
golang:1.25 bash -c 'go test -race ./pkg/networkstream/...'
On Linux, go test -race ./pkg/networkstream/....
15 new tests (18 cases with subtests) cover the keying, the platform gate, entity creation and pruning, the announcement merge in both arrival orders, node-entity restoration, and the nil-inventory guard. I checked each one earns its place by reverting the corresponding behaviour individually and confirming that test — and only that test — fails. The rest of the unit suite is unaffected.
Summary by CodeRabbit
-
New Features
- Network and DNS activity is now attributed to separate container entities.
- Non-Kubernetes environments can create temporary container entities from observed traffic.
- Host activity is consistently associated with the node entity.
- Container metadata and activity recorded before announcement are preserved.
- Network endpoint enrichment safely supports pod and service details when inventory is available.
-
Bug Fixes
- Unknown Kubernetes entities continue to be excluded.
- Idle, unannounced container entities are automatically removed.
- Snapshot and shutdown handling now safely isolate and flush collected data.
-
Documentation
- Updated the host monitoring configuration reference.
- Added documentation for network-stream entity keying and lifecycle behavior.