Monitor network connections

[vpetersson]

In order to determine suspicious network activities, we need to be able to sample the network connections from a given device. In a perfect world, we'd be able to monitor all network connections, but this is unrealistic, so sampling is the next best thing.

A proof of concept illustrating this would be to say run netstat every 60 seconds (with a jitter) and record these transactions. We can then log the connections and submit them upstream for processing.

[a-martynovich]

@vpetersson I don't get why you call network monitoring unrealistic. There's libnetfilter_conntrack designed just for this purpose and there are at least two Python bindings for it: https://github.com/mk-fg/conntrack-logger and https://github.com/ei-grad/python-conntrack.

I believe sampling every 60s is kind of useless in terms of security. If something spoofs data to a remote server it will do that quickly, and sampling might not catch it. And also it's a lot of data to process.

[vpetersson]

@a-martynovich I'm open to exploring. My thinking was simply that doing real time processing will be too expensive both in terms of bandwidth and CPU. Keep in mind that the agent can consume close to no resources to not be in the way of the other workload.

[a-martynovich]

@vpetersson Unless the device is doing heavy p2p networking there shouldn't be too many connections happening. We only listen for opening/closing of connections, right?
We can delay the processing of connection list (like packing and sending to the API server), but we should gather the connections continuously. This also means that agent (or a spawned part of it) should run continuously. And I offer the same for #31 and #20 .

However I understand that what I'm offering is a bit more work, so sampling is an easier starting point. The connection list should arrive with /ping request, right?

[vpetersson]

@a-martynovich yeah let's start with sampling for now and view it as an area we know we need to improve. If you're curious, take a look at this paper that gives you a good overview of the problem and data points we might want to capture.