-
-
Notifications
You must be signed in to change notification settings - Fork 203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE IDEA] improving ports <1024 and >=1024 #59
Comments
Your proposal has been integrated. -B now swappes flows if if src port < dst port. II agree, that it has little impact and brings an advantage. To implement minport, this needs a bit more work. It's on the todo list |
Just an idea for TCP flows: why not track SYN packets and determine who is the server? |
Simple: because Netflow doesn't give you that information. You get one flow from A to B, and a separate flow from B to A, without any information about which had SYN only and which had SYN ACK. The flow start times are probably not accurate enough to deduce it either. (Some devices do bi-directional flows, like ASA NSEL. There's a separate issue regarding the meaning of "in" and "out" in that context) |
The repo code has now an improved handling for swapping flows. -B swaps flows only if:
HighPort traffic is untouched. |
Do you mean it has been rolled back to the old behaviour of -B ? What's the reason? |
I rolled back to 1024 due to many requests having false swaps in high/high port connections. |
Happy to discuss ideas. Obviously in the absence of bidirectional stateful flows, it's only a best guess. Even the current algorithm can be wrong: e.g. I see NFS traffic sourced from low ports, so it might originate from port 750 and connect to port 2049. What reverting the change seems to say is: "there's no point swapping the ports if both the low and high port are over 1024". But services running on high ports (e.g. web servers on 8080) are not uncommon. Maybe it's worth going back to the initial suggestion I made:
This is based on the observation that clients tend to use very high ports for ephemeral ports. |
ok - Agreed and added in repo. |
I think there is an issue in #dce3e36, instead or OR you are using AND, which make the -B option way worse than before. |
I think what you are saying is
should be
Is that right? I think you're right. As it stands, it's impossible for this condition to be true unless src_port is below 1024 and dest_port is above 49152. |
Yes that's what I meant. It seems it was partially fixed in #215 , but I still see some possible issues here: https://github.com/phaag/nfdump/blob/master/bin/nfstat.c#L1728 |
nfdump has this feature:
Unfortunately this doesn't work for services running on higher ports. For example, it's quite common to run a webserver on port 8080, or an NFS server on port 2049.
In theory, all ports below 49152 are available for IANA to assign to services. In practice, the Linux kernel defaults to using ports 32768 and above for ephemeral ports.
So my first thought was to update the algorithm with several tests:
But this begs the question, why not simplify it to this:
That seems to work for the vast majority of cases. It would be rare to initiate a connection from a low port number to a higher port number; either the destination is way up in the ephemeral range, or the source has explicitly bound to a low port number. (Only examples I can think of are peer-to-peer apps and this)
This then suggests another feature. In nfdump you can aggregate by srcport, dstport, or any port. The latter counts each flow twice, once for src and once for dst.
It would be useful to have a new aggregate "minport" to aggregate on the lower of src and dst ports. This would in the vast majority of cases show you the service being used, without the extraneous noise of ephemeral ports mixed in. (A side effect is that flows both to and from that port would be aggregated, so heavy uploads and heavy downloads would both count)
However, I think this can be generalised.
Note that you can also currently aggregate by srcip, dstip or any ip. It would be possible to add new aggregates for the ip address corresponding to the lower port and the higher port in the packet respectively. This would show you traffic in/out of server and traffic in/out of client respectively.
But rather than adding a whole load of new aggregate types like "srvip" and "cliip", I think it would be better to have a single flag which changes the meaning of "src" and "dst" so that "src" is the side with the higher port, and "dst" is the side with the lower port.
Setting this flag, and aggregating on dstport, would give the minport behaviour I described above. Setting this flag and aggregating on dstip would give the total traffic in and out of a given server (i.e. "busiest server"). Aggregating on srcip would give the total traffic in and out of a given client (i.e. "busiest client")
Example:
with this flag becomes:
The text was updated successfully, but these errors were encountered: