Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Improved NetFlowV9 support #21
This PR changes the way templates are handled.
Since for V9 template data flows are not sent with every packet, the implementation must buffer packets until it receives the necessary templates to know how to parse them. The same is true for the option template.
This implementation moves the buffering and template aggregation into a custom codec aggregator, so that the codec itself, which runs after journalling the message, can assume that it has all the templates it needs to successfully parse a packet. This is even more important when processing a journal after a restart.
Thus this implementation does not lose data in the case it doesn't have templates yet. Those are resent regularly by the exporter, for each observation domain.
To be compatible with Graylog 2.3, this change comes with a custom codec aggregator, for 3.0 we can migrate the code back into the server.
Ah shit. Yeah the prefix is broken now. I'll fix it tomorrow…
On Aug 24, 2017 7:59 PM, "Bernd Ahlers" ***@***.***> wrote: ***@***.**** requested changes on this pull request. I am seeing a field nf_nf_field_153 when ingesting v9 via pmacctd. Is this a dynamically generated field? Also, the nf prefix is duplicated. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADLLnGSYWdEbHckJWhd79yvEP6DUAC7ks5sbboMgaJpZM4O9ibc> .
I also saw this. Can this still happen? I thought we are waiting until we get a template before we pass on the packet.
The field 153 is because our default field definition list is missing the type. It is "flow end milliseconds" of https://www.iana.org/assignments/ipfix/ipfix.xhtml
I would strongly advise against this, as IPFIX ("NetFlow version 10") has some incompatible fields with NetFlow version 9.
For example, id 1 is "octetDeltaCount" in IPFIX (8 bytes), while it's the number of incoming bytes in NetFlow 9 (4 bytes) (see http://netflow.caligare.com/netflow_v9.htm).
Aug 25, 2017
added a commit
this pull request
Aug 25, 2017
Hi everybody, i'm having troubles with an Invalid FlowVersion Exception (Invalid NetFlow version 0) when trying to log the flow of a netgear switch.
And this is the server.log result:
Thanks in advance !!!!