IPFIX TIMESTAMP_MIN and TIMESTAMP_MAX are incorrect

[farshadkh]

Hi,

Here's a sample of print plugin output. As you can see, TIMESTAMP_MIN is larger than TIMESTAMP_MAX. This is happening for all records. Tried playing for pro-rating and different aggregations with same results.

sample_ipfix.txt
ipfix_sample.gz.pcap.zip

config:

aggregate[test]: src_host, dst_host, src_port, dst_port, proto, timestamp_start, timestamp_end
print_history[test]: 5m
print_history_roundoff[test]: m
print_output_file_append[test]: true
print_output_file[test]: /tmp/rebin/%Y%m%d-%H%M.txt
print_cache_entries[test]: 100000
print_refresh_time[test]: 15
nfacctd_port: 2055
nfacctd_stitching: true

[paololucente]

Hi @farshadkh, can you say which code you are running, is it latest from master? I ask because we had a bug solved around flow stitching in issue #136 (June 2017). Also the pcap trace you attached, which would be useful to get more insight and reproduce in lab, is unreadable since it is missing a packet containing the IPFIX template (needed to read the IPFIX data packet) - can you please fix and re-send? Thanks, Paolo

[farshadkh]

Sorry about pcap file. A larger file including template is here.
ipfix_large_dump.pcap.zip

The code is latest master on commit febe3ac. Also I think the problem is more serious than just stitching, historical accounting no longer works due to TIMESTAMP_END = 1970-01-01 03:30:00.0. Setting nfacctd_time_new will cause all timestamp_start and ends to be equal.

[farshadkh]

Same result with v1.6.2. The time stamps are not correct.

[paololucente]

Hi @farshadkh ,

I've inspected the latest IPFIX trace you sent and i reckon what is going on. What device is exporting such data? Do you have any control in the content (field types) of the IPFIX messages emitted by the device?

There appears to be a bug in how the timestamps are crafted in those IPFIX packets: field types 21 (flowEndSysUpTime) and 22 (flowStartSysUpTime), that is start and end times relative to the SysUpTime of the device, except IPFIX - that is a difference with NetFlow v9 - has no provision to send SysUpTime as part of its header. So Absolute times should be used instead in IPFIX. The value you see pmacct setting as timestamp_start is the timestamp of the whole IPFIX packet (not the one of the specific flow).

If you get questions on this from who is coding the IPFIX emitter, feel free to do a round of introductions or point them to me via unicast email.

Paolo

[farshadkh]

Hi,
it's a mikrotik device. I'll open a bug report for them. Thanks.

[paololucente]

Hi @farshadkh , thanks for confirming. I will close this issue then. Paolo