Cisco ASA Netflow Missing "switched" and other fields

[sempervictus]

Pulling from a 5525X on ASA 9.2 i'm seeing issues identical to those reported in #18. They seem to be caused by the ingest data missing fields referenced in the searches, the index pattern time calcs, all of it. Cisco docs indicate that while they export at v9, they use non-standard tuples (of course). Here's an event sample for which fields are present, hopefully there's a rational way to compose the missing fields or update the searches for this given its widespread use:

{
  "_index": "netflow-2017.11.28",
  "_type": "netflow",
  "_id": "AWABu2wgmBBMAB5s3hw5",
  "_version": 1,
  "_score": null,
  "_source": {
    "netflow": {
      "icmp_type": 0,
      "dst_locality": "public",
      "egress_acl_id": "00000000-00000000-00000000",
      "flowset_id": 260,
      "fw_event": 3,
      "src_port_name": "TCP/46060",
      "protocol": 6,
      "fw_ext_event": 1001,
      "dst_addr": "192.0.73.2",
      "dst_port_name": "TCP/80 (http)",
      "xlate_src_addr_ipv4": "REDACTED",
      "icmp_code": 0,
      "output_snmp": 2,
      "src_locality": "private",
      "xlate_src_port": 46060,
      "service_name": "TCP/80 (http)",
      "src_addr": "REDACTED",
      "xlate_dst_port": 80,
      "version": "Netflow v9",
      "server_addr": "192.0.73.2",
      "flow_seq_num": 2153,
      "src_port": 46060,
      "flow_locality": "public",
      "event_time_msec": 1511857545764,
      "input_snmp": 3,
      "ingress_acl_id": "f743bbbc-be670862-00000000",
      "dst_port": 80,
      "client_addr": "REDACTED",
      "xlate_dst_addr_ipv4": "192.0.73.2",
      "protocol_name": "TCP",
      "service_port": "80"
    },
    "@timestamp": "2017-11-28T08:25:48.000Z",
    "geoip": {
      "timezone": "America/Los_Angeles",
      "ip": "192.0.73.2",
      "latitude": 37.7484,
      "continent_code": "NA",
      "as_org": "Automattic, Inc",
      "city_name": "San Francisco",
      "country_name": "United States",
      "country_code2": "US",
      "dma_code": 807,
      "country_code3": "US",
      "region_name": "California",
      "location": {
        "lon": -122.4156,
        "lat": 37.7484
      },
      "autonomous_system": "Automattic, Inc (2635)",
      "postal_code": "94110",
      "asn": 2635,
      "region_code": "CA",
      "longitude": -122.4156
    },
    "@version": "1",
    "host": "REDACTED",
    "geoip_dst": {
      "timezone": "America/Los_Angeles",
      "ip": "192.0.73.2",
      "latitude": 37.7484,
      "continent_code": "NA",
      "as_org": "Automattic, Inc",
      "city_name": "San Francisco",
      "country_name": "United States",
      "country_code2": "US",
      "dma_code": 807,
      "country_code3": "US",
      "region_name": "California",
      "location": {
        "lon": -122.4156,
        "lat": 37.7484
      },
      "autonomous_system": "Automattic, Inc (2635)",
      "postal_code": "94110",
      "asn": 2635,
      "region_code": "CA",
      "longitude": -122.4156
    },
    "geoip_src": {
      "autonomous_system": "PRIVATE"
    },
    "type": "netflow",
    "tags": [
      "__netflow_direction_not_recognized"
    ]
  },
  "fields": {
    "@timestamp": [
      1511857548000
    ]
  },
  "sort": [
    1511857548000
  ]
}

This works on pf/opnsense very well, any advice on getting Cisco to play ball with it (or more likely the other way around) would be appreciated.

[Noebas]

ASA is using different fields in unix EPOCH time

date {
  match => ["[netflow][event_time_msec]", "UNIX_MS"]
  timezone => "Europe/Amsterdam"
  target => "[netflow][last_switched]"
      }
date {
  match => ["[netflow][flow_start_msec]", "UNIX_MS"]
  timezone => "Europe/Amsterdam"
  target => "[netflow][first_switched]"
      }

[Noebas]

Even better this works for the new ASA and fixes all errors

[robcowart]

Thanks for the sample event. That is helpful. Let me see what I can come up with.

[dhermans]

@robcowart am happy to test if you need a tester. we don't have an ASA but maybe related? I'm only seeing data in a few viz:
ASN related
Netflow: Services (bytes)
Netflow: Ingress (and Egress) Interfaces (bytes and packets)
Netflow: Servers (and Clients)

i have captures of my data if that helps

[robcowart]

Resolved by #29 and c6d01da