Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
NTA 4.0 Entity Model
A new entity model for querying flow related information was introduced with NetFlow Traffic Analyzer version 4.0. The new schema uses a set of entity types that each provide a different persective on your data and help you to build effective queries. The new schema is supported by SWISv3 only.
The basic entity type is
Orion.Netflow.Flows, and it exposes all flows as they are received by NTA. This entity type contains flows at the finest granularity available. You can use this entity type to query specific flows in a specific time period, or, more typically, you can query for a summary of the most used applications, protocols, etc. An example of such a query is as follows:
-- query top 10 protocols in the first hour of the year 2014 SELECT TOP 10 f.Protocol.Name, SUM(f.Bytes) as TotalBytes FROM Orion.Netflow.Flows f WHERE f.TimeStamp > '2014-01-01 00:00:00' AND f.TimeStamp <= '2014-01-01 01:00:00' GROUP BY f.Protocol.Name ORDER BY SUM(f.Bytes) DESC
You can also use the
Orion.Netflow.Flows entity type to generate a summary of the IP addresses to which your network traffic is targeted, as follows:
-- query top 10 target IPs in the first hour of the year 2014 SELECT TOP 10 f.DestinationIP, SUM(f.Bytes) as TotalBytes FROM Orion.Netflow.Flows f WHERE f.TimeStamp > '2014-01-01 00:00:00' AND f.TimeStamp <= '2014-01-01 01:00:00' GROUP BY f.DestinationIP ORDER BY SUM(f.Bytes) DESC
Orion.Netflow.Flows entity type is useful for any queries that inspect flow information using its raw properties. However, since flows contain pairs of source and destination IP addresses, domains, hostnames, countries, etc, other typical queries are used to query the volume of traffic generated by most active IP addresses or hostnames, regardless of whether they appear at a source or at a destination. For such queries, the model exposes additional, non-directional entity types that provide views on the raw flows, so that they contain an additional property for a respective directional pair and duplicate flows, as follows:
If a source and a destination property of a particular flow record return different values, the flow is duplicated, and the additional non-directional property returns the source value in the first record and the destination value in the second record.
Otherwise, if a source and a destination property return the same value, the record is not duplicated and the additional, non-directional property returns the shared value.
Flow duplication allows you to measure traffic volume of a flow for both its source and destination values when creating a summary by IP addresses, hostnames, etc.
The following table lists the additional entity types and semantics of the additional property:
|Entity type name||Additional property name|
The following query demonstrates the use of these entity types:
-- query top 10 IPs in the first hour of the year 2014 SELECT TOP 10 f.IP, SUM(f.Bytes) as TotalBytes FROM Orion.Netflow.FlowsByIP f WHERE f.TimeStamp > '2014-01-01 00:00:00' AND f.TimeStamp <= '2014-01-01 01:00:00' GROUP BY f.IP ORDER BY SUM(f.Bytes) DESC
There is one more view entity type designed for the conversation-based reports:
Orion.Netflow.FlowsByConversation. This entity type does not contain any additional properties, and it does not duplicate any flows. Instead, it focuses on a conversation point of view, where a conversation is defined as a pair of source and destination values, regardless of their order. This entity type ensures that any directional property value pairs (A, B) and (B, A) are returned, unified, as (A, B). This allows you both to group flows by directional property pairs and to calculate total traffic volume for conversations, as the example below indicates:
-- query top 10 IP conversations in the first hour of the year 2014 SELECT TOP 10 f.SourceIP, f.DestinationIP, SUM(f.Bytes) as TotalBytes FROM Orion.Netflow.FlowsByConversation f WHERE f.TimeStamp > '2014-01-01 00:00:00' AND f.TimeStamp <= '2014-01-01 01:00:00' GROUP BY f.SourceIP, f.DestinationIP ORDER BY SUM(f.Bytes) DESC
The examples above use literal strings for expressing dates and times. If you prefer to use relative dates in your query (like "now" and "seven days ago"), you can do that by starting with
GETUTCDATE() and adding or subtracting days. So to query over the last 7 days, do this:
SELECT ... FROM Orion.Netflow.Flows f WHERE f.TimeStamp > GETUTCDATE() - 7 AND f.TimeStamp < GETUTCDATE() GROUP BY ... ORDER BY ...
To add or subtract hours, use fractional days. So "one hour ago" would be expressed as
GETUTCDATE() - 1/24.
Important note: the SWQL Functions for doing date math (like
ADDHOUR, etc.) do work with flow data, but they aren't optimized. This means a several-orders-of-magnitude difference. Stick with + and - for doing date math when querying flow data.