v1.0.1

There have been a lot of minor changes since v1.0.0, so it felt like a good time to checkpoint.

This tag captures:

• Mbuf reference counting to allow mbufs to be safely shared between data structures
• Some TCP enhancements: discarding OOO buffers when no longer needed, respecting FIN/ACK semantics when determining termination, skipping reassembly in more cases
• Extensions to the filter language (!contains, !equals, raw byte matching)
• Optional prometheus DB integration for statistic exports
• Some additional runtime monitoring stats
• Additional DPDK versions
• SSH parser

Retina v1.0.0

Release Notes (v1.0.0, October 2024)

Retina Version 1.0.0 introduces major changes that allow users to specify multiple subscriptions and for each subscription to accept one or more datatypes. In other words, Retina v1 rearchitects Retina to support multiple concurrent analysis tasks, correlations across connections, and smoother customization in the types of data to collect.

This version is not backward-compatible with v0.1.0.

The notes below focus on the major API and developer changes introduced. More detail on using Retina v1.0.0 can be found on GitHub and in our documentation.

API Changes

Here is an example of a Retina v0 application:

use retina_core::config::default_config;
use retina_core::Runtime;
use retina_core::subscription::TlsHandshake;
use retina_filtergen::filter;

#[filter("tls.sni ~ '^.*\\.com$'")]
fn main() {
    let cfg = default_config();
    let callback = |tls: TlsHandshake| {
        println!("{:?}", tls);
    };
    let mut runtime = Runtime::new(cfg, filter, callback).unwrap();
    runtime.run();
}

And the same application in v1.0.0:

use retina_core::config::default_config;
use retina_core::Runtime;
// Include both `filter` and `retina_main`
use retina_filtergen::{filter, retina_main};
// Include subscribable datatypes from the `retina_datatypes` crate
use retina_datatypes::TlsHandshake;

// The macro `filter` is applied to a single callback
// `callback` will be invoked inline within a generated filter
// `callback` can no longer be a closure
// Parameters must be by reference
#[filter("tls.sni ~ '^.*\\.com$'")]
fn callback(tls: &TlsHandshake) {
    println!("{:?}", tls);
}

// The developer must specify how many subscriptions to expect
#[retina_main(1)]
fn main() {
    let cfg = default_config();
    // SubscribedWrapper must be specified as the generic argument to Runtime
    // (This is generated at compile-time and handles all data tracking)
    let mut runtime: Runtime<SubscribedWrapper> = Runtime::new(config, filter).unwrap();
    runtime.run();
}

In v1.0.0, this application could be extended to take in additional datatypes and/or have additional callbacks:

use retina_core::config::default_config;
use retina_core::Runtime;
use retina_filtergen::{filter, retina_main};
use retina_datatypes::*;

// Request connection-level metadata in addition to the TLS handshake
#[filter("tls.sni ~ '^.*\\.com$'")]
fn callback(tls: &TlsHandshake, conn_record: &ConnRecord) {
    println!("{:?}, {:?}", tls, conn_record);
}

// Specify additional subscription(s)
#[filter("dns")]
fn callback(dns: &DnsTransaction) {
    println!("{:?}", dns);
}

// Update `retina_main` with the new number of subscriptions
#[retina_main(2)]
fn main() {
    let cfg = default_config();
    let mut runtime: Runtime<SubscribedWrapper> = Runtime::new(config, filter).unwrap();
    runtime.run();
}

In Retina v1.0.0, subscriptions can also be specified through a TOML file; see the filtergen crate.

Developer Changes

Subscribable Types

In Retina v0, subscribable types were defined in the core library. In Retina v1, datatypes are in the retina_datatypes crate.

Retina v0 distinguished between subscribable and tracked datatypes. Before delivery, each subscribable type is built from its associated tracked type. In Retina v1, there is no such distinction, and there is no option to "build" a new type before delivery.

Retina v0 defined the Retina Runtime with the subscribable type as a generic parameter. Retina v1 combines requested subscribable types into one "wrapper" struct at compile-time. Thus, subscribable datatypes in Retina v1 must implement traits and associated data that facilitate compile-time analysis.

See retina_datatypes for more.

Filter PTree Display

When compiling a Retina v0 application, the framework would print out the generated filters as trees. These trees were relatively simple:

• There were three filter stages: Packet, Connection, and Session
• There was only one outcome at each stage; traffic was either dropped or kept.

The filter trees displayed by Retina v1 are slightly more complex.

• Filter tree nodes can now contain (1) direct delivery to a callback and/or (2) filter "actions", which tell the framework what to do with the connection next. For more on actions, see core/filter/actions.rs.
• Filters are optimized to remove redundant conditions. See core/filter/ptree::PTree::collapse for more.
• There are more potential filter stages, not all of which will be invoked in every application. See core/subscription/mod.rs for more.

Retina v0.1.0

This is the first released version of Retina released in 2022. It provides the core framework (connection tracking, filter generation, etc.) and the ability to specify a single subscription with pre-defined datatypes.

A detailed description of this version of Retina's architecture and its performance can be found in our SIGCOMM'22 paper: Retina: Analyzing 100 GbE Traffic on Commodity Hardware.

Documentation for v0.1.0 can be found at https://stanford-esrg.github.io/retina/v0.1.0/retina_core/