Skip to content
BranchesTags

Repository files navigation

Netalizer

Netalizer is a packet analysis platform built around a Rust decoding engine and a React analysis UI.

At a high level it does four things:

  • ingests capture files such as PCAP, PCAPNG, MF4, and VSB
  • decodes frames into structured protocol messages using ANDF YAML definitions and CAN DBC databases
  • stores captures and decoded sectors for paged analysis
  • provides a browser UI for upload, filtering, signal inspection, and expression-driven analysis

The repository contains both the core parsing libraries and the product surfaces that use them.

Architecture

flowchart LR
  Upload[Upload UI] --> API[netalizer_server]
  API --> RawStore[(S3 / MinIO)]
  API --> Meta[(PostgreSQL)]
  API --> Parser[message_parser]
  Parser --> YAML[yml_parser]
  Parser --> DBC[dbc_parser]
  Parser --> VSDB[vsdb_parser]
  YAML --> Hooks[protocol_to_hook]
  DBC --> Hooks
  VSDB --> Hooks
  Hooks --> Engine[binary_parser]
  API --> Web[Analysis UI]
Loading

Repository Layout

.
├── apps/
│   └── web/
│       └── netalizer/          # React analysis application
├── crates/                     # Rust libraries
├── data/
│   └── andf-files/             # Built-in protocol definitions
├── infra/                      # Shared deployment and compose assets
├── services/
│   └── rust/
│       └── netalizer-server/   # Rust HTTP API
├── tools/
│   └── rust/                   # CLI tools and live capture apps
├── Cargo.toml                  # Rust workspace manifest
└── Makefile                    # Common build and local-dev commands

Product Surfaces

Web App

Path: apps/web/netalizer

The web app is the user-facing analysis client. It is built with React 19, TypeScript, Vite, Tailwind CSS v4, and a small internal component layer.

Primary routes:

  • /upload
    • upload capture files
    • manage protocol databases
  • /analysis
    • browse decoded messages
    • inspect raw payloads and signal trees
    • apply protocol and text filters
    • evaluate expressions and signals

Notable UI capabilities:

  • tabbed upload workflow with Capture and Databases sections
  • paged capture browsing for large decoded captures
  • visual expression builder extracted into a reusable component
  • signal charts and overview views
  • local mock-capture fallback for UI work when the backend is unavailable

Important frontend files:

  • apps/web/netalizer/src/App.tsx
  • apps/web/netalizer/src/pages/upload-page.tsx
  • apps/web/netalizer/src/pages/analysis-page.tsx
  • apps/web/netalizer/src/components/analysis/expression-dialog.tsx
  • apps/web/netalizer/src/components/analysis/expression-dialog/expression-builder-editor.tsx
  • apps/web/netalizer/src/lib/parser-client.ts

HTTP API

Path: services/rust/netalizer-server

netalizer_server is the main product backend. It exposes a REST API over Axum, stores capture metadata in PostgreSQL, stores raw objects and decoded sectors in S3-compatible storage, and uses the shared parser crates to decode uploaded files.

Core responsibilities:

  • authenticate users via bearer token or development bypass
  • accept persisted capture uploads
  • decode capture formats into structured messages
  • split decoded results into sectors for paging
  • evaluate filter and signal expressions
  • manage user-owned protocol databases
  • support group-based sharing of captures

Primary endpoints:

  • GET /health
  • GET /docs
  • GET /openapi.json
  • POST /captures
  • GET /captures
  • GET /captures/{capture_id}/parsed
  • GET /captures/{capture_id}/messages
  • POST /evaluate-expression
  • POST /evaluate-signal
  • GET /databases
  • POST /databases
  • GET /groups
  • POST /groups
  • POST /groups/{group_id}/members
  • POST /captures/{capture_id}/shares/groups

Important backend files:

  • services/rust/netalizer-server/src/main.rs
  • services/rust/netalizer-server/src/config.rs
  • services/rust/netalizer-server/src/auth.rs
  • services/rust/netalizer-server/src/repository.rs
  • services/rust/netalizer-server/src/storage.rs
  • services/rust/netalizer-server/src/source.rs
  • services/rust/netalizer-server/src/response.rs

CLI Tools And Live Capture Apps

Path: tools/rust

These are useful for lower-level development, offline parsing, and real-time capture scenarios.

  • file_reader
    • offline parser for .pcap, .pcapng, .mf4, and .vsb
    • can load additional .yml, .dbc, and .vsdb protocol files
  • raw_socket
    • live Ethernet capture and WebSocket streaming
  • can_socket
    • live SocketCAN to WebSocket bridge
  • lua_example
    • minimal Lua integration example

Parsing Stack

The Rust crates under crates/ form the decoding pipeline used by both the API service and the standalone tools.

Core crates

Crate Purpose
binary_parser Bit-level decoding engine with hook evaluation and Lua-backed expressions
protocol_to_hook Converts protocol descriptions into executable parser hooks
message_parser Unified entry point that loads protocol files and parses buffers
yml_parser Loads ANDF YAML protocol definitions
dbc_parser Loads CAN DBC files and converts them into protocol definitions
vsdb_parser Loads VSDB XML database files

Capture and format crates

Crate Purpose
pcap_parser PCAP and PCAPNG parsing
mf4_parser MF4 or MDF4 capture support
vsb_io Intrepid VSB I/O via C FFI

Parser flow

  1. A capture file is detected and read by a source adapter.
  2. Built-in ANDF files and any user-supplied protocol databases are loaded.
  3. message_parser routes file types to the correct parser crate.
  4. protocol_to_hook compiles protocol definitions into parser hooks.
  5. binary_parser decodes frames into a signal tree and flattened signal map.
  6. The API turns decoded messages into page-sized sectors for browsing.

Supported Inputs

Capture files

  • .pcap
  • .pcapng
  • .mf4
  • .vsb

The current upload UI also accepts .blf, but the implemented backend readers visible in the Rust sources are centered on PCAP, MF4, and VSB. If BLF support is expected in production, validate that independently against the backend implementation.

Protocol databases and definitions

  • .yml via ANDF definitions
  • .dbc for CAN message databases
  • .vsdb via the offline file reader and parser libraries

The product UI currently exposes upload support for .dbc databases. Built-in ANDF files are loaded from the repository protocol root.

Built-In Protocol Definitions

Path: data/andf-files

The repository ships a set of YAML protocol definitions used as built-in decoders. These include:

  • frame-level dispatch in frame.yml
  • CAN definitions in can/
  • Ethernet and upper-layer protocols in ethernet/

Examples of shipped protocols:

  • Ethernet
  • VLAN and double VLAN
  • ARP
  • IPv4 and IPv6
  • ICMP and IGMP
  • TCP and UDP
  • SOME/IP and SOME/IP-SD
  • DoIP
  • RTP and RTCP
  • PTP
  • EtherCAT
  • MRP, MSRP, MVRP

These definitions are resolved through the hook system. A hook binds a boolean expression and a location in the decoded state tree to a decoding schema that should be applied when the expression matches.

Capture Lifecycle

The persisted capture flow is the main product workflow.

sequenceDiagram
  participant User
  participant Web
  participant API
  participant DB as PostgreSQL
  participant S3 as S3 / MinIO
  participant Parser

  User->>Web: Upload capture
  Web->>API: POST /captures
  API->>S3: Store raw upload
  API->>DB: Insert capture metadata
  API->>Parser: Decode frames
  Parser-->>API: Parsed messages
  API->>S3: Store decoded sectors
  API->>DB: Insert sector metadata
  API-->>Web: Capture record + first page
  Web->>API: GET /captures/{id}/messages
  API-->>Web: Additional pages on demand
Loading

Each decoded message includes:

  • message id
  • message index
  • timestamp
  • protocol name
  • raw hex payload
  • flattened signal values for search and expressions
  • hierarchical signal tree for detail views

Database And Storage Model

Netalizer intentionally splits metadata and large binary content.

PostgreSQL

PostgreSQL stores metadata and access-control state.

Current migrations live in:

  • services/rust/netalizer-server/database/migrations/000001_access_control_storage.up.sql
  • services/rust/netalizer-server/database/migrations/000002_protocol_databases.up.sql

Important tables:

  • app_user
  • capture_group
  • capture_group_member
  • capture_file
  • capture_decode_sector
  • capture_group_grant
  • protocol_database

What is stored there:

  • capture ownership and metadata
  • decoded sector indexes and search text
  • user groups and membership roles
  • capture sharing permissions
  • uploaded protocol database metadata

S3-Compatible Object Storage

Object storage is used for large payloads.

Key families used by the service:

  • raw captures: captures/{owner_subject}/{capture_id}/{file_name}
  • decoded sectors: decoded/{capture_id}/sectors/{sector_index}.json
  • uploaded protocol databases: protocol-databases/{owner_subject}/{database_id}/{file_name}

The default local object store is MinIO.

Authentication And Sharing

Authentication is handled by the backend service.

Modes:

  • standard bearer-token validation against Keycloak issuer and JWKS config
  • local development bypass via NETALIZER_DEV_AUTH_SUBJECT

Sharing model:

  • captures are owned by users
  • groups can be created and managed by users
  • group membership roles are owner, admin, and member
  • capture shares can grant read, write, or admin

This lets a capture be uploaded once and shared across a team without duplicating raw data.

Expression And Signal Evaluation

The analysis UI supports two related workflows:

  • message filtering using POST /evaluate-expression
  • signal extraction using POST /evaluate-signal

Recent UI work extracted the visual expression builder into reusable pieces so the expression-building experience is no longer trapped inside a single dialog implementation.

Relevant files:

  • apps/web/netalizer/src/components/analysis/expression-dialog.tsx
  • apps/web/netalizer/src/components/analysis/expression-dialog/expression-builder-editor.tsx
  • apps/web/netalizer/src/components/analysis/expression-dialog/signal-catalog.ts

Protocol Database Management

Netalizer now supports user-managed protocol databases for decode-time enrichment.

What exists today:

  • backend persistence for uploaded protocol databases
  • GET /databases and POST /databases
  • object-store persistence for database files
  • database metadata in PostgreSQL
  • upload and listing UI in the Databases tab
  • automatic inclusion of uploaded DBC files during future capture decode for the authenticated user

Current scope:

  • the product UI is focused on .dbc upload
  • built-in ANDF files remain part of the baseline decode set
  • delete, rename, and capture-pinned database selection are not implemented yet

Recent Capabilities Visible In Code

The current repository includes several feature slices that are worth calling out because they change how the product is used.

MF4 support

  • dedicated mf4_parser crate
  • integrated into server-side source parsing
  • available from the file_reader CLI
  • intended to support both Ethernet and CAN extraction paths

Reusable expression builder

  • visual builder extracted from the analysis dialog into reusable editor components
  • shared signal catalog generation for builder inputs
  • still supports manual expression editing alongside the visual builder

DBC database tab

  • upload page now contains a Databases tab
  • uploaded DBCs are shown per user
  • uploaded DBCs are applied to future decode requests automatically

Local Development

Prerequisites

  • Rust toolchain
  • Node.js and npm
  • Go toolchain for the service manager under tools/go
  • Docker for local service dependencies and migration tooling

Common commands

From the repository root:

make check
make build
make test
make fmt
make clippy

Web app commands

make web-dev
make web-build

The web app itself lives at apps/web/netalizer and exposes these scripts:

npm run dev
npm run build
npm run lint
npm run preview

Service manager commands

make service-list
make service-up s=netalizer-server
make service-down s=netalizer-server
make service-erase s=netalizer-server
make service-logs s=netalizer-server
make service-config s=netalizer-server

The service metadata for the backend is defined in services/rust/netalizer-server/service.yml.

Database migrations

make migration-add s=netalizer-server a=<migration_name>
make migration-up s=netalizer-server
make migration-down s=netalizer-server

If you pull current code into an existing environment, make sure migration 000002_protocol_databases is applied before using the database-management endpoints.

Offline parser example

cargo run -p file_reader -- \
  --input ./sample.pcapng \
  --protocol ./data/andf-files \
  --protocol ./custom.dbc

Configuration

Backend environment variables

services/rust/netalizer-server/src/config.rs is the source of truth for runtime configuration.

Important variables:

Variable Default Purpose
NETALIZER_SERVER_BIND 0.0.0.0:9002 HTTP bind address
DATABASE_URL none PostgreSQL connection string
NETALIZER_PROTOCOL_ROOT repo data/andf-files built-in protocol definition root
NETALIZER_DECODE_SECTOR_SIZE 250 number of messages per decoded sector
NETALIZER_DEV_AUTH_SUBJECT unset bypass auth in local development
NETALIZER_S3_ENDPOINT http://minio:9000 object-store endpoint
NETALIZER_S3_REGION us-east-1 object-store region
NETALIZER_S3_BUCKET netalizer-captures object-store bucket
NETALIZER_S3_ACCESS_KEY minioadmin object-store access key
NETALIZER_S3_SECRET_KEY minioadmin object-store secret
NETALIZER_S3_ALLOW_HTTP true allow plain HTTP for local MinIO
KEYCLOAK_ISSUER_URL http://keycloak:8080/realms/netalizer token issuer
KEYCLOAK_JWKS_URL derived from issuer JWKS endpoint
KEYCLOAK_AUDIENCE unset expected JWT audience

Frontend environment variables

Important frontend env usage is in apps/web/netalizer/src/lib/parser-client.ts.

Variable Default Purpose
VITE_PARSER_API_URL http://127.0.0.1:9002 backend API base URL
VITE_ENABLE_MOCK_CAPTURE false allow mock capture fallback in UI
VITE_PARSER_BEARER_TOKEN unset optional bearer token for local API calls

Build System Notes

Rust workspace

The root Cargo.toml defines a workspace over:

  • crates/*
  • services/rust/*
  • tools/rust/*

It currently excludes crates/md4_support.

Frontend build

The frontend uses Vite with:

  • React plugin
  • React compiler preset through Babel
  • Tailwind CSS Vite integration
  • path alias @ mapped to apps/web/netalizer/src

Where To Start

Depending on the task, these are the best entry points.

Product behavior

  • web routes: apps/web/netalizer/src/App.tsx
  • upload workflow: apps/web/netalizer/src/pages/upload-page.tsx
  • analysis workflow: apps/web/netalizer/src/pages/analysis-page.tsx

Backend API behavior

  • routes and handlers: services/rust/netalizer-server/src/main.rs
  • auth: services/rust/netalizer-server/src/auth.rs
  • persistence: services/rust/netalizer-server/src/repository.rs
  • object storage: services/rust/netalizer-server/src/storage.rs

Protocol and decode behavior

  • source readers: services/rust/netalizer-server/src/source.rs
  • parser entry point: crates/message_parser/src/lib.rs
  • hook generation: crates/protocol_to_hook/src/lib.rs
  • binary decode engine: crates/binary_parser/src/lib.rs
  • protocol definitions: data/andf-files

Current Gaps And Follow-Up Work

The repository already contains working support for persisted captures, MF4 ingestion, expression-based analysis, and DBC uploads. A few areas are visibly still in progress or intentionally limited.

  • protocol database management currently supports upload and list, but not delete or rename
  • capture decodes use the caller's currently uploaded DBC set rather than a capture-pinned database snapshot
  • the UI advertises .blf in the upload control, but backend support should be verified before treating it as complete
  • the legacy root README previously described only the Rust engine; this document replaces that narrower view

Summary

Netalizer is not just a parser library. It is a full capture-analysis stack built from:

  • a Rust decoding engine
  • a Rust HTTP API with persistence and sharing
  • a React analysis interface
  • built-in YAML protocol definitions
  • user-managed DBC protocol databases
  • tooling for local parsing and live capture work

If you are onboarding to the codebase, start with the upload and analysis flows in the web app, then trace into netalizer_server, and finally drill into the parser crates when you need to change decode behavior.

About

No description, website, or topics provided.

Resources

Readme
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages