Skip to content

v1.4.0

Latest

Choose a tag to compare

@github-actions github-actions released this 21 Jul 18:25
3f93bc2

New Features

  • Added HTTP/SSE server replacing the previous WebSocket service. The server
    subcommand now starts an HTTP server with REST endpoints and SSE search
    streaming. Key additions:
    • Three MVP endpoints: GET /health, GET /api/v1/system/info,
      POST /api/v1/search/stream (SSE streaming search with progress, element
      batches, cancellation on disconnect, and max-results limit)
    • Parallelized SSE search with bounded mpsc channel (capacity 32)
      backpressure; element batches are never dropped, progress events may be
      coalesced
    • Terminal event invariant: exactly one of completed, cancelled, or
      error. All three carry a SearchStreamResult payload with matched
      elements, source-file counts, compressed-byte metadata, rate, and matching
      collectors/files
    • Server admission control (server_max_concurrent_searches, default 3);
      excess requests return HTTP 429. SSE search returns 400 on invalid
      peer_ip values
    • Phase 2 REST API: stateless endpoints (/time/parse, /country/lookup,
      /ip/lookup, GET /ip/public), DB read-only endpoints
      (/database/status, /rpki/roa/lookup, /rpki/aspa/lookup,
      /pfx2as/lookup, /as2rel/relationship, POST /as2rel/search),
      DB refresh endpoints (/database/refresh, /inspect/refresh,
      /as2rel/refresh), and composite endpoints (/rpki/roa/validate,
      /inspect/query). ASPA validation is deferred. DB-backed endpoints
      return NOT_INITIALIZED (HTTP 503) if required data is missing
    • Token-based auth middleware (server_auth_enabled / server_auth_token);
      when enabled, /api/v1/* requires Authorization: Bearer <token>
      (case-insensitive per RFC 7235); /health stays open. Server refuses to
      start if auth is enabled but token is empty
    • CLI remote search mode: --remote-url flag sends queries to a remote
      Monocle HTTP service; --remote-token provides Bearer auth. Consumes SSE
      stream and formats results with existing CLI formatters. Remote search
      exits with non-zero status on error/cancelled SSE events or connection
      drop without completed. Applies --filter-file/--prefix-file merging
      and filter validation before dispatch
    • Docker Compose deployment with health check, persistent volumes, and
      env-based configuration. Docker runtime runs as non-root monocle user
      with pre-created config directory and HOME set
    • Server configuration via monocle.toml and MONOCLE_* environment
      variables: server_address, server_port, server_max_search_batch_size,
      server_max_search_results, server_search_timeout_secs,
      server_max_concurrent_searches, search_concurrency
  • Added Cisco sh ip bgp text dump parsing to the parse command. The parser
    auto-detects the fixed-width plaintext format from the preamble, extracts
    prefix/AS-path/next-hop/metric/local-pref/origin, and supports all standard
    parse filters, output formats, and MRT export (--mrt-type rib|updates).
    Multi-path continuation lines and wrapped-prefix layouts are handled.
    Header column detection uses whitespace split rather than fixed positions.
  • Added --filter-file (JSON) and --prefix-file (newline text) flags to
    monocle parse and monocle search for loading large filter sets from
    files. File filters merge with CLI flags — union within each dimension (OR),
    AND across dimensions. Supports the RIB-extract → filter-updates workflow at
    scale (#117).
  • Added RPKISPOOL as the default historical RPKI source with Sobornost as the
    default mirror. The rpki roas and rpki aspas commands retain ripe and
    rpkiviews; invalid source/collector combinations now return errors (#134).
  • Added --use-cache / --cache-dir MRT-file caching and --fields output
    selection to the rib command (#137).
  • Added --concurrency to local search and server commands. Explicit values
    use local rayon thread pools; unset/0 keeps rayon defaults including
    RAYON_NUM_THREADS.

Breaking Changes

  • Replaced WebSocket server with HTTP/SSE service. All WebSocket modules
    (protocol, handler, sink, op_sink, router, operations,
    handlers/*) have been removed.
  • Updated axum to 0.8 and tower-http to 0.6.
  • Removed uuid and async-trait dependencies (no longer needed without
    WebSocket operation tracking).
  • Updated ServerArgs CLI flags: removed --data-dir, --max-concurrent-ops,
    --max-message-size, --connection-timeout-secs, --ping-interval-secs;
    added --max-search-batch-size, --max-search-results,
    --search-timeout-secs. --address and --port are now optional (default
    to config values).

Performance Improvements

  • Optimized database refresh (bulk insert) performance by 21–49% across all
    repositories. Indexes are now dropped before bulk insert and rebuilt in one
    pass afterward, instead of being updated per-row.
  • Removed redundant idx_pfx2as_prefix_str and low-value
    idx_pfx2as_validation indexes. lookup_exact rewritten to use the
    existing BLOB range index (prefix_start/prefix_end/prefix_length).
  • Fixed PRAGMA restore bug: store() methods no longer toggle
    synchronous/journal_mode PRAGMAs, preserving the connection's
    WAL/NORMAL defaults. Previously, every refresh left the connection
    in DELETE/FULL mode, degrading query performance until restart.
  • Switched INSERT OR REPLACE to plain INSERT in all refresh paths
    (tables are cleared before insert, so no conflicts are possible).
  • Added db_refresh_bench example for measuring refresh performance
    with real or synthetic data.
  • Added tracing::info! progress logging to the rib command (visible with
    --debug): RIB download/parse reports message counts, and update replay
    shows per-file progress.

Bug Fixes

  • Fixed search --dump-type rib applying the requested snapshot time window to
    per-route timestamps inside matching RIB dumps. Stable routes learned before
    the dump timestamp are now retained, restoring AS-path search results (#136,
    #138).
  • Fixed rib command dropping RIB entries whose per-route timestamp predates
    the RIB dump time. The base RIB parser was incorrectly applying a start_ts
    filter at the dump timestamp, silently excluding stable routes learned days
    or weeks earlier (#124).
  • Fixed rib command using an earlier RIB than necessary when the target
    timestamp coincides with a RIB dump time. The broker query's ts_end filter
    is exclusive, so a RIB starting exactly at the target was excluded, forcing
    the code to select the previous RIB and replay unnecessary update files.
    This caused ~3× slowdown for common midnight RIB queries.
  • Fixed --time-format rfc3339 being ignored for json, json-line, and
    json-pretty output formats. JSON output now honors --time-format: unix
    (default) emits a numeric timestamp field (backward compatible); rfc3339
    emits an RFC 3339 string (#123).
  • Validated prefix input in pfx2as_lookup and roa_lookup REST endpoints
    before spawning blocking tasks, so invalid prefixes return 400 instead of
    500.
  • Auth middleware now accepts case-insensitive Bearer scheme tokens per
    RFC 7235.
  • POST /api/v1/database/refresh with source=pfx2as now returns HTTP 501
    instead of 200, so automation can detect the operation is not implemented.
  • GET /api/v1/rpki/roa/lookup now applies AND semantics when both prefix
    and asn are provided (filters covering ROAs by origin ASN).

Build & Infrastructure

  • Replaced QEMU-emulated multi-platform Docker build with native per-platform
    builds: linux/amd64 on ubuntu-latest, linux/arm64 on ubuntu-24.04-arm
    native runners. Each build pushes by digest; a final manifest-merge job
    assembles the multi-architecture OCI image index. Expected to reduce CI build
    time from ~57 minutes to roughly the slower platform's native compile time.
  • Added cargo-chef to the Dockerfile for dependency-layer caching. Source-only
    changes now reuse the pre-compiled dependency layer instead of recompiling all
    dependencies from scratch.
  • Switched Docker BuildKit cache from GitHub Actions cache to registry-based
    cache (bgpkit/monocle:buildcache-{arch}), removing the 10 GB GHA cache
    budget constraint.

Code Improvements

  • Added HistoricalRpkiCollectorOption as the preferred general-purpose alias
    while retaining RpkiViewsCollectorOption compatibility, and clarified
    historical source/collector CLI errors.
  • Removed redundant formatting references in parse command output to satisfy
    current Clippy checks.
  • Made refresh index rebuilds atomic by wrapping index drops, table clears,
    inserts, and index recreation in one transaction; indexes are dropped before
    clearing tables to avoid unnecessary per-row index maintenance during
    refresh.
  • Ensured upgraded pfx2as databases drop removed legacy indexes during
    refresh and kept the benchmark example buildable with
    --no-default-features --features lib.
  • Preserved immediate transaction semantics for pfx2as refreshes and reused
    shared AS2Rel schema constants when rebuilding indexes.

Dependencies

  • Updated bgpkit-parser to v0.18.0.
    • Default parse JSON output and SSE search element batches now include
      peer_bgp_id for TableDumpV2 RIB entries when the peer's BGP Identifier is
      available.
    • Known-but-unsupported BGP attributes are retained in the existing
      unknown/deprecated fields rather than silently dropped, and malformed
      BGP input is validated more strictly.