Skip to content

Releases

v0.9.3

Choose a tag to compare

@github-actions github-actions released this 12 Oct 02:40

Trawl 0.9.3

Trawl collects and searches logs for homelabs and small installations. Run one
Linux server, send it events over HTTP or syslog, and search them from a browser,
the command line, or the terminal UI.

This release makes trawld's request limit a real limit, and it fixes service
names that the shipped Vector configurations sent and trawld refused.

What is included

  • [server] max_concurrent_requests counts the HTTP requests in progress
    across the whole listener. The count covers every route and method, and a
    request keeps it until the ingest or preview work that holds its body ends.
    When the count is full, trawld refuses the next request at once with a 503
    request_limit_reached and no Retry-After. The setting takes 1 to 65536,
    and the default is 32.
  • Two fixed allowances sit beside the count: 3 requests for health and
    metrics, and 4 for listing and cancelling queries. They do not borrow from
    the count or from each other, so a full server still answers its probes.
    trawl_http_requests_refused_total{allowance} counts the refusals.
  • fleet-auth runs at most four argon2 checks at once, which bounds the memory
    that token checks can use.
  • The Helm chart adds the TrawlIngestEventsRejected alert. It fires when
    trawld refuses events for a reason that a sender which reads only status
    codes, such as Vector, does not see.

Fixes

  • The shipped Vector configurations send only service names that trawld
    accepts. A journal entry takes its name from the systemd unit, cut at the
    template @, then from the syslog identifier, then from the command. For
    example, postgresql@17-main.service becomes postgresql. Before this
    fix, trawld refused templated unit names and stored none of those events.
    A name that no field can supply goes to a marked fallback, such as
    journal-unidentified, instead of unknown.
  • The base.toml catch-all no longer walks /var/log/trawl, which Vector
    cannot read, so Vector stops logging a glob error on every scan.
  • trawld keeps serving after an error from accepting a connection, such as
    running out of file descriptors.
  • trawld --doctor reads a request_limit_reached health answer as a
    capacity refusal. trawl-web passes a non-2xx answer to a stream request
    through unchanged.
  • The Helm chart prints its ingress TLS note only when an ingress is
    configured.
  • The Debian and Helm guides install fleet-admin before the schema step
    runs it, build the database Secrets from owner-only files, and say where
    the reverse proxy's certificate comes from.

Existing installations

0.9.3 keeps the 0.9.0 storage epoch and database baseline. The server applies
forward migrations at startup.

The default request limit drops from 256 to 32, and the setting now bounds the
whole server instead of each route. If you set max_concurrent_requests,
check the value against what the server should run at once. trawld refuses to
start on 0 or on a value above 65536. Run trawld --check-config first.

If you copied the Vector configurations from the Debian package, copy them
again from /usr/share/doc/trawl-server/examples/vector/. Service names for
templated units now lose their instance, and the unknown, varlog and
docker fallbacks become journal-unidentified, varlog-unidentified and
docker-unidentified. Update saved searches that match the old names.

Installation choices

Platform Available components
Linux amd64 and arm64 Server and CLI archives, Debian packages, and container images
macOS 15 or newer, Apple silicon and Intel CLI archives
Kubernetes Helm chart for the Linux server and browser

Linux packages target Debian 12. Archives include the DuckDB runtime; keep
their bin/ and lib/trawl/ directories together. macOS archives use ad-hoc
signatures and are not Apple Developer ID signed or notarized.

Start with the installation guide and
your first query.

Requirements and scope

A server needs persistent storage and PostgreSQL, with separate databases for
authentication and application state. Clients need an API key with the required
permissions and trust in the server's TLS certificate. The browser requires
its own public origin and persistent session key.

Trawl is designed for one node. It does not provide clustering, sharding, or
multi-tenancy. Storage and memory needs depend on ingest volume, retention,
and queries.

v0.9.2

Choose a tag to compare

@github-actions github-actions released this 08 Oct 05:17

Trawl 0.9.2

Trawl collects and searches logs for homelabs and small installations. Run one
Linux server, send it events over HTTP or syslog, and search them from a browser,
the command line, or the terminal UI.

This release fixes what a from-scratch install of 0.9.1 found when its
documentation was followed word for word on Debian 13 and on Kubernetes. It
also reads zone-less syslog timestamps in a configured zone.

What is included

  • The syslog listener reads an RFC 3164 timestamp that carries no offset in
    [syslog] default_timezone, or in UTC when that is unset. Set one zone for
    a peer with [syslog.sender_timezones].
  • The Debian package ships the Vector configurations under
    /usr/share/doc/trawl-server/examples/vector/. The sender guide pins the
    Vector version it is tested with and keeps the ingest key in its own
    environment file.
  • The installation guides work as written on Debian and Helm. They cover
    installing PostgreSQL, a reverse proxy for the browser, an operator role
    for ingest preview, curl configuration files that keep tokens off the
    command line, and a first query on your own server. Commands that install
    a chart or copy a file name this release.

Fixes

  • trawld keeps its stored data owner-only on every install channel. It sets
    its own file-creation mask, and at each start it closes its data and
    write-ahead-log directories to its own user. The Debian unit keeps
    /var/lib/trawl at 0750 and /var/log/trawl at 0700 across restarts.
  • A request body over the configured limit is refused with a 413 that names
    the setting. When the server closes the connection during an upload, the
    CLI says so, instead of reporting a generic network error. The configured
    body limits are now the real limits.
  • The Helm chart passes Pod Security restricted when crash dumps are off,
    including its test pod. fleet-admin works inside the trawld container as
    the access guide shows. httpRoute can route to the browser proxy.
  • trawld and trawl-web colour their log lines only on a terminal, and never
    when NO_COLOR is set.

Existing installations

0.9.2 keeps the 0.9.0 storage epoch and database baseline. The server applies
forward migrations at startup.

trawld refuses to start when it cannot close its data or write-ahead-log
directory to its own user, for example when another user owns it. Run
trawld --doctor before you upgrade. A directory it will tighten reports
will_tighten.

On a Debian host that is not on UTC, a sender that logs RFC 3164 local time
without an offset now lands in UTC. Set [syslog] default_timezone to the
senders' zone before you restart trawld. trawld now checks the syslog peer
settings at startup even when the listener is off: a source_service_map key
that is not an IP address, or a malformed allow_cidrs entry, refuses to
start. Run trawld --check-config first.

In the Helm chart, httpRoute.backend now defaults to web, the browser
proxy, like ingress.backend. A route that should reach the HTTPS API needs
httpRoute.backend: trawld. The pod sets seccompProfile: RuntimeDefault.
Crash-dump mode still adds SYS_PTRACE to trawld, so its namespace must
allow that capability.

Installation choices

Platform Available components
Linux amd64 and arm64 Server and CLI archives, Debian packages, and container images
macOS 15 or newer, Apple silicon and Intel CLI archives
Kubernetes Helm chart for the Linux server and browser

Linux packages target Debian 12. Archives include the DuckDB runtime; keep
their bin/ and lib/trawl/ directories together. macOS archives use ad-hoc
signatures and are not Apple Developer ID signed or notarized.

Start with the installation guide and
your first query.

Requirements and scope

A server needs persistent storage and PostgreSQL, with separate databases for
authentication and application state. Clients need an API key with the required
permissions and trust in the server's TLS certificate. The browser requires
its own public origin and persistent session key.

Trawl is designed for one node. It does not provide clustering, sharding, or
multi-tenancy. Storage and memory needs depend on ingest volume, retention,
and queries.

v0.9.1

Choose a tag to compare

@github-actions github-actions released this 05 Oct 01:35

Trawl 0.9.1

Trawl collects and searches logs for homelabs and small installations. Run one
Linux server, send it events over HTTP or syslog, and search them from a browser,
the command line, or the terminal UI.

This release prepares for 1.0. It adds tools for first-run and diagnosis,
fixes correctness and durability defects, and makes the browser proxy always
verify the server's certificate.

What is included

  • trawl trial starts a disposable local installation that the CLI owns and
    removes, for a first look before a real deployment.
  • Doctors check an installation from three places: trawl doctor checks one
    client connection, trawld --doctor checks the server host without changing
    it, and trawl-web --doctor checks the browser proxy and its upstream.
  • Ingest preview shows how the server canonicalizes a sample event. It stores
    nothing from the sample. The sender guides now walk through each source.
  • The Health page shows disk headroom and how many days of data each
    environment keeps.
  • Each query records its time in phases, in one query_timing event.
  • Query history search covers all retained history. After sign-in, the
    browser returns to the page you requested.
  • The filter rail is collapsed by default and opens for a countable page.
    Fields display according to what they hold.
  • A scheduled net can be run by hand, including a net with a schedule window.

Fixes

  • The server publishes each acknowledged event exactly once across crashes.
    At boot it hydrates the surviving write-ahead log, and it refuses corpus
    reads until coverage is proven.
  • The Debian package pins trawld's generated certificate for trawl-web, so a
    stock installation can sign in. trawl-web no longer restarts in a loop
    before trawld's first start.
  • Ingest admits events against hot-buffer capacity instead of evicting
    buffered events.
  • Run results that become unavailable are reported as unavailable, not as an
    empty result. timechart works on aggregated nets. Large integers keep
    their exact value.
  • Every server failure names its request, route and stage. A refused query
    shows the server's verdict on the text that was sent.
  • Several Search page corrections: the chart draws the whole result or
    nothing, a live stream carries no range, and the toolbar has one Save.

Existing installations

0.9.1 keeps the 0.9.0 storage epoch and database baseline. The server applies
forward migrations at startup.

trawl-web now always verifies trawld's certificate. The environment variable
that skipped verification for a loopback upstream has been removed. In the
Helm chart, when web.enabled and tls.mode is secret or
certManager, you must set tls.upstreamCa to secret, system, or a
mounted CA path. tls.mode=auto pins the generated certificate and needs no
change. See the deployment guide.

Installation choices

Platform Available components
Linux amd64 and arm64 Server and CLI archives, Debian packages, and container images
macOS 15 or newer, Apple silicon and Intel CLI archives
Kubernetes Helm chart for the Linux server and browser

Linux packages target Debian 12. Archives include the DuckDB runtime; keep
their bin/ and lib/trawl/ directories together. macOS archives use ad-hoc
signatures and are not Apple Developer ID signed or notarized.

Start with the installation guide and
your first query.

Requirements and scope

A server needs persistent storage and PostgreSQL, with separate databases for
authentication and application state. Clients need an API key with the required
permissions and trust in the server's TLS certificate. The browser requires
its own public origin and persistent session key.

Trawl is designed for one node. It does not provide clustering, sharding, or
multi-tenancy. Storage and memory needs depend on ingest volume, retention,
and queries.

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 18 Sep 02:54

Trawl 0.9.0

Trawl collects and searches logs for homelabs and small installations. Run one
Linux server, send it events over HTTP or syslog, and search them from a browser,
the command line, or the terminal UI. Events are stored in Parquet. The CLI can
also query local Parquet files without a server.

This release brings the current development baseline to packaged installations
ahead of 1.0. It includes substantial changes since 0.4.0 and requires fresh
state when replacing an installation that uses the older database or storage
formats.

What is included

  • A redesigned browser interface with a query console, field inspection,
    responsive tables, a command palette, and a Search quick start.
  • Smaller browser downloads and faster startup, plus a theme setting that can
    follow your system appearance.
  • Live search, saved queries, scheduled reports, and stored-run previews.
  • Health checks and Prometheus rules for operational failures.
  • Access control through Fleet API keys, roles, and browser sessions.
  • Linux and macOS distributions with a bundled DuckDB runtime, and updated
    installation and operation guides.

Existing installations

This release adopts the fresh-state baseline planned for Trawl 1.0. The older
database conversion chains have been removed. Changing only the image or binary
version is not a supported conversion of an older installation.

Keep the existing installation and its backups. Provision a fresh Trawl
application database, a fresh shared Fleet keystore, and a fresh owned data
directory for the new baseline. The Fleet keystore starts without keys, roles,
or grants. If Coastwatch shares that keystore, coordinate its cutover too.

Startup refuses incompatible state without converting or deleting it. Current
storage uses EPOCH = 3. An epoch marker alone does not establish database
compatibility. Normal forward migrations and crash recovery remain supported
for state created on the new baseline.

See the deployment guide and
backup and restore guide before
changing an existing installation.

Installation choices

Platform Available components
Linux amd64 and arm64 Server and CLI archives, Debian packages, and container images
macOS 15 or newer, Apple silicon and Intel CLI archives
Kubernetes Helm chart for the Linux server and browser

Linux packages target Debian 12. Archives include the DuckDB runtime; keep
their bin/ and lib/trawl/ directories together. macOS archives use ad-hoc
signatures and are not Apple Developer ID signed or notarized.

Start with the installation guide and
your first query.

Requirements and scope

A server needs persistent storage and PostgreSQL, with separate databases for
authentication and application state. Clients need an API key with the required
permissions and trust in the server's TLS certificate. The browser requires
its own public origin and persistent session key.

Trawl is designed for one node. It does not provide clustering, sharding, or
multi-tenancy. Storage and memory needs depend on ingest volume, retention,
and queries.

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 24 Jul 19:09
v0.4.0
56431da

What's Changed

  • fix(eval): close streaming-vs-DuckDB scalar parity gap (#22) by @jakub in #24
  • fix(eval): fill all partial strptime formats to match DuckDB (#25) by @jakub in #26
  • migrate trawl-web-ui onto fleet-ui chrome (ADR-0030 step 4, UI half) (#27) by @jakub in #29
  • fleet-ui structural primitives (Modal, Drawer, Tabs, icons) + trawl adoption sweep by @jakub in #30
  • fleet-ui small-widget extraction + unification (issue #31, ADR-0002 slice C) by @jakub in #32
  • fleet-ui content primitives + overlay focus management (issue #33, slice D) by @jakub in #34
  • fleet-admin parity: port revoke-grant and retype from trawl-admin (ADR-0004 slice 0) by @jakub in #37
  • trawld cutover to fleet-auth postgres keystore (ADR-0030 step 4 auth half, ADR-0004 slice 1) by @jakub in #38
  • trawl-web session swap to fleet-auth + fleet_session SSO (ADR-0004 slice 2) by @jakub in #40
  • app stores to postgres trawl database; trawl-auth crate deleted (ADR-0004 slice 3, closes the arc + #12) by @jakub in #42

Full Changelog: v0.3.3...v0.4.0

v0.3.3

Choose a tag to compare

@github-actions github-actions released this 23 Jun 05:57

Full Changelog: v0.3.2...v0.3.3

v0.3.2

Choose a tag to compare

@github-actions github-actions released this 22 Jun 03:32

What's Changed

  • fix(server): harden against DuckDB SIGSEGV — duckdb 1.5.4 + minidump crash capture by @jakub in #20

Full Changelog: v0.3.1...v0.3.2

v0.3.1

Choose a tag to compare

@github-actions github-actions released this 19 Jun 20:24

What's Changed

  • fix(server): stop corrupt WAL files from wedging compaction (quarantine + fsync) by @jakub in #18

Full Changelog: v0.3.0...v0.3.1

What's Changed

  • fix(server): stop corrupt WAL files from wedging compaction (quarantine + fsync) by @jakub in #18

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 19 Jun 05:32

What's Changed

  • feat(web): add coastwatch story viewer by @jakub in #8
  • feat(web-ui): add coastwatch derivation lineage UI by @jakub in #9
  • feat(fleet-auth): Postgres-backed keystore + shared auth types (closes jakub/coastwatch#33) by @jakub in #10
  • feat(fleet-auth): session AEAD + axum middleware + login/logout (coastwatch#34) by @jakub in #13
  • feat: fleet-admin CLI (migrate, generate-session-key, keys) (#36) by @jakub in #14
  • feat(fleet-ui): design tokens, theme system, base components (ADR-0030 step 2) by @jakub in #15
  • feat(fleet-ui): shell components (Shell, TopBar, Rail, Login, Icon) by @jakub in #16
  • fix: heal compaction/query schema drift + quarantine corrupt parquet by @jakub in #17

Full Changelog: v0.2.2...v0.3.0

v0.2.2

Choose a tag to compare

@github-actions github-actions released this 28 Apr 05:11

Full Changelog: v0.2.1...v0.2.2