Skip to content

v0.22.0

Choose a tag to compare

@github-actions github-actions released this 11 Nov 07:54
· 25 commits to main since this release
Immutable release. Only release title and notes can be modified.
045e44d

Lapilli reads by default, and now changes only what a person approves: the
first principle in GOVERNANCE.md, rewritten with its design (#40). Changes
are off until turned on. With them: reports people schedule themselves, what
an alert says checked, and tasks kept in step with a tracker. The ledger
gains schemas 012 (reports) and 013 (proposals and approvals). Reviewed
before release as an attacker would and for correctness, twice, then
checked again once merged; what the reviews found is fixed in what is
described here.

  • Changing rules, with a person's approval (docs/design/changes.md,
    step 1). With --changes, Lapilli proposes changes to rules from what it
    finds: a longer for for a noisy rule, a threshold or for for a near
    miss, and retiring a spare rule or one that never fires (off first, then
    deleted no sooner than four weeks later, approved again). People propose
    too, and agents over MCP behind --mcp-propose. Each proposal comes with
    its backtest over twelve weeks, on /proposals and the rule's page, and
    can be made into a task. A person approves it in the UI only, never
    through the API, MCP or a token, and never their own (--approvers;
    --approvals 2 for two). An approved change is read and replayed again,
    then opened as a pull request by recipes/changes/ (GitHub, GitLab,
    Bitbucket, Gitea or Forgejo, Azure DevOps), which hold the repository's
    credentials and take changes with --changes-token-file, or applied
    through Grafana's provisioning API (--changes-grafana-token-file), with
    the rule as it was kept for undo. Limits (--changes-limits, 5 deletions
    a day and 20 changes an hour by default) and a stop (--changes-paused, a
    file, or an admin's switch, looked at again just before each write) apply.
    One proposal waits for a rule at a time, and a person, with their agents,
    makes twenty an hour at most. A change sent to Grafana is kept as
    applying, with the rule as it was and counted against the limits, until
    it is read back. A Grafana rule is deleted only after its disable, and
    only while it is still off. The change recipes take only their own pull
    requests (same repository, into BASE, by the token's own account; Gitea
    tokens need read:user, and GITHUB_LOGIN names a GitHub App), and
    write only files under RULES_PATHS, never one a CI system runs, so a
    Lapilli someone took over can't push a workflow that would run with the
    repository's secrets.
    A pull request changed in review is amended (told as change.amended)
    rather than holding up its rule, and an admin can close one stuck opened,
    merged or applying, with a reason. Raising --approvals holds proposals
    approved by fewer people. --changes needs roles (--editors or
    --admins) and someone to approve (--approvers or --admins). The
    recipes take the pull request already open from their branch when the host
    refuses another; a proposal's task shows its pull request but keeps its
    own link. The chart adds
    serve.changes; its pause is a ConfigMap named FULLNAME-changes-paused,
    and helm install prints the command that makes it.
    A fix to what a rule says is proposed the same way: one annotation
    (summary, description, runbook_url or runbook) set to text a
    person or an agent wrote, or taken out, offered on the rule's page beside
    each content-check finding text can fix and carrying that finding as its
    evidence; it is written as one quoted line, the rest of the file byte for
    byte, and must parse as a template in the rule's language, call only that
    language's own functions (for Prometheus, those every version since 2.45
    has) and no query, and can't grow without bound when the alert fires
    (no range, no templates, no variables set; an escape or a replacement
    only on a value as it is). Lapilli never writes the text itself.
  • Reports people schedule themselves (docs/design/reports.md). A Reports
    page and /api/v1/reports take reports on a preset (daily@09:00,
    weekly mon@09:00, monthly 1@09:00 or last) or a cron expression,
    hourly at most, in the ledger's zone or the team's, over the calendar span
    before it or a span given (24h, 7d). A report is for everyone or one
    team and holds the sections picked: summary, noise, missed, woken,
    notifications, replay, sources, tasks, content and, for a week or longer,
    suggestions. It goes by name, never a URL, to --report-dest NAME=FILE
    (the chart's serve.reports.destinations), slack, team:NAME, or
    webhook as report.ready. It is sent once for each span through the
    outbox, when the ledger has the span, or after it fell due by six hours or
    half the time to the one before, whichever is less; after a restart, only
    the last one due, if it isn't later than that time again. A time daylight
    saving skips falls due when the clocks jump, and a repeated time the first
    time. A report whose team is removed stays stopped. Admins write reports;
    everyone sees them and a preview. The weekly digest is unchanged and shown
    with them.
  • What an alert says, checked (docs/design/content.md). Lapilli now
    reads what each rule says when it fires: summaries, descriptions, runbook
    links, templates and severities, from every source's own fields. Azure
    Monitor's alert rules and OCI's alarms are now read as rules too, listed
    but not replayed. It checks them. Facts: no summary, no runbook, a
    template that doesn't parse, no severity or one not in --severities, the
    same condition as another rule. Measured: a template label no firing had,
    a noisy rule with no for. Guesses, kept apart, each with why: a label an
    aggregation can't give, a rare severity, a range shorter than two scrapes,
    absent() with no for. Templates are parsed, never run; links are
    checked, never visited. Each rule's page has "What it says", the rules
    list marks problems of fact, /metrics has
    lapilli_rule_content_problems{rule, check}, and MCP carries it all as
    data; a report's content section gives the problems now, by check. The
    check takes time in proportion to an expression's length, is worked out
    again only when the rules or firings change, and leaves out a rule it
    can't read rather than failing. lapilli verdict --check-content does the
    same without a ledger. --check-content=false turns it off.
  • Tasks kept in step with a tracker (docs/design/trackers.md).
    --tasks-token-file lets a job, on the --incidents-listen port, read
    tasks (with who set each one's status, status_by and status_at, and
    updated_after=) and set their link and status, and nothing else. The
    link must be https, at most 2,048 characters, and is set once. The token
    has its own rate and turn and is refused if another token's file holds the
    same value.
    Its writes are recorded as tasks-token, don't count as anyone's use, and
    aren't sent to the notice webhook. recipes/trackers has scripts for
    GitHub, GitLab, Jira and Linear. Each run makes an issue for every open
    task without one and links it. A task follows its issue when the issue is
    closed, and reopens if the job had closed it. With CLOSE_ISSUES=true, an
    issue is closed when a person closes its task. A recipe keeps in step only
    the issues its own account made, labelled lapilli where labels are by
    name: an issue filed ahead with a task's marker, or one linked by hand, is
    neither followed nor closed. An issue is a task's only by the recipe's own
    last marker line, which the task's text can't forge; Jira team labels are
    lapilli-team-NAME. A GitHub App or Actions token names its account with
    GITHUB_LOGIN. A task the tracker refuses is skipped and the run fails at
    the end; a deleted issue is reported and its task left as it is. The
    chart's serve.trackers runs the script as a CronJob. Task links typed in
    the UI can now be 2,048 characters long.

Security

  • go 1.26.11. Go's net/http could crash an HTTP/1 server whose
    handler reads a request's body while writing its headers, past a
    MaxBytesReader (GO-2026-6859, CVE-2026-97054, published on the day
    v0.21.0 was). Lapilli's handlers read a body before they answer, so no
    way to reach it was found, but v0.21.0's archives and image were built
    with Go 1.27.3 and carry the flaw. This release's are built with Go
    1.27.4, and go install now needs 1.26.11 at least.

Fixed

  • lapilli serve --read-only=false and --notify-slack=false were taken as
    --read-only and --notify-slack: the value after = was ignored. They
    now read it, as --check-content=false does.