Repository navigation
v0.22.0
·
25 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
Lapilli reads by default, and now changes only what a person approves: the
first principle in GOVERNANCE.md, rewritten with its design (#40). Changes
are off until turned on. With them: reports people schedule themselves, what
an alert says checked, and tasks kept in step with a tracker. The ledger
gains schemas 012 (reports) and 013 (proposals and approvals). Reviewed
before release as an attacker would and for correctness, twice, then
checked again once merged; what the reviews found is fixed in what is
described here.
- Changing rules, with a person's approval (docs/design/changes.md,
step 1). With--changes, Lapilli proposes changes to rules from what it
finds: a longerforfor a noisy rule, a threshold orforfor a near
miss, and retiring a spare rule or one that never fires (off first, then
deleted no sooner than four weeks later, approved again). People propose
too, and agents over MCP behind--mcp-propose. Each proposal comes with
its backtest over twelve weeks, on/proposalsand the rule's page, and
can be made into a task. A person approves it in the UI only, never
through the API, MCP or a token, and never their own (--approvers;
--approvals 2for two). An approved change is read and replayed again,
then opened as a pull request byrecipes/changes/(GitHub, GitLab,
Bitbucket, Gitea or Forgejo, Azure DevOps), which hold the repository's
credentials and take changes with--changes-token-file, or applied
through Grafana's provisioning API (--changes-grafana-token-file), with
the rule as it was kept for undo. Limits (--changes-limits, 5 deletions
a day and 20 changes an hour by default) and a stop (--changes-paused, a
file, or an admin's switch, looked at again just before each write) apply.
One proposal waits for a rule at a time, and a person, with their agents,
makes twenty an hour at most. A change sent to Grafana is kept as
applying, with the rule as it was and counted against the limits, until
it is read back. A Grafana rule is deleted only after its disable, and
only while it is still off. The change recipes take only their own pull
requests (same repository, intoBASE, by the token's own account; Gitea
tokens needread:user, andGITHUB_LOGINnames a GitHub App), and
write only files underRULES_PATHS, never one a CI system runs, so a
Lapilli someone took over can't push a workflow that would run with the
repository's secrets.
A pull request changed in review isamended(told aschange.amended)
rather than holding up its rule, and an admin can close one stuck opened,
merged or applying, with a reason. Raising--approvalsholds proposals
approved by fewer people.--changesneeds roles (--editorsor
--admins) and someone to approve (--approversor--admins). The
recipes take the pull request already open from their branch when the host
refuses another; a proposal's task shows its pull request but keeps its
own link. The chart adds
serve.changes; its pause is a ConfigMap namedFULLNAME-changes-paused,
andhelm installprints the command that makes it.
A fix to what a rule says is proposed the same way: one annotation
(summary,description,runbook_urlorrunbook) set to text a
person or an agent wrote, or taken out, offered on the rule's page beside
each content-check finding text can fix and carrying that finding as its
evidence; it is written as one quoted line, the rest of the file byte for
byte, and must parse as a template in the rule's language, call only that
language's own functions (for Prometheus, those every version since 2.45
has) and noquery, and can't grow without bound when the alert fires
(norange, no templates, no variables set; an escape or a replacement
only on a value as it is). Lapilli never writes the text itself. - Reports people schedule themselves (docs/design/reports.md). A Reports
page and/api/v1/reportstake reports on a preset (daily@09:00,
weekly mon@09:00,monthly 1@09:00orlast) or a cron expression,
hourly at most, in the ledger's zone or the team's, over the calendar span
before it or a span given (24h,7d). A report is for everyone or one
team and holds the sections picked: summary, noise, missed, woken,
notifications, replay, sources, tasks, content and, for a week or longer,
suggestions. It goes by name, never a URL, to--report-dest NAME=FILE
(the chart'sserve.reports.destinations),slack,team:NAME, or
webhookasreport.ready. It is sent once for each span through the
outbox, when the ledger has the span, or after it fell due by six hours or
half the time to the one before, whichever is less; after a restart, only
the last one due, if it isn't later than that time again. A time daylight
saving skips falls due when the clocks jump, and a repeated time the first
time. A report whose team is removed stays stopped. Admins write reports;
everyone sees them and a preview. The weekly digest is unchanged and shown
with them. - What an alert says, checked (docs/design/content.md). Lapilli now
reads what each rule says when it fires: summaries, descriptions, runbook
links, templates and severities, from every source's own fields. Azure
Monitor's alert rules and OCI's alarms are now read as rules too, listed
but not replayed. It checks them. Facts: no summary, no runbook, a
template that doesn't parse, no severity or one not in--severities, the
same condition as another rule. Measured: a template label no firing had,
a noisy rule with nofor. Guesses, kept apart, each with why: a label an
aggregation can't give, a rare severity, a range shorter than two scrapes,
absent()with nofor. Templates are parsed, never run; links are
checked, never visited. Each rule's page has "What it says", the rules
list marks problems of fact,/metricshas
lapilli_rule_content_problems{rule, check}, and MCP carries it all as
data; a report'scontentsection gives the problems now, by check. The
check takes time in proportion to an expression's length, is worked out
again only when the rules or firings change, and leaves out a rule it
can't read rather than failing.lapilli verdict --check-contentdoes the
same without a ledger.--check-content=falseturns it off. - Tasks kept in step with a tracker (docs/design/trackers.md).
--tasks-token-filelets a job, on the--incidents-listenport, read
tasks (with who set each one's status,status_byandstatus_at, and
updated_after=) and set their link and status, and nothing else. The
link must be https, at most 2,048 characters, and is set once. The token
has its own rate and turn and is refused if another token's file holds the
same value.
Its writes are recorded astasks-token, don't count as anyone's use, and
aren't sent to the notice webhook.recipes/trackershas scripts for
GitHub, GitLab, Jira and Linear. Each run makes an issue for every open
task without one and links it. A task follows its issue when the issue is
closed, and reopens if the job had closed it. WithCLOSE_ISSUES=true, an
issue is closed when a person closes its task. A recipe keeps in step only
the issues its own account made, labelledlapilliwhere labels are by
name: an issue filed ahead with a task's marker, or one linked by hand, is
neither followed nor closed. An issue is a task's only by the recipe's own
last marker line, which the task's text can't forge; Jira team labels are
lapilli-team-NAME. A GitHub App or Actions token names its account with
GITHUB_LOGIN. A task the tracker refuses is skipped and the run fails at
the end; a deleted issue is reported and its task left as it is. The
chart'sserve.trackersruns the script as a CronJob. Task links typed in
the UI can now be 2,048 characters long.
Security
go 1.26.11. Go'snet/httpcould crash an HTTP/1 server whose
handler reads a request's body while writing its headers, past a
MaxBytesReader(GO-2026-6859, CVE-2026-97054, published on the day
v0.21.0 was). Lapilli's handlers read a body before they answer, so no
way to reach it was found, but v0.21.0's archives and image were built
with Go 1.27.3 and carry the flaw. This release's are built with Go
1.27.4, andgo installnow needs 1.26.11 at least.
Fixed
lapilli serve --read-only=falseand--notify-slack=falsewere taken as
--read-onlyand--notify-slack: the value after=was ignored. They
now read it, as--check-content=falsedoes.