Repository navigation
v0.23.0
·
5 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
Changing rules through a managed Prometheus ruler, step 2 of changing rules
with a person's approval (docs/design/rulers.md): Mimir and Cortex, Amazon
Managed Service for Prometheus and Azure's Prometheus rule groups, read
with --ruler and changed through their own APIs, once a person approves,
only in the namespaces --changes-in names. Mimir's path runs against a
real Mimir 3.2.4 in CI; Cortex's, Amazon's and Azure's against stand-ins
answering as their documents show, not an account. No ledger schema
change: a rule read otherwise keeps its key, so no ledger sees a new
version of its rules on upgrade.
--ruler URL, forlapilli verdictandlapilli serve: the alert
rules a ruler holds, as written, from the API that writes them. Mimir's
and Cortex's configuration API (--ruler-token-file, and
--ruler-tenantfor one with tenants, sent asX-Scope-OrgID), Amazon's
control plane (https://aps.REGION.amazonaws.com/workspaces/ID, with
--auth aws), or Azure Resource Manager's rule groups in a subscription
or a resource group (with--auth azure). A rule knows its namespace;
rules of one name in two namespaces are told apart by both
(ScrapeSlow [payments/shop]).- The ruler path. A proposal for a rule read with
--ruleris applied
through the ruler's API when--changes-innames its namespace (Azure:
RG/NAME; a prefix ends in*) and no tag says another tool keeps it
(CloudFormation's own, or one--changes-managed KEY=VALUEnames);
otherwise it is for a person, and its page says why. What one write
replaces (a group, an Amazon namespace, an Azure rule group) is read
again just before: the rule itself changed since is refused as stale;
other rules in it changed, the change is made afresh. It is kept for
undo, written with its kind's write credential
(--changes-ruler-token-file,--changes-aws-role,
--changes-azure-client-id), and read back as rules: an Amazon namespace
stays applying while it isUPDATING, and a unit that reads as neither
it was nor as proposed fails. A group goes with its last rule; an Amazon
namespace's or an Azure rule group's last rule is for a person. - Undo puts back the rule as it was kept in the group as it is now,
others' changes since left as they are. - Retiring on Azure turns the rule off (its own
enabled: false) and
deletes it four weeks later, approved again, as Grafana's. - Every request goes to
--ruler's host only, follows no redirect
(nor an AzurenextLinkelsewhere), and reads at most 64 MB for a list
and 4 MB for one unit, within 30 seconds; names read from a ruler go into
a path as one escaped segment. - The chart:
rules.ruler(url,tenant,tokenSecret), and
serve.changes.in,managed,rulerTokenSecret,awsRoleand
azureClientId, the write credential mounted only with changes on. Its
README gives the IAM policy and the Azure role. - Google Cloud's
Rules,ClusterRulesandGlobalRulesstay on the git
path: a test now holds that a newforand a new threshold are written
into one, the rest of the file as it was. test/e2e/changes.shstarts Mimir 3.2.4, which its Prometheus writes to,
and runs the ruler path against it. The weekly fuzzing reads the three
rulers' answers too.- Reviewed before release as an attacker would and for correctness; what
the reviews found is fixed: undo puts back one rule, only over the rule
as the change left it, every key of it, never a group's other rules
with it; a unit is read back by its rules' whole entries; one change to
a unit at a time, read back only where it was sent; names counted by
namespace and group together and escaped, so two can't read alike, and
none with a..segment written to, nor a rule's own name taken for one
Lapilli makes; one group Mimir writes in a form YAML can't read back
(|4), or one Amazon namespace whose file doesn't read, is passed over
and logged, not the whole read, and a Mimir namespace of more than 128
bytes is read; an Azure rule group's id is kept in one letter case; a
Mimir read back as it was is read again for two minutes.