Bug 1885235: bump Prometheus to v2.22.0

.github/workflows/prombench.yml

@@ -34,7 +34,7 @@ jobs:
       uses: docker://prominfra/prombench:master
       with:
         args: >-
-          until make all_nodepools_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done;
+          until make all_nodes_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done;
           make deploy;
     - name: Update status to failure
       if: failure()
@@ -69,7 +69,7 @@ jobs:
       uses: docker://prominfra/prombench:master
       with:
         args: >-
-          until make all_nodepools_running; do echo "waiting for nodepools to be created"; sleep 10; done;
+          until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done;
           make clean;
     - name: Update status to failure
       if: failure()
@@ -104,9 +104,9 @@ jobs:
       uses: docker://prominfra/prombench:master
       with:
         args: >-
-          until make all_nodepools_running; do echo "waiting for nodepools to be created"; sleep 10; done;
+          until make all_nodes_running; do echo "waiting for nodepools to be created"; sleep 10; done;
           make clean;
-          until make all_nodepools_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done;
+          until make all_nodes_deleted; do echo "waiting for nodepools to be deleted"; sleep 10; done;
           make deploy;
     - name: Update status to failure
       if: failure()

CHANGELOG.md

@@ -1,3 +1,21 @@
+## 2.22.0 / 2020-10-07
+
+As announced in the 2.21.0 release notes, the experimental gRPC API v2 has been
+removed.
+
+* [CHANGE] web: Remove APIv2. #7935
+* [ENHANCEMENT] React UI: Implement missing TSDB head stats section. #7876
+* [ENHANCEMENT] UI: Add Collapse all button to targets page. #6957
+* [ENHANCEMENT] UI: Clarify alert state toggle via checkbox icon. #7936
+* [ENHANCEMENT] Add `rule_group_last_evaluation_samples` and `prometheus_tsdb_data_replay_duration_seconds` metrics. #7737 #7977
+* [ENHANCEMENT] Gracefully handle unknown WAL record types. #8004
+* [ENHANCEMENT] Issue a warning for 64 bit systems running 32 bit binaries. #8012
+* [BUGFIX] Adjust scrape timestamps to align them to the intended schedule, effectively reducing block size. Workaround for a regression in go1.14+. #7976
+* [BUGFIX] promtool: Ensure alert rules are marked as restored in unit tests. #7661
+* [BUGFIX] Eureka: Fix service discovery when compiled in 32-bit. #7964
+* [BUGFIX] Don't do literal regex matching optimisation when case insensitive. #8013
+* [BUGFIX] Fix classic UI sometimes running queries for instant query when in range query mode. #7984
+
 ## 2.21.0 / 2020-09-11
 
 This release is built with Go 1.15, which deprecates [X.509 CommonName](https://golang.org/doc/go1.15#commonname)

Makefile.common

@@ -82,7 +82,7 @@ ifneq ($(shell which gotestsum),)
 endif
 endif
 
-PROMU_VERSION ?= 0.5.0
+PROMU_VERSION ?= 0.6.0
 PROMU_URL     := https://github.com/prometheus/promu/releases/download/v$(PROMU_VERSION)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM).tar.gz
 
 GOLANGCI_LINT :=

README.md

@@ -93,6 +93,7 @@ The Makefile provides several targets:
   * *format*: format the source code
   * *vet*: check the source code for common errors
   * *docker*: build a docker container for the current `HEAD`
+  * *assets*: build the new experimental React UI
 
 ## React UI Development
 

RELEASE.md

@@ -38,7 +38,7 @@ The release shepherd is responsible for the entire release series of a minor rel
 * We aim to keep the master branch in a working state at all times. In principle, it should be possible to cut a release from master at any time. In practice, things might not work out as nicely. A few days before the pre-release is scheduled, the shepherd should check the state of master. Following their best judgement, the shepherd should try to expedite bug fixes that are still in progress but should make it into the release. On the other hand, the shepherd may hold back merging last-minute invasive and risky changes that are better suited for the next minor release.
 * On the date listed in the table above, the release shepherd cuts the first pre-release (using the suffix `-rc.0`) and creates a new branch called  `release-<major>.<minor>` starting at the commit tagged for the pre-release. In general, a pre-release is considered a release candidate (that's what `rc` stands for) and should therefore not contain any known bugs that are planned to be fixed in the final release.
 * With the pre-release, the release shepherd is responsible for running and monitoring a benchmark run of the pre-release for 3 days, after which, if successful, the pre-release is promoted to a stable release.
-* If regressions or critical bugs are detected, they need to get fixed before cutting a new pre-release (called `-rc.1`, `-rc.2`, etc.). 
+* If regressions or critical bugs are detected, they need to get fixed before cutting a new pre-release (called `-rc.1`, `-rc.2`, etc.).
 
 See the next section for details on cutting an individual release.
 
@@ -62,13 +62,7 @@ Maintaining the release branches for older minor releases happens on a best effo
 
 ### 0. Updating dependencies
 
-A few days before a major or minor release, consider updating the dependencies:
-
-```
-make update-go-deps
-git add go.mod go.sum vendor
-git commit -m "Update dependencies"
-```
+A few days before a major or minor release, consider updating the dependencies.
 
 Then create a pull request against the master branch.
 
@@ -81,6 +75,32 @@ you can skip the dependency update or only update select dependencies. In such a
 case, you have to create an issue or pull request in the GitHub project for
 later follow-up.
 
+#### Updating Go dependencies
+
+```
+make update-go-deps
+git add go.mod go.sum vendor
+git commit -m "Update dependencies"
+```
+
+#### Updating React dependencies
+
+Either upgrade the dependencies within their existing version constraints as specified in the `package.json` file (see https://docs.npmjs.com/files/package.json#dependencies):
+
+```
+cd web/ui/react-app
+yarn upgrade
+git add yarn.lock
+```
+
+Or alternatively, update all dependencies to their latest major versions. This is potentially more disruptive and will require more follow-up fixes, but should be done from time to time (use your best judgement):
+
+```
+cd web/ui/react-app
+yarn upgrade --latest
+git add package.json yarn.lock
+```
+
 ### 1. Prepare your release
 
 At the start of a new major or minor release cycle create the corresponding release branch based on the master branch. For example if we're releasing `2.17.0` and the previous stable release is `2.16.0` we need to create a `release-2.17` branch. Note that all releases are handled in protected release branches, see the above `Branch management and versioning` section. Release candidates and patch releases for any given major or minor release happen in the same `release-<major>.<minor>` branch. Do not create `release-<version>` for patch or release candidate releases.

VERSION

@@ -1 +1 @@
-2.21.0
+2.22.0

cmd/prometheus/main.go

@@ -19,6 +19,7 @@ import (
 	"fmt"
 	"io"
 	"math"
+	"math/bits"
 	"net"
 	"net/http"
 	_ "net/http/pprof" // Comment this line to disable pprof endpoint.
@@ -48,7 +49,8 @@ import (
 	jprom "github.com/uber/jaeger-lib/metrics/prometheus"
 	"go.uber.org/atomic"
 	kingpin "gopkg.in/alecthomas/kingpin.v2"
-	"k8s.io/klog"
+	klog "k8s.io/klog"
+	klogv2 "k8s.io/klog/v2"
 
 	promlogflag "github.com/prometheus/common/promlog/flag"
 	"github.com/prometheus/prometheus/config"
@@ -239,6 +241,9 @@ func main() {
 	a.Flag("rules.alert.resend-delay", "Minimum amount of time to wait before resending an alert to Alertmanager.").
 		Default("1m").SetValue(&cfg.resendDelay)
 
+	a.Flag("scrape.adjust-timestamps", "Adjust scrape timestamps by up to 2ms to align them to the intended schedule. See https://github.com/prometheus/prometheus/issues/7846 for more context. Experimental. This flag will be removed in a future release.").
+		Hidden().Default("true").BoolVar(&scrape.AlignScrapeTimestamps)
+
 	a.Flag("alertmanager.notification-queue-capacity", "The capacity of the queue for pending Alertmanager notifications.").
 		Default("10000").IntVar(&cfg.notifier.QueueCapacity)
 
@@ -342,8 +347,14 @@ func main() {
 	// Above level 6, the k8s client would log bearer tokens in clear-text.
 	klog.ClampLevel(6)
 	klog.SetLogger(log.With(logger, "component", "k8s_client_runtime"))
+	klogv2.ClampLevel(6)
+	klogv2.SetLogger(log.With(logger, "component", "k8s_client_runtime"))
 
 	level.Info(logger).Log("msg", "Starting Prometheus", "version", version.Info())
+	if bits.UintSize < 64 {
+		level.Warn(logger).Log("msg", "This Prometheus binary has not been compiled for a 64-bit architecture. Due to virtual memory constraints of 32-bit systems, it is highly recommended to switch to a 64-bit binary of Prometheus.", "GOARCH", runtime.GOARCH)
+	}
+
 	level.Info(logger).Log("build_context", version.BuildContext())
 	level.Info(logger).Log("host_details", prom_runtime.Uname())
 	level.Info(logger).Log("fd_limits", prom_runtime.FdLimits())

cmd/promtool/testdata/unittest.yml

@@ -8,6 +8,14 @@ tests:
     input_series:
       - series: 'up{job="prometheus", instance="localhost:9090"}'
         values: "0+0x1440"
+
+    promql_expr_test:
+      - expr: count(ALERTS) by (alertname, alertstate)
+        eval_time: 4m
+        exp_samples:
+        - labels: '{alertname="InstanceDown",alertstate="pending"}'
+          value: 1
+
     alert_rule_test:
       - eval_time: 1d
         alertname: InstanceDown

cmd/promtool/unittest.go

@@ -221,6 +221,16 @@ func (tg *testGroup) test(mint, maxt time.Time, evalInterval time.Duration, grou
 	// Current index in alertEvalTimes what we are looking at.
 	curr := 0
 
+	for _, g := range groups {
+		for _, r := range g.Rules() {
+			if alertRule, ok := r.(*rules.AlertingRule); ok {
+				// Mark alerting rules as restored, to ensure the ALERTS timeseries is
+				// created when they run.
+				alertRule.SetRestored(true)
+			}
+		}
+	}
+
 	var errs []error
 	for ts := mint; ts.Before(maxt); ts = ts.Add(evalInterval) {
 		// Collects the alerts asked for unit testing.

config/config.go

@@ -104,16 +104,16 @@ var (
 
 	// DefaultQueueConfig is the default remote queue configuration.
 	DefaultQueueConfig = QueueConfig{
-		// With a maximum of 1000 shards, assuming an average of 100ms remote write
-		// time and 100 samples per batch, we will be able to push 1M samples/s.
-		MaxShards:         1000,
+		// With a maximum of 200 shards, assuming an average of 100ms remote write
+		// time and 500 samples per batch, we will be able to push 1M samples/s.
+		MaxShards:         200,
 		MinShards:         1,
-		MaxSamplesPerSend: 100,
+		MaxSamplesPerSend: 500,
 
-		// Each shard will have a max of 500 samples pending in it's channel, plus the pending
-		// samples that have been enqueued. Theoretically we should only ever have about 600 samples
-		// per shard pending. At 1000 shards that's 600k.
-		Capacity:          500,
+		// Each shard will have a max of 2500 samples pending in its channel, plus the pending
+		// samples that have been enqueued. Theoretically we should only ever have about 3000 samples
+		// per shard pending. At 200 shards that's 600k.
+		Capacity:          2500,
 		BatchSendDeadline: model.Duration(5 * time.Second),
 
 		// Backoff times for retrying a batch of samples on recoverable errors.

discovery/eureka/client.go

@@ -22,8 +22,11 @@ import (
 	"net/http"
 
 	"github.com/pkg/errors"
+	"github.com/prometheus/common/version"
 )
 
+var userAgent = fmt.Sprintf("Prometheus/%s", version.Version)
+
 type Applications struct {
 	VersionsDelta int           `xml:"versions__delta"`
 	AppsHashcode  string        `xml:"apps__hashcode"`
@@ -55,8 +58,6 @@ type Instance struct {
 	DataCenterInfo                *DataCenterInfo `xml:"dataCenterInfo"`
 	Metadata                      *MetaData       `xml:"metadata"`
 	IsCoordinatingDiscoveryServer bool            `xml:"isCoordinatingDiscoveryServer"`
-	LastUpdatedTimestamp          int             `xml:"lastUpdatedTimestamp"`
-	LastDirtyTimestamp            int             `xml:"lastDirtyTimestamp"`
 	ActionType                    string          `xml:"actionType"`
 	CountryID                     int             `xml:"countryId"`
 	InstanceID                    string          `xml:"instanceId"`
@@ -87,6 +88,7 @@ func fetchApps(ctx context.Context, server string, client *http.Client) (*Applic
 		return nil, err
 	}
 	request = request.WithContext(ctx)
+	request.Header.Add("User-Agent", userAgent)
 
 	resp, err := client.Do(request)
 	if err != nil {

discovery/hetzner/robot.go

@@ -29,6 +29,7 @@ import (
 	"github.com/pkg/errors"
 	"github.com/prometheus/common/config"
 	"github.com/prometheus/common/model"
+	"github.com/prometheus/common/version"
 	"github.com/prometheus/prometheus/discovery/refresh"
 	"github.com/prometheus/prometheus/discovery/targetgroup"
 )
@@ -39,6 +40,8 @@ const (
 	hetznerLabelRobotCancelled = hetznerRobotLabelPrefix + "cancelled"
 )
 
+var userAgent = fmt.Sprintf("Prometheus/%s", version.Version)
+
 // Discovery periodically performs Hetzner Robot requests. It implements
 // the Discoverer interface.
 type robotDiscovery struct {
@@ -67,7 +70,14 @@ func newRobotDiscovery(conf *SDConfig, logger log.Logger) (*robotDiscovery, erro
 	return d, nil
 }
 func (d *robotDiscovery) refresh(ctx context.Context) ([]*targetgroup.Group, error) {
-	resp, err := d.client.Get(d.endpoint + "/server")
+	req, err := http.NewRequest("GET", d.endpoint+"/server", nil)
+	if err != nil {
+		return nil, err
+	}
+
+	req.Header.Add("User-Agent", userAgent)
+
+	resp, err := d.client.Do(req)
 	if err != nil {
 		return nil, err
 	}

docs/configuration/alerting_rules.md

@@ -86,7 +86,7 @@ the "Alerts" tab of your Prometheus instance. This will show you the exact
 label sets for which each defined alert is currently active.
 
 For pending and firing alerts, Prometheus also stores synthetic time series of
-the form `ALERTS{alertname="<alert name>", alertstate="pending|firing", <additional alert labels>}`.
+the form `ALERTS{alertname="<alert name>", alertstate="<pending or firing>", <additional alert labels>}`.
 The sample value is set to `1` as long as the alert is in the indicated active
 (pending or firing) state, and the series is marked stale when this is no
 longer the case.

docs/installation.md

@@ -27,9 +27,9 @@ prom/prometheus`. This starts Prometheus with a sample
 configuration and exposes it on port 9090.
 
 The Prometheus image uses a volume to store the actual metrics. For
-production deployments it is highly recommended to use the
-[Data Volume Container](https://docs.docker.com/engine/admin/volumes/volumes/)
-pattern to ease managing the data on Prometheus upgrades.
+production deployments it is highly recommended to use a
+[named volume](https://docs.docker.com/storage/volumes/)
+to ease managing the data on Prometheus upgrades.
 
 To provide your own configuration, there are several options. Here are
 two examples.
@@ -41,11 +41,12 @@ Bind-mount your `prometheus.yml` from the host by running:
 ```bash
 docker run \
     -p 9090:9090 \
-    -v /tmp/prometheus.yml:/etc/prometheus/prometheus.yml \
+    -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
     prom/prometheus
 ```
 
-Or use an additional volume for the config:
+Or bind-mount the directory containing `prometheus.yml` onto
+`/etc/prometheus` by running:
 
 ```bash
 docker run \

docs/migration.md

@@ -120,7 +120,7 @@ new format. For example:
 $ promtool update rules example.rules
 ```
 
-Note that you will need to use promtool from 2.0, not 1.8.
+You will need to use `promtool` from [Prometheus 2.5](https://github.com/prometheus/prometheus/releases/tag/v2.5.0) as later versions no longer contain the above subcommand.
 
 ## Storage