Fix Peter's comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

CHANGELOG.md

@@ -19,8 +19,9 @@
 * [CHANGE] Memberlist: `-memberlist.abort-if-join-fails` now defaults to false. Previously it defaulted to true. #2168
 * [CHANGE] Ingester: `-ingester.exemplars-update-period` has been renamed to `-ingester.tsdb-config-update-period` and is used to update multiple per-tenant TSDB config. #2187
 * [FEATURE] Ingester: Add experimental ability to ingest out of order samples up to an allowed limit. Enabling this takes additional memory and disk space. It also enables a write behind log that could lead to longer ingester start replays. There is no overhead on memory, disk space, startup times, with it being disabled. #2187
-  * `-ingester.out-of-order-allowance` allows setting how back in time a sample can be as duration string. Defaults to `0s`.
+  * `-ingester.out-of-order-time-window` allows setting how back in time a sample can be as duration string. Defaults to `0s`.
   * `cortex_ingester_tsdb_out_of_order_samples_appended_total` metric tracks the total number of out of samples ingested by the ingester.
+  * `cortex_discarded_samples_total` gets a new label `reason="sample-too-old"` when the `-ingester.out-of-order-time-window` flag is >0 to track number of samples that were discarded for being too old (out of order but beyond the time window allowed). 
 * [ENHANCEMENT] Distributor: Added limit to prevent tenants from sending excessive number of requests: #1843
   * The following CLI flags (and their respective YAML config options) have been added:
     * `-distributor.request-rate-limit`

cmd/mimir/config-descriptor.json

@@ -2652,7 +2652,7 @@
           "kind": "field",
           "name": "out_of_order_time_window",
           "required": false,
-          "desc": "Non-zero value enables out-of-order support for most recent samples in this time window. Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.",
+          "desc": "Non-zero value enables out-of-order support for most recent samples that are within the time window w.r.t. the newest sample for that time series if it exists (i.e. within [series.maxTime-timeWindow, series.maxTime]) or w.r.t. the TSDB's max time if the series does not exist (i.e. within [db.maxTime-timeWindow, db.maxTime]). Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.",
           "fieldValue": null,
           "fieldDefaultValue": 0,
           "fieldFlag": "ingester.out-of-order-time-window",

cmd/mimir/help-all.txt.tmpl

@@ -866,7 +866,7 @@ Usage of ./cmd/mimir/mimir:
   -ingester.metadata-retain-period duration
     	Period at which metadata we have not seen will remain in memory before being deleted. (default 10m0s)
   -ingester.out-of-order-time-window value
-    	[experimental] Non-zero value enables out-of-order support for most recent samples in this time window. Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.
+    	[experimental] Non-zero value enables out-of-order support for most recent samples that are within the time window w.r.t. the newest sample for that time series if it exists (i.e. within [series.maxTime-timeWindow, series.maxTime]) or w.r.t. the TSDB's max time if the series does not exist (i.e. within [db.maxTime-timeWindow, db.maxTime]). Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.
   -ingester.rate-update-period duration
     	Period with which to update the per-tenant ingestion rates. (default 15s)
   -ingester.ring.consul.acl-token string

docs/sources/operators-guide/configuring/reference-configuration-parameters/index.md

@@ -2721,9 +2721,12 @@ The `limits` block configures default and per-tenant limits imposed by component
 [active_series_custom_trackers: <map of tracker name (string) to matcher (string)> | default = ]
 
 # (experimental) Non-zero value enables out-of-order support for most recent
-# samples in this time window. Ingester will need more memory that is a factor
-# of rate of out of order sample being ingested and number of series getting out
-# of order samples. It can be configured per-tenant.
+# samples that are within the time window w.r.t. the newest sample for that time
+# series if it exists (i.e. within [series.maxTime-timeWindow, series.maxTime])
+# or w.r.t. the TSDB's max time if the series does not exist (i.e. within
+# [db.maxTime-timeWindow, db.maxTime]). Ingester will need more memory that is a
+# factor of rate of out of order sample being ingested and number of series
+# getting out of order samples. It can be configured per-tenant.
 # CLI flag: -ingester.out-of-order-time-window
 [out_of_order_time_window: <duration> | default = 0s]
 

docs/sources/operators-guide/mimir-runbooks/_index.md

@@ -1415,7 +1415,7 @@ Common **causes**:
 ### err-mimir-sample-too-old
 
 This error is very similar to `err-mimir-sample-out-of-order` above. The main difference is that, the out-of-order support was enabled, but the sample was
-older than the out-of-order allowance w.r.t. the latest sample for that particular time series or the TSDB.
+older than the out-of-order time window w.r.t. the latest sample for that particular time series or the TSDB.
 
 ### err-mimir-sample-duplicate-timestamp
 

pkg/util/validation/limits.go

@@ -181,7 +181,7 @@ func (l *Limits) RegisterFlags(f *flag.FlagSet) {
 	f.IntVar(&l.MaxGlobalMetadataPerMetric, MaxMetadataPerMetricFlag, 0, "The maximum number of metadata per metric, across the cluster. 0 to disable.")
 	f.IntVar(&l.MaxGlobalExemplarsPerUser, "ingester.max-global-exemplars-per-user", 0, "The maximum number of exemplars in memory, across the cluster. 0 to disable exemplars ingestion.")
 	f.Var(&l.ActiveSeriesCustomTrackersConfig, "ingester.active-series-custom-trackers", "Additional active series metrics, matching the provided matchers. Matchers should be in form <name>:<matcher>, like 'foobar:{foo=\"bar\"}'. Multiple matchers can be provided either providing the flag multiple times or providing multiple semicolon-separated values to a single flag.")
-	f.Var(&l.OutOfOrderTimeWindow, "ingester.out-of-order-time-window", "Non-zero value enables out-of-order support for most recent samples in this time window. Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.")
+	f.Var(&l.OutOfOrderTimeWindow, "ingester.out-of-order-time-window", "Non-zero value enables out-of-order support for most recent samples that are within the time window w.r.t. the newest sample for that time series if it exists (i.e. within [series.maxTime-timeWindow, series.maxTime]) or w.r.t. the TSDB's max time if the series does not exist (i.e. within [db.maxTime-timeWindow, db.maxTime]). Ingester will need more memory that is a factor of rate of out of order sample being ingested and number of series getting out of order samples. It can be configured per-tenant.")
 
 	f.IntVar(&l.MaxChunksPerQuery, MaxChunksPerQueryFlag, 2e6, "Maximum number of chunks that can be fetched in a single query from ingesters and long-term storage. This limit is enforced in the querier, ruler and store-gateway. 0 to disable.")
 	f.IntVar(&l.MaxFetchedSeriesPerQuery, MaxSeriesPerQueryFlag, 0, "The maximum number of unique series for which a query can fetch samples from each ingesters and storage. This limit is enforced in the querier and ruler. 0 to disable")