Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from upstream repository incl. go1.20 compatibily patches #240

Merged
merged 48 commits into from
Jan 11, 2024

Conversation

periklis
Copy link

@periklis periklis commented Jan 11, 2024

JoaoBraveCoding and others added 30 commits December 20, 2023 20:49
The bloom gateway client panicked, because the ring, from which it gets
the subring for the tenant was never passed as argument in the
constructor, even though it was part of the client struct.

---------

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
The index gateways discovers the bloom gateways by using the bloom
gateway ring.
In order to initialize the ring before the index gateway, it needs to be
added as dependency of the index gateway.

The dependency is conditional, because we don't want the ring to be
initialized even though the bloom gateway is not used.

**Special notes for your reviewer**:

This regression was introduced with
grafana#11537

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…rafana#11505)

**What this PR does / why we need it**:
Adds support for tracking queries referencing structured metadata. It
tracks the following kinds of references:
* Fields in structured metadata used as label filters, `drop`/`keep` stages etc. in the query.
* Fields in structured metadata used in metric queries like `uwrap` and `by` or `without` clauses in
aggregations.

**Checklist**
- [x] Tests updated
**What this PR does / why we need it**:
Bloom-compactor panics for duplicate metrics registration in
bloom-tokenizers.
This PR adds labels to bloom-tokenizer metrics, adopting same solution
in grafana#11536
**What this PR does / why we need it**:
Currently you suggest to download the files manually instead of using
apt/rpm which means you don't get any updates. I thing using apt/rpm
would be better.

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

I admit I have not read the full
https://grafana.com/docs/writers-toolkit/ guide, seems a lot of work for
such a small change 😅

---------

Co-authored-by: J Stickler <julie.stickler@grafana.com>
**What this PR does / why we need it**:
We saw that `JoinQuantileSketchVector` was using too much memory. This
is a small step to improve the allocated memory.

```
pkg: github.com/grafana/loki/pkg/logql
                            │  main.log   │              pool.log              │
                            │   sec/op    │   sec/op     vs base               │
JoinQuantileSketchVector-10   3.603µ ± 4%   2.631µ ± 0%  -26.99% (p=0.002 n=6)

                            │   main.log   │              pool.log               │
                            │     B/op     │     B/op      vs base               │
JoinQuantileSketchVector-10   8.344Ki ± 0%   2.539Ki ± 0%  -69.57% (p=0.002 n=6)

                            │  main.log  │             pool.log             │
                            │ allocs/op  │ allocs/op   vs base              │
JoinQuantileSketchVector-10   109.0 ± 0%   104.0 ± 0%  -4.59% (p=0.002 n=6)
```

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Callum Styan <callumstyan@gmail.com>
…1566)

The `Ref` of `MetaRef` and `BlockRef` do have start and end timestamps,
which are of type `int64`.
This however, can lead to the problem that different components might
use different precisions (`s`, `ms`, `ns`) for these timestamps.

This PR changes the current implementation of the bloom shipper to use
`ms` precision timestamps (using `model.Time`) instead of `s`, to match
the behaviour that we have to encode ChunkRefs to object store keys.

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…1568)

Also updated the Docker image to match the latest version

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
It is useful for implementations of Tee to also know the tenant who sent
the logs. This PR adds tenant to the tee interface.
…rafana#11539)

**What this PR does / why we need it**:
Add support for caching metadata queries (both series and labels).
caching happens after splitting similar to other types of queries.

This pr adds the following configs to enable them.
```
cache_series_results: true|false (default false)
cache_label_results: true|false (default false)
```
And the cache backend for them can be configured using
`series_results_cache` and `label_results_cache` blocks under the
`query_range` section.

Currently the split interval for metadata queries is fixed and defaults
to 24h, this pr makes it configurable by introducing
`split_metadata_queries_by_interval`

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

---------

Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>
Co-authored-by: Ashwanth Goli <iamashwanth@gmail.com>
**What this PR does / why we need it**:
`BloomCompactorMinTableAge` was added with the idea that we will skip
most recent index files from ingesters that are not compacted into TSDB
indexes yet.

The default value is an hour. This blocks bloom-compacter processing
TSBD of the day, because end timestamp is always in the last 15 mins. We
can either reduce it something lower than the frequency of TSDB indexes
being built (< 15mins).
Here I choose to remove it altogether, assuming uncompacted indexes from
ingesters will not be processed as a table as there won't be a schema
for that table with the [check
here](https://github.com/grafana/loki/blob/main/pkg/bloomcompactor/bloomcompactor.go#L268-L272).
**What this PR does / why we need it**:

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
…og tenant (grafana#11579)

**What this PR does / why we need it**:
Adds example for querying against Grafana Cloud log tenant to API docs. 


**Which issue(s) this PR fixes**:
Addresses internal doc request that came about via a Support ticket

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
**What this PR does / why we need it**:

Add a metrics.go log line when a querier reads from an ingester. The
output is like this:

```
level=info ts=2024-01-03T13:59:06.919965546Z caller=metrics.go:275 component=ingester org_id=fake traceID=0b216e7d014c5f87 latency=fast query_type=ingester_series start=2024-01-03T13:00:00Z end=2024-01-03T13:59:06.834Z start_delta=59m6.91996413s end_delta=85.964255ms length=59m6.834s duration=110.792µs status=200 query=<omitted> query_hash=2166136261 total_entries=9
```

**Checklist**
- [X] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [X] Tests updated
- [X] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

---------

Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
…fectiveness (grafana#11589)

**What this PR does / why we need it**:
cache hit rate that is currently being measured using metrics and stats
does not account for the fact that a cache hit could return partial
results. When we query the cache for a key, we get back a list of
extents and these need no cover the entire (split) range of the cache
key.

This pr adds a new cache stat called `query_length_served` to better
measure the cache effectiveness.
```
query_length - sum(length of downstream queries)
```

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
**What this PR does / why we need it**:
We've found that the index mapping of our quantile over time
approximation has a big impact on the CPU. Changing the implementation
gives us around 50%. The mapping is used during the `At` calls to the
iterator.

```
› benchstat logarithmic.log cubic.log sort.log pool.log 
goos: linux
goarch: amd64
pkg: github.com/grafana/loki/pkg/logql
cpu: AMD Ryzen 7 3700X 8-Core Processor             
                                                     │ logarithmic.log │              cubic.log               │              sort.log               │              pool.log              │
                                                     │     sec/op      │    sec/op      vs base               │    sec/op     vs base               │   sec/op     vs base               │
QuantileBatchRangeVectorIteratorAt/1-samples-16            819.7n ± 2%   1052.5n ±  5%  +28.40% (p=0.002 n=6)   1055.0n ± 1%  +28.71% (p=0.002 n=6)   303.7n ± 4%  -62.96% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/1000-samples-16         60.34µ ± 8%    49.32µ ± 13%  -18.26% (p=0.002 n=6)    45.94µ ± 4%  -23.86% (p=0.002 n=6)   24.97µ ± 3%  -58.61% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/100000-samples-16       3.032m ± 3%    1.319m ±  1%  -56.50% (p=0.002 n=6)    1.316m ± 4%  -56.58% (p=0.002 n=6)   1.278m ± 3%  -57.86% (p=0.002 n=6)
geomean                                                    53.13µ         40.91µ        -23.00%                  39.96µ       -24.79%                 21.32µ       -59.87%

                                                     │ logarithmic.log │              cubic.log               │               sort.log               │             pool.log              │
                                                     │      B/op       │     B/op      vs base                │     B/op      vs base                │    B/op     vs base               │
QuantileBatchRangeVectorIteratorAt/1-samples-16            368.00 ± 0%    368.00 ± 0%       ~ (p=1.000 n=6) ¹    368.00 ± 0%       ~ (p=1.000 n=6) ¹   32.00 ± 0%  -91.30% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/1000-samples-16         4048.0 ± 0%    4048.0 ± 0%       ~ (p=1.000 n=6) ¹    3920.0 ± 0%  -3.16% (p=0.002 n=6)     104.0 ± 0%  -97.43% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/100000-samples-16       6192.0 ± 0%    6192.0 ± 0%       ~ (p=1.000 n=6) ¹    5936.0 ± 0%  -4.13% (p=0.002 n=6)     202.0 ± 5%  -96.74% (p=0.002 n=6)
geomean                                                   2.048Ki        2.048Ki       +0.00%                   1.998Ki       -2.45%                   87.60       -95.82%
¹ all samples are equal

                                                     │ logarithmic.log │              cubic.log              │               sort.log               │             pool.log              │
                                                     │    allocs/op    │  allocs/op   vs base                │  allocs/op   vs base                 │ allocs/op   vs base               │
QuantileBatchRangeVectorIteratorAt/1-samples-16             8.000 ± 0%    8.000 ± 0%       ~ (p=1.000 n=6) ¹    8.000 ± 0%        ~ (p=1.000 n=6) ¹   2.000 ± 0%  -75.00% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/1000-samples-16         27.000 ± 0%   27.000 ± 0%       ~ (p=1.000 n=6) ¹   23.000 ± 0%  -14.81% (p=0.002 n=6)     5.000 ± 0%  -81.48% (p=0.002 n=6)
QuantileBatchRangeVectorIteratorAt/100000-samples-16       42.000 ± 0%   42.000 ± 0%       ~ (p=1.000 n=6) ¹   34.000 ± 0%  -19.05% (p=0.002 n=6)     9.000 ± 0%  -78.57% (p=0.002 n=6)
geomean                                                     20.86         20.86       +0.00%                    18.43       -11.65%                   4.481       -78.51%
¹ all samples are equal

```

 

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
**What this PR does / why we need it**:
This PR is the result of debugging sprint on bloom-compactor on a dev
cell.
With these changes, compactor now can create and upload blooms and
relevant meta files from scratch.

- Queries tsdb correctly with empty matchers to get all series
- Seeks to the beginning of the bloom data file before uploading
- Adds uuid strings to local file name, as checksum is not available at
the time of directory creation
- Creates meta object with all fields populated
- Additional logging

edit: confirmed that it runs as expected in the dev cell.

---------

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Paul Rogers <paul.rogers@grafana.com>
**Which issue(s) this PR fixes**:
Fixes grafana#11565
**What this PR does / why we need it**:

Updates the `description` field in the metadata to support the Docs
Landing Page project

---------

Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
…rafana#11595)

The clone function in rangemapper has been a persistent source of panics
due to the way stringifying exprs changed the way the original query was
represented. This PR replaces the `clone` function with
`syntax.MustClone`. It clones the expression by actually walking the AST
rather than doing `expr -> string -> parse`. It might still panic but
for a more fundamental reason than because of issues with the string
representation.
…grafana#11601)

**What this PR does / why we need it**:
A ruler handling many hundreds of rules can provoke a situation where
the WAL appender reads & modifies tenant configs concurrently in an
unsafe way; this PR protects that with a mutex.

**Which issue(s) this PR fixes**:
Fixes grafana#11569
…cutively (grafana#11604)

**What this PR does / why we need it**:
Earlier we had a counter for the number of times tables were skipped by
compaction due to them being locked by retention. Counter makes it hard
to reliably write an alert without making it noisy. I am changing the
counter to gauge which would count the number of times tables were
consecutively skipped by compaction. This would make it easier to write
alerts.

**Checklist**
- [x] Tests updated

---------

Co-authored-by: Ashwanth <iamashwanth@gmail.com>
…dest table that has it indexed (grafana#11600)

**What this PR does / why we need it**:
When a delete request has a line filter and/or is deleting a chunk
partially, we create a new chunk from the source chunk and mark the
original chunk for deletion. However, when a chunk overlaps multiple
tables, we apply heuristics to retain the chunk until all the tables
that have it indexed get processed since we need the original chunk to
build a new one.

Earlier, we used to process tables from oldest to newest, so the logic
to detect if all the tables indexing a chunk are processed was done by
seeing if `chunk-end-time` <= `table-end-time`. To put it in simple
terms, if the chunk ends in the current table we are processing, it
means this is the last table that we are processing, which has the chunk
indexed.

However, this does not work anymore since we changed the order of
processing tables to process the newest tables first. To get it working
back, I have updated the logic to `chunk-start-time` >=
`table-start-time`. In simple terms, if the chunk starts in the current
table we are processing, it means this is the last table we are
processing, which has the chunk indexed.

**Checklist**
- [x] Tests updated
…when server is disabled (grafana#11420)

**What this PR does / why we need it**:
This PR improves the clarity of log message when the Promtail server is
disabled (if `server.disable` is true in config file, or flag
`-server.disable` is used).

Previously, the reason "promtailServer cast fail" in log message (as
below) was ambiguous, leading to confusion about potential error in
config file.
```
level=warn ts=2023-12-09T15:40:24.064376445Z caller=promtail.go:260 msg="disable watchConfig" reason="promtailServer cast fail"
```

The updated message is more understandable, indicating directly that the
Promtail server is disabled.
```
level=warn ts=2023-12-09T15:50:58.723168763Z caller=promtail.go:260 msg="disable watchConfig" reason="Promtail server is disabled"
```

**Which issue(s) this PR fixes**:
N/A.

**Special notes for your reviewer**:
Should we turn down the log level to INFO?

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
…onse bodies (grafana#11606)

**What this PR does / why we need it**:
grafana#11487 introduced a regression which added trailing newlines to HTTP
error response bodies, which may affect client integrations.

This regression hasn't made it into the 2.9.x or 2.8.x releases yet so
no backporting is necessary.
edsoncelio and others added 18 commits January 8, 2024 18:02
**What this PR does / why we need it**:

**Which issue(s) this PR fixes**:
Fixes grafana#9928

**Special notes for your reviewer**:
This PR adds a note about an important requirement in the format of Loki
labels (with a small example).

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

---------

Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
…#9831)

**What this PR does / why we need it**:

In promtail, when a file matching the exclude path is created in a
directory that is being watched, this file should not be tailed by
promtail. However, previously this was the case. As described in grafana#7115.
This change checks the filename to the excludePath and if it matches,
ignores the file.

**Which issue(s) this PR fixes**:
Fixes grafana#7115

**Special notes for your reviewer**:
We tested manually this by running Promtail locally and creating a bunch
of files which should and should not be tailed by Promtail.

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Tests updated
- [x] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`
Expands on grafana#11601

**What this PR does / why we need it**:
Turns out the previous tests didn't expose all possible causes for data
races (another one occurs at
https://github.com/grafana/loki/blob/5a55158cc751465846383bc758aa0c169363b292/pkg/ruler/registry.go#L204).
Moving the mutex to the calling function adds more safety.

**Which issue(s) this PR fixes**:
Fixes grafana#11569

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
**What this PR does / why we need it**:
Updates links to path for a couple of example files that were missed in
the information architecture moves.

**Which issue(s) this PR fixes**:
Fixes grafana#11617 

(This is in addition to the updates in
grafana#11568)
Fix the link to the docker-driver documentation.

Co-authored-by: J Stickler <julie.stickler@grafana.com>
Big thanks to @slim-bean, even after ~1 day of trying to find a cause
here I didn't track down this issue, that I was returning something to
the pool that hadn't actually been retrieved from the pool.

The test here in the 2nd commit will fail if you pass the `-race` flag
without the fix from the 3rd commit.

---------

Signed-off-by: Callum Styan <callumstyan@gmail.com>
…hart error (grafana#11621)

**What this PR does / why we need it**:
The code in
https://github.com/grafana/loki/blob/main/production/helm/loki/templates/_helpers.tpl
file doesn't include "swift" type object storage.
when user using swift by configure `loki.storage.swift.*`, then
`loki.storage.type` shall = `swift` , but the
"loki.isUsingObjectStorage" cannot get the proper value in this case.

As of now "swift" type object storage has been added to resolve Loki
HELM Chart error.
**Which issue(s) this PR fixes**:.

Fixes grafana#11611 

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [x] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Fix incorrect example

**What this PR does / why we need it**:
Fix incorrect example in [Template
functions](https://grafana.com/docs/loki/latest/query/template_functions/)

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)

Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
…d. (grafana#11628)

**What this PR does / why we need it**:
If `quantile_over_time` sharding is disabled the code would still shard
it naively. That was a bug.

**Checklist**
- [ ] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
Adds support for users to configure the S3 object storage with AWS STS.

Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>
Co-authored-by: Periklis Tsirakidis <periklis@redhat.com>
**What this PR does / why we need it**:
Completing the work that was started in grafana#9409 but not completed because
there were multiple variations on how to format notes in the docs source
files. We rolled out different colors for admonitions recently, which
highlighted that there were still a few notes not using the admonition
shortcode. I searched the docs for the following words:
* tip
* caution
* important
* warning
* note

So hopefully I've caught all of the admonitions that weren't using the
shortcode.
**What this PR does / why we need it**:

Bumps alpine in the loki build image and ass the build image to the
`scan-vulnerabilities` target to make it easier to scan in the future

---------

Co-authored-by: Danny Kopping <danny.kopping@grafana.com>
…#11635)

Binary expressions with one un-shardable side have been broken for quite some time. The error `unimplemented` however was shadowed by some other code. It surfaced only with a fix for an issue with `quantile_over_time` with was solved with grafana#11629  

---------

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Co-authored-by: Christian Haudum <christian.haudum@gmail.com>
**What this PR does / why we need it**:

doc is confusing/lacking detail on setting up regular fluentbit
container and we needed an example config for fluent-bit that works with
HG Loki.

**Which issue(s) this PR fixes**:
Fixes #<issue number>

**Special notes for your reviewer**:

**Checklist**
- [x] Reviewed the
[`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md)
guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] `CHANGELOG.md` updated
- [ ] If the change is worth mentioning in the release notes, add
`add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are
documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in
`production/helm/loki/Chart.yaml` and update
`production/helm/loki/CHANGELOG.md` and
`production/helm/loki/README.md`. [Example
PR](grafana@d10549e)
- [ ] If the change is deprecating or removing a configuration option,
update the `deprecated-config.yaml` and `deleted-config.yaml` files
respectively in the `tools/deprecated-config-checker` directory.
[Example
PR](grafana@0d4416a)
**What this PR does / why we need it**:
This upates the loki-build-image tag for Drone so it's also published
for ARM.
…ana#11620)

Updated RBAC now match the new updated requirements by UWM Alertmanager introduced on:
openshift/cluster-monitoring-operator#2099
@periklis periklis self-assigned this Jan 11, 2024
Copy link

openshift-ci bot commented Jan 11, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: periklis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 11, 2024
Copy link

openshift-ci bot commented Jan 11, 2024

@periklis: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@periklis periklis merged commit c7e5b4d into openshift:main Jan 11, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet