Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from upstream repository #276

Merged
merged 62 commits into from Mar 25, 2024
Merged

Update from upstream repository #276

merged 62 commits into from Mar 25, 2024

Conversation

periklis
Copy link

@periklis periklis commented Mar 25, 2024

Refs:

salvacorts and others added 30 commits March 15, 2024 09:10
…cs. (#12142)

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
…emaconfig (#11852)

Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
Co-authored-by: Dylan Guedes <djmgguedes@gmail.com>
…rpc_listen_port instead of hardcoded value (#11646)

Signed-off-by: Sheikh-Abubaker <sheikhabubaker761@gmail.com>
Co-authored-by: Alberto Chiusole <chiusole@seqera.io>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
…n templates (#12242)

Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
Signed-off-by: Jan-Otto Kröpke <joe@cloudeteer.de>
Co-authored-by: Salva Corts <salva.corts@grafana.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
Every time when reading a bloom page into memory, a new byte slice with the size of the decoded page was allocated.

This PR changes bloom page decoder to use a byte slice from the `BlockPool`. The caller (`LazyBloomIter` of the block querier) is responsible for closing the page whenever a new one is loaded, so the byte slice is put back to its pool. 

Additionally, this PR changes the max bucket of the `BlockPool` byte slice pool to 64MB so that pages bigger than 16MB are correctly put back to the pool, since [slices exceeding the capacity of the largest bucket are not put back](https://github.com/grafana/loki/blob/main/vendor/github.com/prometheus/prometheus/util/pool/pool.go#L74-L87).

---
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
**What this PR does / why we need it**:
This change allows the retrieval of the `TenantConfig` or the `DefaultConfig`. It also defines an interface instead of a tenant config function which makes it a little easier to grok the code.

Eventually this will enable us to introduce GEL specific configurations.

**Checklist**
- [ ] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**)
- [ ] Documentation added
- [x] Tests updated
- [ ] `CHANGELOG.md` updated
  - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md`
- [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](d10549e)
- [ ] If the change is deprecating or removing a configuration option, update the `deprecated-config.yaml` and `deleted-config.yaml` files respectively in the `tools/deprecated-config-checker` directory. [Example PR](0d4416a)
… wal (#12219)

Co-authored-by: J Stickler <julie.stickler@grafana.com>
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
The blocks cache is an in-memory cache that represents the single source of truth of what blocks are available on the file system and manages their end-of-life behaviour.

There is a follow-up branch to wire up the new cache, based on this branch. 

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
jeschkies and others added 23 commits March 21, 2024 12:37
…e size for querying (#12297)

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
A concurrency of 10 per worker seems to be too high for a reasonable 8GiB memory limit on bloom gateways. So this PR makes the concurrency configurable and uses a default of 4.

Memory usage of loading block pages into memory therefore being:

```
worker_concurrency x block_query_concurrency x max_block_page_size x 2
```

With current defaults:

```
4 x 4 x 32MiB x 2 = 1GiB
```

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…12311)

Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Signed-off-by: depthlending <bikangning@outlook.com>
Co-authored-by: loki-gh-app[bot] <160051081+loki-gh-app[bot]@users.noreply.github.com>
Co-authored-by: loki-gh-app[bot] <160051081+loki-gh-app[bot]@users.noreply.github.com>
Signed-off-by: Lukas Juozas Janušaitis <lukas.janusaitis@hostinger.com>
Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…on (#12316)

Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
…12283)

Signed-off-by: Callum Styan <callumstyan@gmail.com>
@periklis periklis self-assigned this Mar 25, 2024
Copy link

openshift-ci bot commented Mar 25, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: periklis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 25, 2024
Copy link

openshift-ci bot commented Mar 25, 2024

@periklis: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@periklis periklis merged commit 4fdc9b5 into openshift:main Mar 25, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet