Compactor: add per tenant compaction delete enabled flag

[MichelHollands]

Signed-off-by: Michel Hollands michel.hollands@grafana.com

What this PR does / why we need it:

This PR adds a per tenant compaction delete enabled flag called compactor_deletion_enabled. A unit test was added for the http middleware that checks access.

This was also manually tested. The following snippet was added in the Loki config file:

limits_config:
  compactor_deletion_enabled: false

The runtime config contained the following:

overrides:
  tenant1:
    compactor_deletion_enabled: true
  tenant2:
    compactor_deletion_enabled: false

Running curl commands against the API gives the expected results:

% curl  http://localhost:3100/loki/api/v1/delete
no org id
% curl -H "X-Scope-OrgID: tenant1"  http://localhost:3100/loki/api/v1/delete
[]
% curl -H "X-Scope-OrgID: tenant2"  http://localhost:3100/loki/api/v1/delete
access denied
% curl -H "X-Scope-OrgID: tenant3"  http://localhost:3100/loki/api/v1/delete
[]

Checklist

• Documentation added
• Tests updated
• Is this an important fix or new feature? Add an entry in the CHANGELOG.md.
• Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0.3%
+            querier	0%
- querier/queryrange	-0.1%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[MasslessParticle]

Would it be cleaner to just make a middleware to wrap our handlers?

If we took that approach, we wouldn't have to implement this same check in every handler.

[MichelHollands]

Would it be cleaner to just make a middleware to wrap our handlers?

If we took that approach, we wouldn't have to implement this same check in every handler.

Yes, it's been added. It even has unit tests :)

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0.3%
+            querier	0%
+ querier/queryrange	0.1%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[MasslessParticle]

This looks good! Were we going remove WholeStreamDelete in this PR or another one?

[MichelHollands]

This looks good! Were we going remove WholeStreamDelete in this PR or another one?

Due to the size of this one I was going to do it in a separate PR.

[dannykopping]

Added two usability nits, otherwise looks good

[dannykopping]

LGTM, thanks @MichelHollands 👍
The docs changes here are fairly minor @KMiller-Grafana so I'm going to merge, I hope you don't mind?

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
-        distributor	-0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
- querier/queryrange	-0.1%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

The backport to k102 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-6410-to-k102 origin/k102
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x b4e6c599368278b8da20852a9acc4722f70af603
# Push it to GitHub
git push --set-upstream origin backport-6410-to-k102
git switch main
# Remove the local backport branch
git branch -D backport-6410-to-k102

Then, create a pull request where the base branch is k102 and the compare/head branch is backport-6410-to-k102.