[helm] Single binary object storage

[trevorwhitney]

What this PR does / why we need it:

The PR allows an operator to install Loki in Single Binary mode in an HA fashion, meaning more than 1 Single Binary replica. In order for this to work, it is not possible to specify an object storage backend when running in Single Binary mode. The gateway is also now always enabled, for both Single Binary and Scalable modes.

Which issue(s) this PR fixes:
Fixes #8269 #8185

Special notes for your reviewer:

This PR builds off #8261, will rebase after that is merged and diff will be smaller.

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• CHANGELOG.md updated
• Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

[grafanabot]

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[JStickler]

@trevorwhitney, you tagged me on this, but I don't see much in the way of documentation changes.

Not sure if we need to update this topic, which mentions using a shared object store? https://grafana.com/docs/loki/latest/fundamentals/architecture/deployment-modes/

[trevorwhitney]

@JStickler sorry the bot tagged you when I accidentally was removing changes to a docs file due to a botched rebase. No docs need on this one 😄

[grafanabot]

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[grafanabot]

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

[jeschkies]

Could you add some documentation on how this could be used?

[JStickler]

[Docs squad] One small suggestion for rewording to make it clearer that there are now two options here.

[sapslaj]

This is great @trevorwhitney! I just had that one thing about the replica counts come up in my testing, otherwise it looks great.

[sapslaj]

One (potentially negative?) side effect of this check is that you have to explicitly scale down the backend, read, and write replicas if you're using single binary with filesystem storage. If you don't, then this check will fail.

I wonder if we need to also and in something like (eq (int .Values.singleBinary.replicas) 0) in here, like we do when defining loki.deployment.isScalable

[trevorwhitney]

Good catch, added that extra check to the condition.