Update production-ready Loki in docker-compose #6691

dannykopping · 2022-07-15T09:00:17Z

What this PR does / why we need it:
This PR updates the docker-compose setup with a few features:

Loki v2.6.0
SSD mode
minio for storage
log generator
configured as a local lab setup or a production-ready cluster

Which issue(s) this PR fixes:
N/A

Special notes for your reviewer:
I've been using this as a local lab setup to test & debug groupcache, and it's very useful.
Having a full distributed system running locally with memberlist was essential for troubleshooting some issues I was having.

I've made some potentially breaking changes, but I've documented them in the upgrade guide.

Checklist

Documentation added
Tests updated
Is this an important fix or new feature? Add an entry in the CHANGELOG.md.
Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

- S3-like storage - Interactive debugging by optionally exposing dlv ports - Improve performance - Use SSD mode (read/write) - GroupCache - Latest versions of loki, promtail, grafana - Scaling

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

grafanabot · 2022-07-15T09:54:24Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
-               loki	-0.6%

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

grafanabot · 2022-07-15T14:26:41Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

grafanabot · 2022-07-15T14:43:57Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

trevorwhitney

LGTM! Love the use of the common config and that diagram of the components. I think in the future we could add the jaeger all-in-one container, and add the necessary tracing integration to the loki datasource. Thanks Danny!

trevorwhitney · 2022-07-15T16:48:01Z

production/docker/README.md

-   ```bash
-   docker-compose pull
-   ```
+```mermaid


kavirajk · 2022-07-15T18:46:39Z

production/docker/README.md


-1. When adding data source in the grafana dashboard, using `http://loki-gateway:3100` for the URL field.
+    subgraph LokiRead["loki -target=read"]


Can we still run query-scheduler as different component?

Sure, why not?

kavirajk · 2022-07-15T19:03:08Z

production/docker/docker-compose.yaml

+  grafana:
+    image: grafana/grafana:8.5.9
+    ports:
+      - "3000:3000"


Can we make these ports customizable with $ENVs? People may be running their own services on these ports
(still can have default to 3000)

I'm not sure if that's possible, but they can just edit the docker-compose if that's the case

* Update production-ready Loki in docker-compose (#6691) * Improved docker-compose setup: - S3-like storage - Interactive debugging by optionally exposing dlv ports - Improve performance - Use SSD mode (read/write) - GroupCache - Latest versions of loki, promtail, grafana - Scaling * Adding diagram & updated README Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Upgrade guide Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding steps to build debug image Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add prometheus Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add datasources and disable Grafana auth Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add configurable groupcache capacity (#6678) * Add configurable groupcache capacity Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Improving the verbiage for clarity - thanks Travis! Changed the default size to 100MB Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding config docs Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Removed test - it was pretty redundant, plus groupcache panics if you try init its http server more than once Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * updated versions to the latest release v2.6.1 (#6703) * updated versions to the latest release v2.6.1 (#6705) * Add function to reset boltDBIndexClientWithShipper singleton In a downstream project I have a test case that creates multiple stores with different BoltDB shipper schema configurations. Since the client for the store is a singleton and is re-used, there needs to be a function to reset it as well. This function must only be used in tests. Signed-off-by: Christian Haudum <christian.haudum@gmail.com> * promtail: Inject tenant ID when receiving X-Scope-OrgID in heroku target (#6695) * inject tenant ID header * fix import order * Add push route (#6616) * Loki: Return an __error_details__ label for any line which incurs a __error__ while being processed (#6543) * adds an __error_details__ label which includes any details available about a logql error. Signed-off-by: Edward Welch <edward.welch@grafana.com> * fix lint, changed a test a little Signed-off-by: Edward Welch <edward.welch@grafana.com> * reset error details in Reset() function Signed-off-by: Edward Welch <edward.welch@grafana.com> * Loki: Do not store exact log line duplicates (now including unordered inserts) (#6642) * do not insert log lines which are an exact duplicate in both timestamp and log content Signed-off-by: Edward Welch <edward.welch@grafana.com> * update test to now ignore dupes pushed out of order Signed-off-by: Edward Welch <edward.welch@grafana.com> * make the test unordered to be more thorough Signed-off-by: Edward Welch <edward.welch@grafana.com> Co-authored-by: Danny Kopping <danny.kopping@grafana.com> Co-authored-by: Vladyslav Diachenko <82767850+vlad-diachenko@users.noreply.github.com> Co-authored-by: Christian Haudum <christian.haudum@gmail.com> Co-authored-by: Pablo <2617411+thepalbi@users.noreply.github.com> Co-authored-by: Salva Corts <salva.corts@grafana.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>

This was removed in grafana#6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

This was removed in #6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

* Improved docker-compose setup: - S3-like storage - Interactive debugging by optionally exposing dlv ports - Improve performance - Use SSD mode (read/write) - GroupCache - Latest versions of loki, promtail, grafana - Scaling * Adding diagram & updated README Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Upgrade guide Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Adding steps to build debug image Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add prometheus Signed-off-by: Danny Kopping <danny.kopping@grafana.com> * Add datasources and disable Grafana auth Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

This was removed in grafana#6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

pull-request-size bot added the size/XL label Jul 15, 2022

dannykopping changed the title ~~dannykopping/local env prod~~ Update production-ready Loki in docker-compose Jul 15, 2022

github-actions bot added the area/docs label Jul 15, 2022

Danny Kopping added 3 commits July 15, 2022 11:48

Improved docker-compose setup:

3e126f2

- S3-like storage - Interactive debugging by optionally exposing dlv ports - Improve performance - Use SSD mode (read/write) - GroupCache - Latest versions of loki, promtail, grafana - Scaling

Adding diagram & updated README

5acb336

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

Upgrade guide

6898f0d

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

dannykopping force-pushed the dannykopping/local-env-prod branch from 3fbe484 to 6898f0d Compare July 15, 2022 09:49

dannykopping marked this pull request as ready for review July 15, 2022 09:55

dannykopping requested review from KMiller-Grafana and a team as code owners July 15, 2022 09:55

Danny Kopping added 2 commits July 15, 2022 15:44

Adding steps to build debug image

3be5f1f

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

Add prometheus

05e38d8

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

Add datasources and disable Grafana auth

9c1efcd

Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

trevorwhitney approved these changes Jul 15, 2022

View reviewed changes

production/docker/README.md

```bash

docker-compose pull

```

```mermaid

Copy link

Collaborator

trevorwhitney Jul 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fancy!

kavirajk reviewed Jul 15, 2022

View reviewed changes

kavirajk reviewed Jul 18, 2022

View reviewed changes

dannykopping merged commit 305936a into grafana:main Jul 18, 2022

dannykopping deleted the dannykopping/local-env-prod branch July 18, 2022 07:47

dannykopping mentioned this pull request Jul 20, 2022

chore(make): reinclude .git dir in docker builds #6723

Merged

dannykopping pushed a commit to dannykopping/loki that referenced this pull request Jul 20, 2022

Reinclude .git dir in docker builds

cc72f7b

This was removed in grafana#6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

dannykopping pushed a commit that referenced this pull request Jul 20, 2022

Reinclude .git dir in docker builds (#6723)

b8d260e

This was removed in #6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

DANic-git mentioned this pull request Aug 18, 2022

Loki frontend returns duplicates #6929

Closed

osg-grafana added type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories and removed area/docs labels Oct 19, 2022

lxwzy pushed a commit to lxwzy/loki that referenced this pull request Nov 7, 2022

Reinclude .git dir in docker builds (grafana#6723)

76b25d3

This was removed in grafana#6691, but caused errors in docker builds Signed-off-by: Danny Kopping <danny.kopping@grafana.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update production-ready Loki in docker-compose #6691

Update production-ready Loki in docker-compose #6691

dannykopping commented Jul 15, 2022 •

edited

grafanabot commented Jul 15, 2022

grafanabot commented Jul 15, 2022

grafanabot commented Jul 15, 2022

trevorwhitney left a comment

trevorwhitney Jul 15, 2022

kavirajk Jul 15, 2022

dannykopping Jul 18, 2022

kavirajk Jul 15, 2022

dannykopping Jul 18, 2022


		1. When adding data source in the grafana dashboard, using `http://loki-gateway:3100` for the URL field.
		subgraph LokiRead["loki -target=read"]

Update production-ready Loki in docker-compose #6691

Update production-ready Loki in docker-compose #6691

Conversation

dannykopping commented Jul 15, 2022 • edited

grafanabot commented Jul 15, 2022

grafanabot commented Jul 15, 2022

grafanabot commented Jul 15, 2022

trevorwhitney left a comment

Choose a reason for hiding this comment

trevorwhitney Jul 15, 2022

Choose a reason for hiding this comment

kavirajk Jul 15, 2022

Choose a reason for hiding this comment

dannykopping Jul 18, 2022

Choose a reason for hiding this comment

kavirajk Jul 15, 2022

Choose a reason for hiding this comment

dannykopping Jul 18, 2022

Choose a reason for hiding this comment

dannykopping commented Jul 15, 2022 •

edited