failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided

[Dunge]

Describe the bug
This is a follow up to #6370. I believe there's something that needs to be fixed to work with different network configurations instead of forcing a workaround so that other users like me don't encounter the same issue on a simple base installation.

I have kube-prometheus-stack installed with metrics and grafana dashboards functional. I want to install loki to add functionality to see my pods logs. To be honest being new to this ecosystem I'm a bit confused about the dozens of different helm charts available and which is the proper one for my simple needs. I first tried with loki-stack (since it comes bundled with promtail) and then with loki-simple-scalable (based on this guide) and they both have the same issue.

#6370 offers a workaround by providing the pod IP as an environment variable and using it back in the loki config. Unfortunately I believe this workaround require us to download the chart locally and modifying it, and can't be just fixed providing a values.yaml. It also apply to the base loki chart (the "expert" configuration) and doesn't work as-is with the simple charts mentioned above.

To Reproduce
Steps to reproduce the behavior:

1. Have a kubernetes cluster running on microk8s
2. Change the CIDR from 10.1.x.x to 100.1.x.x (https://microk8s.io/docs/change-cidr) (not sure if that's what causing the issue or not).
3. Install loki via one of the above mentioned helm charts

Expected behavior
A clear and concise description of what you expected to happen.

Environment:

• Infrastructure: Kubernetes via microk8s 1.26.1 on Ubuntu
• Deployment tool: helm

Screenshots, Promtail config, or terminal output

level=info ts=2023-02-24T07:52:23.973846262Z caller=main.go:103 msg="Starting Loki" version="(version=2.6.1, branch=HEAD, revision=6bd05c9a4)"
level=info ts=2023-02-24T07:52:23.974285735Z caller=server.go:288 http=[::]:3100 grpc=[::]:9095 msg="server listening on addresses"
level=info ts=2023-02-24T07:52:23.975431507Z caller=modules.go:736 msg="RulerStorage is not configured in single binary mode and will not be started."
level=warn ts=2023-02-24T07:52:23.978586546Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache - chunksfifocache"
level=info ts=2023-02-24T07:52:23.978956446Z caller=table_manager.go:252 msg="query readiness setup completed" duration=1.473µs distinct_users_len=0
level=info ts=2023-02-24T07:52:23.978983685Z caller=shipper.go:124 msg="starting index shipper in RW mode"
level=info ts=2023-02-24T07:52:23.979051282Z caller=table_manager.go:134 msg="uploading tables"
level=info ts=2023-02-24T07:52:23.98001867Z caller=shipper_index_client.go:79 msg="starting boltdb shipper in RW mode"
level=info ts=2023-02-24T07:52:23.98111501Z caller=modules.go:761 msg="RulerStorage is nil.  Not starting the ruler."
level=info ts=2023-02-24T07:52:23.981718066Z caller=table_manager.go:167 msg="handing over indexes to shipper"
level=info ts=2023-02-24T07:52:23.985863701Z caller=worker.go:112 msg="Starting querier worker using query-scheduler and scheduler ring for addresses"
level=info ts=2023-02-24T07:52:23.989253896Z caller=module_service.go:82 msg=initialising module=server
level=info ts=2023-02-24T07:52:23.989432381Z caller=module_service.go:82 msg=initialising module=query-frontend-tripperware
level=info ts=2023-02-24T07:52:23.989453249Z caller=module_service.go:82 msg=initialising module=memberlist-kv
level=error ts=2023-02-24T07:52:23.989585575Z caller=loki.go:386 msg="module failed" module=memberlist-kv error="invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.989727317Z caller=loki.go:386 msg="module failed" module=querier error="failed to start querier, because it depends on module ring, which has failed: invalid service state: Failed, expected: Running, failure: failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.989900367Z caller=loki.go:386 msg="module failed" module=ingester error="failed to start ingester, because it depends on module store, which has failed: invalid service state: Failed, expected: Running, failure: failed to start store, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.98999053Z caller=loki.go:386 msg="module failed" module=usage-report error="failed to start usage-report, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990085587Z caller=loki.go:386 msg="module failed" module=ring error="failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990181431Z caller=loki.go:386 msg="module failed" module=store error="failed to start store, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990344372Z caller=loki.go:386 msg="module failed" module=ingester-querier error="failed to start ingester-querier, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990503905Z caller=loki.go:386 msg="module failed" module=compactor error="failed to start compactor, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990603705Z caller=loki.go:386 msg="module failed" module=distributor error="failed to start distributor, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.990709554Z caller=loki.go:386 msg="module failed" module=query-frontend error="failed to start query-frontend, because it depends on module usage-report, which has failed: invalid service state: Failed, expected: Running, failure: failed to start usage-report, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=error ts=2023-02-24T07:52:23.99082232Z caller=loki.go:386 msg="module failed" module=query-scheduler error="failed to start query-scheduler, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0008cf9a0 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc00009aa50 loki false {    false}} 0xc00009aa50 loki [{ringDesc 0xe43cc0} {}]} 0xc00091a1e0 0xc00009aa50 0xc0001cf650 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00013ca20 0xc000164840 0xc000164900 0xc0001649c0 0xc000164a80 0xc000164cc0 0xc000164b40 0xc000164d80 0xc000164c00 0xc000488040 0xc000488080 0xc000164f00 0xc000164fc0 0xc000165140 0xc000165080 0xc000240588 0xc0001e0bd0 0xc000240578 0xc000240580 0xc0004880c0 0xc000488100 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
level=info ts=2023-02-24T07:52:23.989824Z caller=module_service.go:114 msg="module stopped" module=query-frontend-tripperware
level=info ts=2023-02-24T07:52:23.992347249Z caller=modules.go:1090 msg="server stopped"
level=info ts=2023-02-24T07:52:23.992375764Z caller=module_service.go:114 msg="module stopped" module=server
level=info ts=2023-02-24T07:52:23.992390572Z caller=loki.go:375 msg="Loki stopped"
failed services
github.com/grafana/loki/pkg/loki.(*Loki).Run
 /src/loki/pkg/loki/loki.go:421
main.main
 /src/loki/cmd/loki/main.go:105
runtime.main
 /usr/local/go/src/runtime/proc.go:255
runtime.goexit
 /usr/local/go/src/runtime/asm_amd64.s:1581
level=error ts=2023-02-24T07:52:23.992474923Z caller=log.go:103 msg="error running loki" err="failed services\ngithub.com/grafana/loki/pkg/loki.(*Loki).Run\n\t/src/loki/pkg/loki/loki.go:421\nmain.main\n\t/src/loki/cmd/loki/main.go:105\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581"

[m1ll3nn1umf4lc0n]

I have the same issue i think. When trying to install the loki-stack on GKE.

loki
level=info ts=2023-03-09T20:54:56.000971891Z caller=main.go:103 msg="Starting Loki" version="(version=2.6.1, branch=HEAD, revision=6bd05c9a4)"
loki
level=info ts=2023-03-09T20:54:56.001408032Z caller=modules.go:736 msg="RulerStorage is not configured in single binary mode and will not be started."
loki
level=info ts=2023-03-09T20:54:56.002035759Z caller=server.go:288 http=[::]:3100 grpc=[::]:9095 msg="server listening on addresses"
loki
level=warn ts=2023-03-09T20:54:56.004006493Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache - chunksfifocache"
loki
level=info ts=2023-03-09T20:54:56.006249917Z caller=table_manager.go:252 msg="query readiness setup completed" duration=3.511µs distinct_users_len=0
loki
level=info ts=2023-03-09T20:54:56.006502494Z caller=shipper.go:124 msg="starting index shipper in RW mode"
loki
level=info ts=2023-03-09T20:54:56.007201108Z caller=shipper_index_client.go:79 msg="starting boltdb shipper in RW mode"
loki
level=info ts=2023-03-09T20:54:56.00997078Z caller=table_manager.go:134 msg="uploading tables"
loki
level=info ts=2023-03-09T20:54:56.009981736Z caller=table_manager.go:167 msg="handing over indexes to shipper"
loki
level=info ts=2023-03-09T20:54:56.014723084Z caller=modules.go:761 msg="RulerStorage is nil. Not starting the ruler."
loki
level=info ts=2023-03-09T20:54:56.015291818Z caller=worker.go:112 msg="Starting querier worker using query-scheduler and scheduler ring for addresses"
loki
level=info ts=2023-03-09T20:54:56.017825137Z caller=module_service.go:82 msg=initialising module=server
loki
level=info ts=2023-03-09T20:54:56.020654554Z caller=module_service.go:82 msg=initialising module=memberlist-kv
loki
level=info ts=2023-03-09T20:54:56.020976749Z caller=module_service.go:82 msg=initialising module=query-frontend-tripperware
loki
level=error ts=2023-03-09T20:54:56.021647459Z caller=loki.go:386 msg="module failed" module=memberlist-kv error="invalid service state: Failed, expected: Running, failure: service &{0xc000707540 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true 7946 false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 5000000000 0 {[] 7946 5000000000 5000000000 false 0xc000100a50 loki false { false}} 0xc000100a50 loki [{ringDesc 0xe43cc0} {}]} 0xc00042d950 0xc000100a50 0xc0004c7e30 {{} [0 1 0]} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xe43cc0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc00005af00 0xc0008b79e0 0xc0008b7aa0 0xc0008b7b60 0xc0008b7c20 0xc0008b7e60 0xc0008b7ce0 0xc0008b7f20 0xc0008b7da0 0xc00089ae00 0xc00089ae40 0xc0008c60c0 0xc0008c6180 0xc0008c6300 0xc0008c6240 0xc00000f780 0xc000121180 0xc00000f760 0xc00000f770 0xc00089aec0 0xc00089af00 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"

[seanmorton]

Same here on EKS:

level=error ts=2023-04-07T01:59:45.695757424Z caller=loki.go:511 msg="module failed" module=memberlist-kv error="invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
18
level=error ts=2023-04-07T01:59:45.69581295Z caller=loki.go:511 msg="module failed" module=ingester error="failed to start ingester, because it depends on module runtime-config, which has failed: context canceled"
17
level=error ts=2023-04-07T01:59:45.695822223Z caller=loki.go:511 msg="module failed" module=query-scheduler error="failed to start query-scheduler, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
16
level=error ts=2023-04-07T01:59:45.695838142Z caller=loki.go:511 msg="module failed" module=query-frontend error="failed to start query-frontend, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
15
level=error ts=2023-04-07T01:59:45.695848147Z caller=loki.go:511 msg="module failed" module=distributor error="failed to start distributor, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
14
level=error ts=2023-04-07T01:59:45.695857812Z caller=loki.go:511 msg="module failed" module=ingester-querier error="failed to start ingester-querier, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
13
level=error ts=2023-04-07T01:59:45.695869892Z caller=loki.go:511 msg="module failed" module=usage-report error="failed to start usage-report, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
12
level=error ts=2023-04-07T01:59:45.69588719Z caller=loki.go:511 msg="module failed" module=store error="failed to start store, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
11
level=error ts=2023-04-07T01:59:45.695942469Z caller=loki.go:511 msg="module failed" module=ring error="failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: invalid service state: Failed, expected: Running, failure: service &{0xc0001e5040 { true 10000000000 4 30000000000 200000000 3 30000000000 0 true  7946  false [loki-memberlist] 1000000000 60000000000 10 false 0 300000000000 20000000000 0 {[] 7946 2000000000 5000000000 false 0x3c877c0 loki false {    false  }} 0x3c877c0 loki [{ringDesc 0xedbaa0} {}]} 0xc0003c6190 0x3c877c0 0xc0007688c0 {[] {[] 0}} <nil> <nil> {0 0} map[] map[ringDesc:{ringDesc 0xedbaa0} usagestats.jsonCodec:{}] {0 0} map[] map[] {0 0} [] 0 [] 0 0 {0 0} map[] 0xc000826240 0xc00081dec0 0xc00081df80 0xc000854060 0xc000854120 0xc000854360 0xc0008541e0 0xc000854420 0xc0008542a0 0xc000859840 0xc000859880 0xc0008545a0 0xc000854660 0xc0008547e0 0xc000854720 0xc0001ba558 0xc0001b7ab0 0xc0001ba4e8 0xc0001ba4f0 0xc0008598c0 0xc000859900 10} failed: failed to create memberlist: Failed to get final advertise address: no private IP address found, and explicit IP not provided"
10
level=error ts=2023-04-07T01:59:45.695953598Z caller=loki.go:511 msg="module failed" module=query-frontend-tripperware error="failed to start query-frontend-tripperware, because it depends on module runtime-config, which has failed: context canceled"
9
level=error ts=2023-04-07T01:59:45.695964138Z caller=loki.go:511 msg="module failed" module=ruler error="failed to start ruler, because it depends on module ring, which has failed: context canceled"
8
level=error ts=2023-04-07T01:59:45.695970967Z caller=loki.go:511 msg="module failed" module=runtime-config error="context canceled"
7
level=error ts=2023-04-07T01:59:45.695991971Z caller=loki.go:511 msg="module failed" module=querier error="failed to start querier, because it depends on module ring, which has failed: context canceled"
6
level=error ts=2023-04-07T01:59:45.696007516Z caller=loki.go:511 msg="module failed" module=compactor error="failed to start compactor, because it depends on module usage-report, which has failed: context canceled"
5
level=info ts=2023-04-07T01:59:45.696075754Z caller=module_service.go:114 msg="module stopped" module=cache-generation-loader