-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to fetch labels from Loki (Failed to call resource), please check the server logs for more details #6729
Comments
Just to add, I'm following the guide in https://grafana.com/docs/loki/latest/installation/simple-scalable-helm, deploying to a k3s cluster, getting the same error:
|
When I try to be in local runtime , there is error ,
my solution
|
Hi wubai7749, many thanks for your response. I will try it out |
I had this error, but in my scenario I had a Promtail, Grafana and Loki in virtual machines. There was a firewall block of grafana reaching loki on port 3100. |
I had this error , I deployed loki, promtail and grafana on windows 10, and encountered the same problem . these are config files. these files type actually yaml |
hi did u solve it?i also having the same issue,how did u solve it?thank u |
Also having same issue |
I had the same issue namely I was getting first
Promtailed stopped complaining but I still wasn't able to connect via grafana on
(When quoting these yaml snippets here, I am refering to loki helm chart I'm guessing im just exposing my stupidity here, but maybe this helps someone... |
Hi, |
for me the problem came from updating the Loki Helm chart from
|
For me this happens when I enable the query-scheduler. It seems to work initially then after awhile the labels error reappears and then only when I remove the scheduler again by setting it to false does it work. This happens even if I change the datasource URL to the scheduler as described. |
I ran into this issue when Loki is running in multi-tenant mode (same
Obviously this ties the datasource to a particular tenant, which might be a problem if you need some kind of multi-tenant dashboard. |
I have the same error using the latest docker container of Grafana + Loki + Promtail on a Raspberry Pi 4 8Gb. |
same error here. Unfortunately, the solutions provided are not working. |
If you are running loki and promtail from docker then promtail try to rich localhost:3100 but this address isn't loki but promtail itself , you can try to put the name_of_docker_container:3100 and in the docker-compose file use same network |
I also ran into this bug when following along with this tutorial. This tutorial uses the This is because when promtail configures its url it is rendering as
A work around for this bug is to create a
Changing the release name to be
and applying your manifest.
|
Same issue with Docker swarm. I've set :
And now it's working fine. More info here : https://community.grafana.com/t/loki-error-on-port-9095-error-contacting-scheduler/67263 |
My silly mistake was setting Loki port in Grafana to 9096 instead of 3100. Therefore i had those same symptoms: no any errors, but |
I think this should be closed since everything here is user error. |
Same issue, loki 2.7.4 |
Actually i just ran into the same issue here even using the current docker based evaluation-build. |
auth_enabled: false common: query_range: schema_config: filesystem: chunk_store_config: table_manager: ruler: |
这样就可以了 |
I had a similar problem with Grafana cloud Loki, mine was funny when I discovered the error I made; Putting this down so someone searching will crosscheck what they are doing and not suffer how I suffered lol. |
same here
grafana v9.5.5
add datasource in grafana
grafana logs
|
im fixing the problem by uninstall loki with helm, and install the 2.16.0 loki version
|
This really work for me with
|
Apart from all the other necessary fixes (auth_enabled:false and other new defaults), I also had to set the following when running without TLS, after updating my Loki version from helmchart 2.16.0 to 4.10.0.
Because the defaults of the new helmcharts defined gateway.loki.example.com with secret loki-gateway-tls. For some reason this "wrong" setup worked just fine on my testcluster, but started failing only on nonprod, probably having to do with the permissiveness of my ingress or proxy servers. |
This is what worked for me solving this error in my setup. Change loki url to k8s fully qualified DNS address
|
Describe the bug
Installed loki using helm(steps below) and tried to add in grafana loki datasource using URL
http://loki-gateway.loki.svc.cluster.local
and get an error in UIUnable to fetch labels from Loki (Failed to call resource), please check the server logs for more details
To Reproduce
Steps to reproduce the behavior:
0. Deployed loki & grafana in namespace
loki
helm template loki ./loki-simple-scalable-1.7.4.tgz --include-crds --namespace loki --set monitoring.selfMonitoring.enabled=false --set loki.image.registry=our-org-acr-registry.azurecr.io --set loki.image.repository=docker.io/grafana/loki --set gateway.image.registry=our-org-acr-registry.azurecr.io --set gateway.image.repository=docker.io/nginxinc/nginx-unprivileged | sed -E -b "s/(image: )"{0,1}(docker.io)(.*)/\1"our-org-acr-registry.azurecr.io/\2\3/g" | sed -E -b "s/(image: )(busybox)/\1our-org-acr-registry.azurecr.io/docker/\2/g" > loki-simple-scalable-our-org-no-selfmonitoring.yaml
helm template loki-grafana grafana-6.32.2.tgz --include-crds --set image.repository=our-org-acr-registry.azurecr.io/docker.io/grafana/grafana --set image.tag=9.0.2 --set testFramework.enabled=false > grafana2.yaml
k apply -f loki-simple-scalable-our-org-no-selfmonitoring.yaml
k apply -f grafana2.yaml
Unable to fetch labels from Loki (Failed to call resource), please check the server logs for more details
In the logs
Expected behavior
I expect Loki DataSource added as this was in 4 minutes video
Environment:
The text was updated successfully, but these errors were encountered: