Skip to content

Commit

Permalink
Added a page for documenting beginner issues (#2461)
Browse files Browse the repository at this point in the history
* Added some documentation for beginner issues

Signed-off-by: Yash <yashrsharma44@gmail.com>

* Edited some lines

Signed-off-by: Yash <yashrsharma44@gmail.com>

* Update docs/operating/troubleshooting.md

Co-Authored-By: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Yash <yashrsharma44@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
  • Loading branch information
yashrsharma44 and bwplotka committed Apr 20, 2020
1 parent 392c843 commit 853175a
Showing 1 changed file with 36 additions and 0 deletions.
36 changes: 36 additions & 0 deletions docs/operating/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,39 @@ Checking producers log for such ULID, and checking meta.json (e.g if sample stat
- Who uploaded the block? Search for logs with this ULID across all sidecars/rulers. Check access logs to object storage. Check debug/metas or meta.json of problematic block to see how blocks looks like and what is the `source`.
- Determine what you misconfigured.
- If all looks sane and you double-checked everything: Then post an issue on Github, Bugs can happen but we heavily test against such problems.

# Sidecar

## Connection Refused

### Description

```shell
level=warn ts=2020-04-18T03:07:00.512902927Z caller=intrumentation.go:54 msg="changing probe status" status=not-ready reason="request flags against http://localhost:9090/api/v1/status/config: Get \"http://localhost:9090/api/v1/status/config\": dial tcp 127.0.0.1:9090: connect: connection refused"
```
* This issue might happen when thanos is not configured properly.

### Possible Solution

* Make sure that prometheus is running while thanos is started. The `connection_refused` states that there is no server running in the `localhost:9090`, which is the address for prometheus in this case.


## Thanos not identifying Prometheus

### Description

```shell
level=info ts=2020-04-18T03:16:32.158536285Z caller=grpc.go:137 service=gRPC/server component=sidecar msg="internal server shutdown" err="no external labels configured on Prometheus server, uniquely identifying external labels must be configured"
```
* This issue happens when thanos doesn't recognise prometheus

### Possible Solution

* Thanos requires **unique** `external_labels` for further processing. So make sure that the `external_labels` are not empty and globally unique in the prometheus config file. A possible example -

```yml
global:
external_labels:
cluster: eu1
replica: 0
```

0 comments on commit 853175a

Please sign in to comment.