Skip to content

Commit

Permalink
Create debug.md tousers debug grafana-operator (grafana#600)
Browse files Browse the repository at this point in the history
Should be a document where the maintainers can help users to debug
the grafana-operator.

Co-authored-by: Hubert Stefanski <35736504+HubertStefanski@users.noreply.github.com>
  • Loading branch information
Edvin N and HVBE committed Nov 8, 2021
1 parent 07bc9bb commit a0fc281
Show file tree
Hide file tree
Showing 3 changed files with 109 additions and 1 deletion.
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ labels: bug, needs triage
assignees: ''

---
<!-- Before creating a issue please read though [documentation/debug.md](documentation/debug.md) -->

**Describe the bug**
A clear and concise description of what the bug is.
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ An operator to provision and manage Grafana Instances, Dashboards, Datasources a

## Companies and teams that trust and use the Grafana operator


| Company | Logo | Company | Logo
| :--- | :----: | :--- | :----: |
| [Red Hat](https://www.redhat.com)| <img src= "./media/users/redhat.png" width=100 height=100 > | [Integreatly](https://www.redhat.com/en/products/integration)|<img src= "./media/users/integreatly.png" width=100 height=100> |
Expand Down Expand Up @@ -124,3 +123,7 @@ Represents a Grafana datasource. See [the documentation](./documentation/notifie
### Using the Makefile

If you want to develop/build/test the operator, here are some instructions how to set up your dev-environment: [follow me](./documentation/develop.md)

## Debug

We have documented a few steps to help you debug the [grafana-operator](documentation/debug.md).
104 changes: 104 additions & 0 deletions documentation/debug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Debug grafana-operator

This document is created to help people debug the grafana-operator and it's Custom Resources (CR).

First of all read the documentation, they are not perfect but they will help you allong in most cases.

## Grafana instance

After you have setup your [grafana instance](deploy_grafana.md) it should at least generate a kubernetes deployment, but it can also generate more depending on how you have connfigured your grafana instance.

If that is not the case you need to debug it.

Start by checking the events and status field of the the grafana CR.

```shell
$ kubectl describe grafana example-grafana

Status:
Message: Ingress.extensions "grafana-ingress" is invalid: spec.rules[0].http.paths[0].pathType: Required value: pathType must be specified
Phase: failing
Previous Service Name: grafana-service
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProcessingError 9s (x4 over 27s) GrafanaDashboard Ingress.extensions "grafana-ingress" is invalid: spec.rules[0].http.paths[0].pathType: Required value: pathType must be specified
```

`kubectl get grafana example-grafana -o yaml` will also give you the status output.

For example in this case my grafana CR is missing the definition of `ingress.pathType`

```.yaml
spec:
ingress:
enabled: True
pathType: Prefix
```

In general if you are having issues setting up your grafana instance simply your grafana CR.
First try [deploy/examples/Grafana.yaml](deploy/examples/Grafana.yaml) and get that
grafana instance up and running.
When that is done add small config changes to start build the config you want.

## Grafana Dashboards

There are generally two reasons why you can't find your grafana dashboard.

1. You are creating a grafanadashboard in another namespace then the grafana-operator
but you haven't defined --scan-all in the [operator](deploy_grafana.md).
2. Or you haven't defined the correct labelSelector that matches the operator on your dashboard.

In your grafana CR you have deinfed somehing like:

```.yaml
dashboardLabelSelector:
- matchExpressions:
- { key: app, operator: In, values: [grafana] }
```

Just like in [deploy/examples/dashboards/SimpleDashboard.yaml](deploy/examples/dashboards/SimpleDashboard.yaml)
you need to define the correct labels that matches the grafana CR.

```.yaml
apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata:
name: simple-dashboard
labels:
app: grafana
spec:
json: >
{
```

Another good place to see if the dashboard have been applied to the grafana dashboard
is to check the kubernetes events.

``` shell
$ kubectl get events

2d16h Warning ProcessingError grafanadashboard/simple-dashboard Get "http://admin:***@grafana-service.grafana-operator-system.svc.cluster.local:3000/api/folders": dial tcp 10.96.171.19:3000: connect: connection refused
2d16h Normal Success grafanadashboard/simple-dashboard dashboard grafana-operator-system/simple-dashboard successfully submitted
```

For example the events that you can see above the grafana dashboard CR:s was most likley created before
the grafana CR was up and running and thus coulden't answer the grafana operator when it tried
to apply the dashboard.

## Operator logs

Check the logs from the operator and see what you can find.

Depending on how you have setup the operator you should be able to get the logs running:

```shell
kubectl logs -l control-plane=controller-manager -c manager
```

## Bugs

If you still are unable to deploy the grafana instance that you want and you think it's a bug please create an issue.

When creating the issue please add as much information as possible from this debug documentation.
It will make it easier for us to debug the issues faster.

0 comments on commit a0fc281

Please sign in to comment.