Skip to content

Commit

Permalink
add troubleshooting reports section (#861)
Browse files Browse the repository at this point in the history
* add troubleshooting reports section

Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>

* fixes

Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>

---------

Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
Co-authored-by: Chip Zoller <chipzoller@gmail.com>
  • Loading branch information
eddycharly and chipzoller committed May 24, 2023
1 parent 5cb02c4 commit 4ac99b8
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions content/en/docs/Troubleshooting/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,12 @@ You can also follow the steps on the [Kyverno wiki](https://github.com/kyverno/k
**Solution**: There can be many reasons why a policy may fail to work as intended, assuming other policies work. One of the most common reasons is that the API server is sending different contents than what you have accounted for in your policy. To see the full contents of the AdmissionReview request the Kubernetes API server sends to Kyverno, add the `dumpPayload` [container flag](/docs/installation/customization/#container-flags) set to `true` and check the logs. This has performance impact so it should be removed or set back to `false` when complete.
The second most common reason policies may fail to operate per design is due to variables. To see the values Kyverno is substituting for variables, increase logging to level `4` by setting the container flag `-v=4`. You can `grep` for the string `variable` (or use tools such as [stern](https://github.com/stern/stern)) and only see the values being substituted for those variables.
## Admission reports are stacking up
**Symptom**: Admission reports keep accumulating in the cluster, taking more and more etcd space and slowing down requests.
**Diagnose**: Please follow the [troubleshooting docs](https://github.com/kyverno/kyverno/blob/main/docs/dev/troubleshooting/reports.md) to determine if you are affected by this issue.
**Solution**: Admission reports can accumulate if the reports controller is not working properly so the first thing to check is if the reports controller is running and does not continuously restarts. If the controller works as expected, another potential cause is that it fails to aggregate admission reports fast enough. This usually happens when the controller is throttled. You can fix this by increasing QPS and burst rates for the controller by setting `--clientRateLimitQPS=500` and `--clientRateLimitBurst=500`.
Note that starting with Kyverno 1.10, two cron jobs are responsible for deleting admission reports automatically if they accumulate over a certain threshold.

0 comments on commit 4ac99b8

Please sign in to comment.