fix: deferred loader panic when mutate and generate policies are applied

[vishal-chdhry]

Explanation

Previously mutate and generate rules were sharing the same policy context and were operating concurrently. This causes a race condition problem where if a checkpoint is created by either of the handlers too soon then a deferred loader creates a loader entry at the wrong level. This causes the deferred loader being deleted too soon and causes a panic in some scenario.

Related issue

Closes #9413

Milestone of this PR

Documentation (required for features)

My PR contains new or altered behavior to Kyverno.

• I have sent the draft PR to add or update the documentation and the link is:

What type of PR is this

Proposed Changes

Deep copy the policy context before passing it to generate and mutate handlers

Proof Manifests

Checklist

• I have read the contributing guidelines.
• I have read the PR documentation guide and followed the process including adding proof manifests to this PR.
• This is a bug fix and I have added unit tests that prove my fix is effective.
• This is a feature and I have added CLI tests that are applicable.
• My PR needs to be cherry picked to a specific release branch which is .
• My PR contains new or altered behavior to Kyverno and
  • CLI support should be added and my PR doesn't contain that functionality.

Further Comments

[codecov]

Codecov Report

Attention: Patch coverage is 64.47368% with 27 lines in your changes are missing coverage. Please review.

Project coverage is 33.65%. Comparing base (1a19540) to head (17ad807).

❗ Current head 17ad807 differs from pull request most recent head 08b588a. Consider uploading reports for the commit 08b588a to get more accurate results

Files	Patch %	Lines
pkg/engine/context/mock_deferred.go	66.66%	10 Missing and 4 partials ⚠️
pkg/webhooks/resource/updaterequest.go	55.00%	5 Missing and 4 partials ⚠️
pkg/webhooks/resource/handlers.go	69.23%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9935      +/-   ##
==========================================
+ Coverage   33.58%   33.65%   +0.06%     
==========================================
  Files         346      347       +1     
  Lines       23682    23740      +58     
==========================================
+ Hits         7954     7990      +36     
- Misses      14853    14868      +15     
- Partials      875      882       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[realshuting]

/cherry-pick release-1.12

[JimBugwadia]

I am thinking it may be best to pass in the context factory here for the mutate and generate rule invocations:

kyverno/pkg/webhooks/resource/handlers.go

Line 134 in 8369ab6

go h.handleBackgroundApplies(ctx, logger, request.AdmissionRequest, policyContext, generatePolicies, mutatePolicies, startTime)

[realshuting]

👍