fix: add check for existence of HTTPRoute in KongUpstreamPolicy controller

[randmonkey]

What this PR does / why we need it:

Add check of existence of HTTPRoutes in KongUpstreamPolicy controller and only enable the ancestor status update of related KongUpstreamPolicy when HTTPRoute CRD exists before KIC starts.
Removes the dependency of HTTPRoute of KongUpstreamPolicy.

Which issue this PR fixes:

fixes #5729

Special notes for your reviewer:

We may need to backport it to 3.1.x.

PR Readiness Checklist:

Complete these before marking the PR as ready to review:

• the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

[codecov]

Codecov Report

Attention: Patch coverage is 86.20690% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 73.9%. Comparing base (d82716c) to head (d9c4984).
Report is 7 commits behind head on main.

Files	Patch %	Lines
...ers/configuration/kongupstreampolicy_controller.go	71.4%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #5780     +/-   ##
=======================================
- Coverage   73.9%   73.9%   -0.1%     
=======================================
  Files        176     176             
  Lines      18187   18185      -2     
=======================================
- Hits       13456   13452      -4     
- Misses      3734    3736      +2     
  Partials     997     997             

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[czeslavo]

I think we should add an envtest in which we ensure that KongUpstreamPolicies used with Services and Ingress work as expected when Gateway API CRDs are not installed. I'm afraid this could still be a problem because of this HTTPRoute list in the original reconciler.

[randmonkey]

I think we should add an envtest in which we ensure that KongUpstreamPolicies used with Services and Ingress work as expected when Gateway API CRDs are not installed. I'm afraid this could still be a problem because of this HTTPRoute list in the original reconciler.

Removed the extra dynamic controller and added a CRDExists check for HTTPRoute. When HTTPRoute exists, KongUpstreamPolicy controller enables features requiring HTTPRoute. Also added test cases in envtest to verify that KongUpstreamPolicy works both with and without HTTPRoute.

[team-k8s-bot]

The backport to release/3.1.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-release/3.1.x release/3.1.x
# Navigate to the new working tree
cd .worktrees/backport-release/3.1.x
# Create a new branch
git switch --create backport-5780-to-release/3.1.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 30a2991056044636804da46c7847b44e9497de50
# Push it to GitHub
git push --set-upstream origin backport-5780-to-release/3.1.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-release/3.1.x

Then, create a pull request where the base branch is release/3.1.x and the compare/head branch is backport-5780-to-release/3.1.x.