Report resource errors in DB mode

[rainest]

What this PR does / why we need it:

Uses the GDR Syncer result channel to receive individual CRUD events and errors from sync operations. Log events and generate resource errors from sync errors, which are then output as Kubernetes Events.

Which issue this PR fixes:

Fix #3492

Special notes for your reviewer:

You can finally set leadership mode via argument again! I don't actually really want this, but it's the easiest and least hacky way to disable it for testing. For debugging integration I usually had it just off in a temporary code change, but for DB mode envtest it breaks always, so I went ahead and added the flag. The other option was to use some special environment variable if we wanted to make it more hidden.

---

GDR discards most type information internally and just tosses any objects around. We would need a large scale refactor to establish something like the controller-runtime client.Object interface to handle common features of Kong entities to change this, and I doubt that's happening any time soon. As such, this does horrible things with reflect that rely on reasonable assumptions about Kong entities generated by KIC, with some error handling in case that breaks horribly.

GDR errors lack the per-field errors that DB-less flattened errors provide. We can mostly work around this by just not including field information, though this does result in a cosmetic blemish in Events. We end up with an empty space where the field name should be. The field is still mentioned in the error string, so 🤷

  message: 'invalid : HTTP status 400 (message: "schema violation (methods: cannot
    set ''methods'' when ''protocols'' is ''grpc'' or ''grpcs'')")'

Rest of the event looks like event.yaml.txt

PR Readiness Checklist:

Complete these before marking the PR as ready to review:

• the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

[codecov]

Codecov Report

Attention: Patch coverage is 77.92208% with 17 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (main@c104c88). Click here to learn what that means.
Report is 10 commits behind head on main.

Files	Patch %	Lines
internal/dataplane/sendconfig/dbmode.go	79.1%	8 Missing and 6 partials ⚠️
internal/manager/setup.go	50.0%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##             main   #5785   +/-   ##
======================================
  Coverage        ?   72.0%           
======================================
  Files           ?     178           
  Lines           ?   18312           
  Branches        ?       0           
======================================
  Hits            ?   13188           
  Misses          ?    4132           
  Partials        ?     992           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rainest]

Testing error event generation is always kinda wonky because we don't really want to do it in integration (breaking config in integration is bad because it's poorly isolated) or E2E (expensive) and don't have an admin API otherwise. I was able to follow #4813 somewhat to produce a very basic approximation of DB mode in envtest for only one entity type, which IMO is good enough.

[pmalek]

... breaking config in integration is bad because it's poorly isolated ...

It is isolated in https://github.com/Kong/kubernetes-ingress-controller/tree/main/test/integration/isolated

[randmonkey]

Mostly LGTM, one simple comment reamaining.