Streamline gardener-controller-manager controllers after refactorings

[rfranzke]

How to categorize this PR?

/area dev-productivity
/kind enhancement

What this PR does / why we need it:
This PR streamlines a few nits after the many refactors of the controllers part of gardener-controller-manager.

Which issue(s) this PR fixes:
Part of #4251

Release note:

NONE

[rfranzke]

/cc @ary1992

[ary1992]

/assign

[ary1992]

Thank you for the PR!
Just a minor comment. Apart from that lgtm.

[gardener-prow]

@rfranzke: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gardener-check-vulnerabilities	0cbe2be	link	false	/test pull-gardener-check-vulnerabilities

Full PR test history. Your PR dashboard. Command help for this repository.
Please help us cut down on flakes by linking this test failure to an open flake report or filing a new flake report if you can't find an existing one. Also see our testing guideline for how to avoid and hunt flakes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[plkokanov]

/lgtm

[ary1992]

/approve

[gardener-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ary1992

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ary1992]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment