implement SchedulerQueueingHints feature gate

[sanposhiho]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Which issue(s) this PR fixes:

Ref: #118438 (comment)

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Added a new feature gate, `SchedulerQueueingHints` (enabled by default).
The new feature gate activates a framework for fine-grained filtering of events related to scheduler plugins.
In this release, no default scheduling plugins make use of the hinting framework, so you should not expect any behavior changes.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[sanposhiho]

/triage accepted
/cc @alculquicondor @Huang-Wei @ahg-g
/sig scheduling

[alculquicondor]

/approve
/lgtm
/assign @ahg-g

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 90c8e7f74986e83b5e8e2b0f73a55da5234d2c91

[ahg-g]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, alculquicondor, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/features/OWNERS [ahg-g]
• pkg/scheduler/OWNERS [ahg-g,alculquicondor,sanposhiho]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tengqm]

Is there a docs proposal to k8s.io/website?

The text in the commit message, especially the "user-facing" change is not easy to parse.
The question from a user's perspective is ... Why this new gate? Do I have to know which
functions are called by the scheduler? What will happen to my workload if I enable/disable
this gate?

[alculquicondor]

It's more of a release note to developers of scheduler plugins.

Maybe we should just remove the release note here, as the public for them is end users?

[alculquicondor]

OTOH, the feature gate is user facing. Let me try:

/release-note-edit

The feature gate SchedulerQueueingHints (enabled by default) provides a framework for fine-grained filtering of events for scheduler plugins. In this release, no default scheduling plugins make use of the hinting framework, so you should not expect any behavior changes.

[sanposhiho]

@alculquicondor Thanks for handling 🙏

[sftim]

Changelog suggestion

Added a new feature gate, `SchedulerQueueingHints` (enabled by default)
The new feature gate activates a framework for fine-grained filtering of events that related to
scheduler plugins.
In this release, no default scheduling plugins make use of the hinting framework, so you should not
expect any behavior changes.

[alculquicondor]

/release-note-edit

Added a new feature gate, `SchedulerQueueingHints` (enabled by default).
The new feature gate activates a framework for fine-grained filtering of events related to scheduler plugins.
In this release, no default scheduling plugins make use of the hinting framework, so you should not expect any behavior changes.