feat(flowcontrol): Add Foundational Types and Architecture

[LukeAVanDrie]

Description

This PR introduces the foundational building blocks for the new Flow Control system, designed to manage priority, fairness, and queuing for inference workloads.

The goal is to provide a sophisticated mechanism for managing diverse SLOs and preventing issues like Head-of-Line blocking and system overload, which are not addressed by simple FCFS temportal scheduling. This is a relatively sophisticated and complex system that will need to be split across several PRs. This initial submission lays the groundwork for the entire module by establishing its core concepts and data contracts (basically all types with no cross-package dependencies).

This work tracks issue #674

For historical context, the initial KEP-like proposal can be found here. Please note that the design has evolved significantly, and this PR represents the most current architecture (or at least a small slice of it). I will do some documentation cleanup once more of these PRs with the canonical architectural decisions are out for review.

Contribution

This PR includes two main pieces:

1. Top-Level README.md: Provides a high-level overview of the Flow Controller, including:

   • A clear Motivation for why this component is necessary.
   • A high-level Architectural Diagram showing the component interactions and request flow.
   • A summary of the core Architectural Pillars (Controller, Framework, Registry, etc.).

2. The types Package: This new package establishes the core "vocabulary" for the entire Flow Controller module. It includes:

   • Request Lifecycle Interfaces (FlowControlRequest, QueueItemAccessor, QueueItemHandle).
   • Final Outcome Reporting via the QueueOutcome enum and a structured set of error types (ErrRejected, ErrEvicted).
   • The FlowSpecification interface for defining workload identity and priority.
   • A README.md detailing the concepts within the types package.

Review Focus

As this is a foundational PR, I'm particularly looking for feedback on:

1. Does the high-level architecture and motivation make sense?
2. Are the data models in the types package clear, logical, and sufficient for the tasks ahead?
3. Is the vocabulary (e.g., QueueOutcome, the error hierarchy) well-defined and intuitive?

Future PRs will build upon these types to implement the framework, registry, and controller packages.

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	25a1e08
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6851efe911955d0008622782
😎 Deploy Preview	https://deploy-preview-997--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: LukeAVanDrie
Once this PR has been reviewed and has the lgtm label, please assign kfswain for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @LukeAVanDrie. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[LukeAVanDrie]

I have the followup PRs for the rest of the type system that contextualize this queued up, but I am currently on a 🚂 and my connection is spotty. Will send those out soon.

The next PR will detail the framework package (all the extension points) which should be a greater source of discussion compared to this PR. I will then have followups for the ports (saturation detector and flow registry) and the core execution engine (flow controller). These will have much more detailed architecture diagrams as well. The top-level README is intended to have the high level view only.

[danehans]

/ok-to-test