Docs: Create EPP Operations Guide

[danehans]

Questions such as "How many InferencePools does an EPP support?" have been asked several times recently. A user guide should be added that provides steps for performing common EPP operations tasks.

[kfswain]

This issue's context is currently scoped to those familiar with GIE, and I think it will need to stay that way given the subject matter.

However, if someone feels confident in picking this issue up, but requires more context, feel free to ping anyone here: https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/OWNERS_ALIASES

[danehans]

cc: @smarterclayton @kfswain @ahg-g @nirrozenbaum, please add comments for EPP operations tasks that should be documented. Here are a few that come to mind:

1. Upgrades/Rollbacks
2. Troubleshooting
3. Scaling EPP or provide auto scaling details (when supported).
4. Monitoring and observability- We already have an EPP metrics doc and Grafana dashboards but users still need to manually stitch things together and fill-in some gaps.
5. Securing EPP- Currently EPP uses a self-signed cert (xref). Should cert manager be used for this doc?
6. EPP Perf Optimization- How can a user optimize EPP for perf based on common use cases.

Should the guide cover inference platform and inference workload tasks?

[nirrozenbaum]

• Building EPP with pluggable scheduler logic, e.g., configure scheduler plugins during build time (support added in Move filter and scorer plugins registration to a separate file #729, usage added in Weighted scorers #737).
• How to configure EPP collected metrics (when supported) as documented in extensible data layer: EPP should allow configurable metrics collection #703.
• not exactly EPP, but also good candidate for something to document - how to use BBR together with EPP

other than operations tasks, I think this document should explain the mental model of EPP and the why.

[danehans]

I think this document should explain the mental model of EPP and the why.

IMHO, this should be in the concepts section of the docs.