feat: make model metrics endpoints configurable

[nayihz]

fix: #16

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	96bee04
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68613eaa31c5bd0008942443
😎 Deploy Preview	https://deploy-preview-1000--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[nirrozenbaum]

I have some doubts about adding additional fields to InferencePool.
there are alternative ways to achieve the same goal, like using command line args.
changes to CRDs should be discussed and get broad agreement using proposals.

cc @kfswain @ahg-g @robscott @danehans @elevran

/hold for others to comment.

[nayihz]

@nirrozenbaum , thanks for you advise. I think you are right.
Command line args or environment variable, which way do you prefer?

[nirrozenbaum]

@nirrozenbaum , thanks for you advise. I think you are right. Command line args or environment variable, which way do you prefer?

@nayihz I would start with command-line args with default values (the existing ones).
we can always iterate if needed.

[elevran]

@nayihz @nirrozenbaum this introduces a fixed endpoint for all model servers in the pool.
Would it make sense to use the Prometheus-formatted annotations as the source of truth when present and fallback to the configuration when missing?
For example:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/path: "/metrics"
    prometheus.io/port: "8080"

[nirrozenbaum]

@nayihz @nirrozenbaum this introduces a fixed endpoint for all model servers in the pool. Would it make sense to use the Prometheus-formatted annotations as the source of truth when present and fallback to the configuration when missing? For example:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/path: "/metrics"
    prometheus.io/port: "8080”

@elevran we already have a fixed endpoint, so this PR is not introducing it :). the intention was to make that endpoint configurable.
but your suggestion makes sense as an improvement.

[nirrozenbaum]

/unhold

[kfswain]

looks good for the most part, added a comment about switching to env vars to have better interfacing with our helm chart

[nirrozenbaum]

/lgtm

leaving final stamp to the other reviewers.

[nayihz]

Kindly ping @ahg-g @kfswain @danehans , any chance you could have another look at this PR.

[kfswain]

looks great. Thanks!
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kfswain, nayihz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment