Improve metric capture on error

[kfswain]

Our metrics currently capture errors and associate them with an inferenceModel and targetModel:

gateway-api-inference-extension/pkg/epp/metrics/metrics.go

Line 170 in 189f0dc

func RecordRequestErrCounter(modelName, targetModelName string, code string) {

We should also include the endpoint that the error is affiliated with to better catch faulty pods/nodes.

Additionally errors such as ERR_CONNECTION_REFUSED don't have an explicit error code. The deferred error capture here:

gateway-api-inference-extension/pkg/epp/handlers/streamingserver.go

Line 65 in 189f0dc

defer func(error) {

likely covers this. But would be good to validate manually & with test cases. (I didn't see a test case from a brief search of the repo)

[JeffLuoo]

Do you have any examples of triggering errors without an explicit error code like ERR_CONNECTION_REFUSED ?

[kfswain]

I don't quite remember when I hit that, I think it was load testing.

We can scope this issue to just capturing the endpoint on error