adds instrumentation to azure object client

[owen-d]

Eventually, I'd like to make a higher order utility for instrumented object clients, but the plumbing seemed a bit tedious. Instead, I've just added azure specific instrumentation here.

[ssncferreira]

LGTM 🙂
Just some small comments

[ssncferreira]

Suggested change

	// Latency seems to range from a few ms to a few secs and is
	// important. So use 6 buckets from 5ms to 5s.
	// Latency seems to range from a few ms to a few secs and is
	// important. So use 6 buckets from 5ms to 5s.

[kavirajk]

LGTM 👍

[cyriltovena]

Nit: we're trying to avoid global registration across the board. No need to remove it, just a comment.

I don't specifically like this instrument package.

[owen-d]

Yeah, I largely copied this from our s3 client :(

[cyriltovena]

LGTM.

I think it's hard to make this generics for all clients because you need to do it on the transport level, at which point you don't really know what is a delete/get/list and trying to find this with http information might be a pain.

We could though allow to inject and abstract those operationName via the context. Assuming the context is well propagated.

We could do this:

ctx := metrics.WithOperationName(ctx,"azure.GetObject")

rc, err = b.getObject(ctx, objectKey)
return err

Then the transport middleware could get that out, figure automatically the latency and the status code from the http response.

GCS uses this technique but map only Method to OperationName which could be enough if all those API are correctly resty ? But then you don't know across multiple cluster which cloud provider it is See https://github.com/grafana/loki/blob/main/pkg/storage/chunk/gcp/instrumentation.go#L83

type instrumentedTransport struct {
	observer prometheus.ObserverVec
	next     http.RoundTripper
}

func (i instrumentedTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	start := time.Now()
	resp, err := i.next.RoundTrip(req)
	if err == nil {
		i.observer.WithLabelValues(req.Method, strconv.Itoa(resp.StatusCode)).Observe(time.Since(start).Seconds())
	}
	return resp, err
}

I'm rooting for adding all the above using a new transport package in chunk package.

package transport

// NewTransport creates a new Transport for the given cloud provider name.
func NewTransport(providerName string) *http.Transport {}