Add trace span for time spent lazily loading index-header in store-gateway

[charleskorn]

What this PR does

This PR adds further information to trace spans emitted by store-gateways while lazily loading an index-header.

I'd suggest reviewing the commits separately - the first introduces a context.Context argument to the methods on the indexheader.Reader interface, and the second adds the trace span.

Which issue(s) this PR fixes or relates to

(none)

Checklist

• [n/a] Tests updated.
• [n/a] Documentation added.
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
• [n/a] about-versioning.md updated with experimental features.

[56quarters]

LGTM. My only concern is the number of spans we're going to be adding to a typical request since we're already running into trace size limits in some cases. I suppose this shouldn't be too bad in practice since we only add a span when doing lazy loading which is infrequent. WDYT?

[charleskorn]

I suppose this shouldn't be too bad in practice since we only add a span when doing lazy loading which is infrequent. WDYT?

That was my assumption too, so I'm not expecting this to be particularly impactful.

[dimitarvdimitrov]

before the store-gateway starts serving a request it creates an index header reader. This happens here

mimir/pkg/storegateway/bucket.go

Lines 1923 to 1924 in 0a7eb89

	// Call IndexVersion to lazy load the index header if it lazy-loaded.
	_, _ = b.indexHeaderReader.IndexVersion()

the idea was to trigger a method of the index header reader that will load it before starting to use it for general-purpose operations.

I suspect that this is the only place which may load an index header and we might not need all these extra context args.

[charleskorn]

Sorry, might have been a little too enthusiastic merging this before you had a chance to review it @dimitarvdimitrov. Would you like me to revert this and explore an alternative approach?

[dimitarvdimitrov]

I think that would keep the interfaces slightly simpler and still achieve the same. Sorry for not reviewing it on the day.

[charleskorn]

I've reverted this change in #7065 - would you be able to take a look @dimitarvdimitrov?