feat: add grpc tag to indicate request had been throttled

[adriantam]

Description

We need to convey request has been throttled so that caller can distinguish whether timeout is due to throttling.
Note that we cannot return timeout error in the distpatch throttler itself because the caller will not wait for error response in case of context timeout.

The caller (for example), the user of the OpenFGA library, may choose to use this grpcTag ("Throttled") to modify the error message returned due to request timeout. They may also update the returned header to indicate the request has been throttled.

References

Continuation of #1440

Review Checklist

• I have clicked on "allow edits by maintainers".
• I have added documentation for new/changed functionality in this PR or in a PR to openfga.dev [Provide a link to any relevant PRs in the references section above]
• The correct base branch is being used, if not main
• I have added tests to validate that the change in functionality is working as expected

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.06%. Comparing base (d12d58b) to head (28be398).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1531      +/-   ##
==========================================
+ Coverage   86.06%   86.06%   +0.01%     
==========================================
  Files          85       85              
  Lines        8038     8039       +1     
==========================================
+ Hits         6917     6918       +1     
  Misses        790      790              
  Partials      331      331              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[miparnisari]

We need to convey request has been throttled so that caller can distinguish whether timeout is due to throttling.

Can you ellaborate on this in the PR description? Who is the caller that will take action based on this tag? And what will that action be?

FWIW, today, any context tags are logged automatically. Also once you apply a tag to a context, any parent or children contexts also get that tag.

[adriantam]

We need to convey request has been throttled so that caller can distinguish whether timeout is due to throttling.

Can you ellaborate on this in the PR description? Who is the caller that will take action based on this tag? And what will that action be?

FWIW, today, any context tags are logged automatically. Also once you apply a tag to a context, any parent or children contexts also get that tag.

Done.

[miparnisari]

the user of the OpenFGA library, may choose to use this grpcTag ("Throttled") to modify the error message returned due to request timeout

This won't work if a person uses OpenFGA as a library and doesn't wrap it within a server. The grpc tags middleware must be running. I.e. this doesn't print anything:

    ctx := context.Background()

	grpc_ctxtags.Extract(ctx).Set("key-a", true)

	tags := grpc_ctxtags.Extract(ctx)
	for k, v := range tags.Values() {
		fmt.Println("Tag: %s, Value: %s", k, v)
	}