Allow complete override of User-Agent Header on a per-RPC level from grpc client libraries

[asadali]

Please answer these questions before submitting your issue.

What version of gRPC are you using?

1.20.0

What did you expect to see?

tl;dr

This is a followup from a discussion that was initiated on gitter grpc/grpc channel.
Currently, the grpc/java library reuses the User-Agent from the channel for each RPC
and discards User-Agent by treating it as a reserved header.

However, User-Agent is not a reserved header and this creates complications
when trying to write a proxy-like gRPC service for HTTP endpoints that care
about User-Agent for response customization.

Posting it here to get more ideas about how to resolve this.

quoting the conversation below:

Asad @asadali Jun 04 13:10
What was the underlying reason for the restriction on not allowing User-Agent
to be overridden on a per-call basis? can't seem to find a spec which reserves
the User-Agent string for gRPC/HTTP2 and yet there is code in place in the
libraries (grpc-java/ grpc-go ..) to discard any user-supplied metadata
regarding User-Agent and always use the channel's value eg:
Utils.convertServerHeaders

Asad @asadali Jun 04 13:16
use-case:
client ---> httpSVC-A ---> grpcSVC-B ---> httpSVC-C
how can the client's user-agent be conveyed to httpSVC-C? if A-B have ONLY one
channel open between them with a channel-level User-Agent that can't be
overridden
@ejona86 ^ question regarding user-agent behavior

Eric Anderson @ejona86 Jun 04 14:34
@asadali, user-agent is a built-in feature as gRPC itself sends it. There is
an API to change what gRPC sends, but there didn't seem to be any need to allow
it to be changed per-RPC.

Asad @asadali Jun 04 14:43
@ejona86 we seem to have a use-case in which a per-RPC user-agent will make
things easier for us. The alternate is to use custom metadata fields to
preserve this information. that approach seems non-standard and we were hoping
to avoid it. Will it be possible to include a per-RPC user-agent in gRPC? i
will be happy to code it up. but based on what i read in past issues, this
request was repeatedly turned down.

Eric Anderson @ejona86 Jun 04 14:44
That is a cross-language decision. You would need to make clear what the
use-case for it is. Right now, it isn't clear what the use-case is.
Oh. I see now.
You want to communicate the origin client's user-agent to SVC-C
Yeah. That's not appropriate for user-agent.

Asad @asadali Jun 04 14:45
ack

Eric Anderson @ejona86 Jun 04 14:46
.... unless you are making something closer to a proxy. Maybe. It sort of
seems like a can of worms. It just makes a mess of things.
But I think I understand now.

Asad @asadali Jun 04 14:48
so the intermediate gateways aren't pure proxies but maybe more like
aggregators. in the non-GRPC world, the implementation made an assumption that
User-Agent is the originating client's user-agent. and all intermediate hops
honored that. I agree, that this is a very loose reading of the spec. I feel
the more logical method is to update the user-agent on each hop
however, systems built around that assumption aren't happy when they lose this
info :( IMO, gRPC clients can default to per-channel behavior but the choice
should ultimately be left to the user if they want to override it

Eric Anderson @ejona86 Jun 04 14:51
Well, today the application can't set the entire user-agent. gRPC will always
include itself in the user-agent. I'm trying to check what HTTP says to do for
user-agent and proxies.

Asad @asadali Jun 04 14:52
yeah i can use another opinion on this. and current gRPC behavior is what I am
trying to rationalize. does it need to always include its user-agent?

Eric Anderson @ejona86 Jun 04 17:31
@asadali, proxies do forward the user-agent. We do want to enable grpc proxies,
so that does mean we should forward the user-agent. Although on the server, any
compatibility quirks would generally be with the proxy, not the end-client. So
it still seems muddled, but it does seem we should consider it.

[ejona86]

I still need the float the idea with others. It'll require a gRFC if we choose to do it.

[asadali]

fyi, our current hacky workaround is to have a pair of client/server interceptors that use an interim header key to replace the user-agent metadata.
but will really like to see the library allow overrides so that the intercepting isn't needed at all.

[sudo-nan0-RaySK]

Any updated on this?
A similar issue with Content-Type header is posing a problem for us as we need to communicate with a Go gRPC application using JSON content sub-type. The way encoding works on go-grpc side is that it chooses one of the registered encoders based on content subtype from Content-Type header, so application/grpc+json will use the registered encoder with name json. However since Netty transport library is overriding it with application/grpc (considering it a reserved header), go-grpc is falling back on using the default encoder with content sub-type name of proto.

[ejona86]

@sudo-nan0-RaySK, Content-Type is unrelated to this issue, which is about User-Agent. Please open a new issue. See also #7321 where it has been discussed before.

[sudo-nan0-RaySK]

@ejona86 I am aware about both the issue at hand and prior discussion about supporting content sub-type. However the reason I mentioned it here was that ideally Content-Type very much like User-Agent should not be treated as a reserved header. Specially when gRPC's reference mention it's use in content sub-type based encoder selection. Maybe we could handle both of these together?

[ejona86]

@sudo-nan0-RaySK, open a separate issue. That can be discussed there, and it can reference this.