Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds: Add server features support to Bootstrapper #7271

Merged
merged 7 commits into from Jul 31, 2020

Conversation

dapengzhang0
Copy link
Member

@dapengzhang0 dapengzhang0 commented Jul 30, 2020

In preparation for xds-v3 support.

* after {@link #createChannel} is called.
*/
@Nullable
List<?> getSelectedServerFeatures() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't make sense to put such an API here. This makes a factory class stateful.

Maybe it is time to eliminate XdsChannelFactory, instead we do need something like Helper.createResolvingOobChannel(), but accessible inside NameResolver.

Last time when I was doing some local e2e test for traffic splitting and routing, the gRPC channel is using plaintext, but the xDS channel is fixed to use TLS (if not google_default) instead of falling back to use the same configuration as its parent channel. This is broken for Java. It is not bitting us because we are always using TLS if not google_default in production.

We should support such a feature correctly. Basically, the creation of xDS channel will be done outside XdsClientImpl. For EDS only's code path, we are roughly able to do it with Helper.createResolvingOobChannel(), so not a problem for migrating.

For now, I think a better way is to change the createChannel API for XdsChannelFactory. You could define a simple class that contains a Channel and server features.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is coming from the factory or whatever structure being used for XdsClientIml consumes a list of server_urls. When you create the channel from it, you must know the index of the server_url that is selected, so that you can know the corresponding server feature. If it's only a single server_url, there is no problem at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A workaround is we pass just one ServerInfo instead of List<ServerInfo> to XdsClientImpl as well as XdsChannelFactory. Because we are actually not supporting multiple management server yet, and the current XdsChannelFactory has a hack to pick the hard coded first ServerInfo in the list.

Copy link
Contributor

@voidzcy voidzcy Jul 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I mean:

class XdsChannel {
  ManagedChannel channel;
  boolean useProtocolV3;  // or something similar if you have a better idea
}

XdsChannel createChannel(List<ServerInfo> servers) {
  ...
}

The output of creating channel contains the server feature.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also thought about that, which introduces yet another class and I had thought you wouldn't like it. I'm totally okay with it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:( Why would you think introducing a new class is bad? 😢 This looks fairly natural with channel and protocol feature encapsulated together.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -111,7 +112,11 @@ static BootstrapInfo parseConfig(String rawData) throws IOException {
channelCredsOptions.add(creds);
}
}
servers.add(new ServerInfo(serverUri, channelCredsOptions));
List<?> serverFeatures = JsonUtil.getList(serverConfig, "server_features");
if (serverFeatures != null && serverFeatures.contains(XDS_V3_SERVER_FEATURE)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the format of server_features is undefined. Maybe it's better to just parse it to a String list.

The parser should not try to interpret what is being parsed. It's just a list of string. "xds_v3" should be defined in the consumer class, which is XdsChannelFactory (e.g., String PROTOCOL_VERSION = "xds_v3). The channel factory looks up the parsed server features and check if the expected protocol version setting is there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log here should just print out what is parsed literally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the format of server_features is undefined. Maybe it's better to just parse it to a String list.

What do you mean actually? The following?

List<String> serverFeatures = JsonUtil.getListOfStrings(serverConfig, "server_features");

Or just pick all string type items in the list?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, just use JsonUtil.getListOfStrings.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@dapengzhang0
Copy link
Member Author

Added reading env variable for XdsChannelFactory. PTAL.

@dapengzhang0 dapengzhang0 merged commit 22b5480 into grpc:master Jul 31, 2020
@dapengzhang0 dapengzhang0 deleted the bootstrap-server-features branch July 31, 2020 01:00
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants