balancer: provides resolver.State Attributes in PickerBuildInfo

[nexcode]

Related to this pull request: #8696

Use case(s) - what problem will this feature solve?

Solves the issue of passing attributes for configuring the base balancer. Currently, you can only get connection-specific Attributes, but not general ones.

Proposed Solution

Add an Attributes field to the PickerBuildInfo:

// PickerBuildInfo contains information needed by the picker builder to
// construct a picker.
type PickerBuildInfo struct {
	// ReadySCs is a map from all ready SubConns to the Addresses used to
	// create them.
	ReadySCs map[balancer.SubConn]SubConnInfo

	// Attributes contains arbitrary data about the resolver intended for
	// consumption by the load balancing policy. This is a pointer to resolver.State Attributes.
	Attributes *attributes.Attributes
}

Additional Context

The manual resolver has a method for updating state. The state object, which this method receives, contains a field with Attributes, which is currently impossible to obtain when implementing the PickerBuilder interface. Need to add a field with these Attributes to the PickerBuilder.

Now we can only get the field with Addresses passed in the state, there are Attributes there too, but these Attributes provide parameters of a specific address, for example, weight, but the general settings of the balancer are specified at the top level in the field with Attributes.

[nexcode]

Currently, to obtain attributes common to the balancer, it is necessary to write these attributes to each available address and take the value from the first address, since the current API does not provide top-level Attributes.

In my project, instead of using top-level Attributes (they are already available in resolver.State), I write a common balancer attributres to each address:

state := resolver.State{
	Addresses: make([]resolver.Address, 0, len(serverInfoTree)),
	// Attributes: grpcattrs.SetClientConfig(nil, c.config), // https://github.com/grpc/grpc-go/pull/8696
}

for _, value := range serverInfoTree {
	state.Addresses = append(state.Addresses, resolver.Address{
		Addr:       value.Address,
		Attributes: grpcattrs.SetClientConfig(grpcattrs.SetAttributes(nil, value.Attributes), c.config),
	})
}

And then I just take the value from the first address when implementing the PickerBuilder:

var config *config.Client

for subConn, subConnInfo := range pickerInfo.ReadySCs {
	if config == nil {
		config = grpcattrs.GetClientConfig(subConnInfo.Address.Attributes)
	}

But instead of using the hack of getting the value from the first address, it should be possible to do this:

config := grpcattrs.GetClientConfig(pickerInfo.Attributes)

[easwars]

It feels a little weird to me that the picker needs access to resolver attributes.

Are you able to share what you plan to store in these attributes and why the picker needs access to them?

As a workaround, it should be possible to wrap the balancer.Builder and the balancer.Balancer returned by the base package in your own builder and balancer types. Once that is done, your balancer type will have access to these attributes (as part of the UpdateClientConnState method) and it can arrange for your base.PickerBuilder implementation to get access to the same. It does sound like a bit of a dance to make it happen though, which is why I'd be interested in knowing what you plan to store in those attributes and why the picker needs access to them.

Thanks.

[nexcode]

Balancer registration can only occur during init() func, so we can't create different balancers for different clients from our structure with different field values. Therefore, we have to use an empty structure that simply implements the interface.

I just want to fix the current API inconsistency. The resolver state has high-level Attributes that are currently unavailable when implementing the base balancer, and this definitely shouldn't be the case.

What data exactly will be there? You can take a comment from the current code:

Attributes contains arbitrary data about the resolver intended for consumption by the load balancing policy.

These Attributes are needed to be able to configure the balancer's logic differently for each client that will use it.

[nexcode]

I'm not really proposing anything new, I'm just solving the problem I discovered of inconsistent API between different gRPC packages.

We have a resolver.State:

type State struct {
	// Addresses is the latest set of resolved addresses for the target.
	//
	// If a resolver sets Addresses but does not set Endpoints, one Endpoint
	// will be created for each Address before the State is passed to the LB
	// policy.  The BalancerAttributes of each entry in Addresses will be set
	// in Endpoints.Attributes, and be cleared in the Endpoint's Address's
	// BalancerAttributes.
	//
	// Soon, Addresses will be deprecated and replaced fully by Endpoints.
	Addresses []Address

	// Endpoints is the latest set of resolved endpoints for the target.
	//
	// If a resolver produces a State containing Endpoints but not Addresses,
	// it must take care to ensure the LB policies it selects will support
	// Endpoints.
	Endpoints []Endpoint

	// ServiceConfig contains the result from parsing the latest service
	// config.  If it is nil, it indicates no service config is present or the
	// resolver does not provide service configs.
	ServiceConfig *serviceconfig.ParseResult

	// Attributes contains arbitrary data about the resolver intended for
	// consumption by the load balancing policy.
	Attributes *attributes.Attributes
}

This structure has top-level Attributes that are currently impossible to obtain in a balancer created from NewBalancerBuilder, because it receives a PickerBuilder that does not have these Attributes in PickerBuildInfo.

[easwars]

A couple of things upfront:

• The base balancer is in an experimental API and we can change it or delete it at any point in time. So, please be aware of that before using it.
• There used to be usages of the base balancer inside of gRPC in the past. That is not the case anymore.
• All the LB policies that are implemented by gRPC eventually delegate to pick_first, which is considered the universal leaf policy.
• We have a utility balancer that allows folks to build an LB policy that delegates to child policies. You can find that here: https://github.com/grpc/grpc-go/blob/master/balancer/endpointsharding/endpointsharding.go

You still haven't given us a picture on what exactly it is that you are intending to do. We might be able to help you use the most appropriate APIs if you tell us exactly what you are trying to do. Without that, we are shooting in the dark.