Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds: server-side listener network filter validation #4312

Merged
merged 4 commits into from
Apr 9, 2021

Conversation

easwars
Copy link
Contributor

@easwars easwars commented Apr 1, 2021

Summary of changes:

  • Main filter validation change is in the validateNetworkFilterChains() function in xds/internal/client/xds.go
    • it uses the processHTTPFilters() which validates http_filters on the client-side
  • server-side placeholder support for the Router filter
  • reduce test bloat

#psm-security-server-side-filter-support

@easwars easwars added the Type: Feature New features or improvements in behavior label Apr 1, 2021
@easwars easwars added this to the 1.38 Release milestone Apr 1, 2021
Copy link
Member

@dfawley dfawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future PRs, please try to keep non-trivial cleanup changes (e.g. the MarshalAny change which results in quite a bit of diffs) separated from non-cleanup changes.

* limitations under the License.
*/

package testutils
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to internal/testutils instead? Seems generically useful, not bound to xds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,36 @@
/*
*
* Copyright 2020 gRPC authors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2021 now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)
Done

Comment on lines -69 to -70
// Server side filters are not currently supported, but this interface is
// defined for clarity.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still mostly true, but removing this is fine. Just a note for future reference: I doubt we'll want to define server interceptors in iresolver since servers and resolvers have nothing to do with each other.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.

func validateNetworkFilterChains(filterChains []*v3listenerpb.FilterChain) error {
for _, filterChain := range filterChains {
if filterChain == nil {
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would there be a nil message in a repeated FilterChain element? I'm not even sure this is possible using the proto APIs in any language.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to special case for the default_filter_chain at the call site. So, I decided to add this nil check here. If you see around line 272, I make the following single call:

	if err := validateNetworkFilterChains(append(lis.GetFilterChains(), lis.GetDefaultFilterChain())); err != nil {
		return nil, err
	}

Even if I take two arguments for this function: one a slice of filter_chains, and one a default_filter chain, i would have to do a nil check for the latter.

What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, how about appending only if it's non-nil?

chains := lis.GetFilterChains()
if def := lis.GetDefaultFilterChain(); def != nil {
  chains = append(chains, def)
}
if err := validateNetworkFilterChains(chains); err != nil {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

for _, filter := range filterChain.GetFilters() {
name := filter.GetName()
if name == "" {
return errors.New("filter missing name field")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we improve this error message, and the ones below, at all? (e.g. there is a name field in the FilterChain this filter is within, although it seems optional.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried adding the filter_chain and the filter wherever appropriate. let me know if this looks better.

@dfawley dfawley assigned easwars and unassigned dfawley Apr 9, 2021
Copy link
Contributor Author

@easwars easwars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future PRs, please try to keep non-trivial cleanup changes (e.g. the MarshalAny change which results in quite a bit of diffs) separated from non-cleanup changes.

Sorry about that. I didn't want to get into it in this PR. But somehow got pulled into it as I was adding so many protos with anys.

func validateNetworkFilterChains(filterChains []*v3listenerpb.FilterChain) error {
for _, filterChain := range filterChains {
if filterChain == nil {
continue
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to special case for the default_filter_chain at the call site. So, I decided to add this nil check here. If you see around line 272, I make the following single call:

	if err := validateNetworkFilterChains(append(lis.GetFilterChains(), lis.GetDefaultFilterChain())); err != nil {
		return nil, err
	}

Even if I take two arguments for this function: one a slice of filter_chains, and one a default_filter chain, i would have to do a nil check for the latter.

What do you think?

for _, filter := range filterChain.GetFilters() {
name := filter.GetName()
if name == "" {
return errors.New("filter missing name field")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried adding the filter_chain and the filter wherever appropriate. let me know if this looks better.

Comment on lines -69 to -70
// Server side filters are not currently supported, but this interface is
// defined for clarity.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.

@@ -0,0 +1,36 @@
/*
*
* Copyright 2020 gRPC authors.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)
Done

* limitations under the License.
*/

package testutils
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@easwars easwars assigned dfawley and unassigned easwars Apr 9, 2021
func validateNetworkFilterChains(filterChains []*v3listenerpb.FilterChain) error {
for _, filterChain := range filterChains {
if filterChain == nil {
continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, how about appending only if it's non-nil?

chains := lis.GetFilterChains()
if def := lis.GetDefaultFilterChain(); def != nil {
  chains = append(chains, def)
}
if err := validateNetworkFilterChains(chains); err != nil {

Comment on lines 318 to 322
return fmt.Errorf("filter chain {%+v} has unsupported network filter: %s", filterChain, tc.GetTypeUrl())
}
hcm := &v3httppb.HttpConnectionManager{}
if err := ptypes.UnmarshalAny(tc, hcm); err != nil {
return fmt.Errorf("failed to unmarshal network filter: %v", err)
return fmt.Errorf("filter chain {%+v} failed unmarshaling of network filter: %v", filterChain, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the filter name to these two?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/known/anypb"

"google.golang.org/grpc/internal/testutils"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reorganize these imports?

Usually we do:

Standard packages
<blank line>
Almost everything else
<blank line>
Protos and, sometimes, renamed imports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intellij sometimes messes with these imports way too much. Not sure if I have to change any of my settings.

@dfawley dfawley assigned easwars and unassigned dfawley Apr 9, 2021
@easwars easwars merged commit fab5982 into grpc:master Apr 9, 2021
@easwars easwars deleted the inbound_listener_validation branch April 9, 2021 23:49
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Type: Feature New features or improvements in behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants