Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pilot not crash from invalid EnvoyFilters #17266

Closed
howardjohn opened this issue Sep 20, 2019 · 2 comments · Fixed by #17292
Closed

Make pilot not crash from invalid EnvoyFilters #17266

howardjohn opened this issue Sep 20, 2019 · 2 comments · Fixed by #17292

Comments

@howardjohn
Copy link
Member

@howardjohn howardjohn commented Sep 20, 2019

Right now it is pretty easy to kill Pilot with an EnvoyFilter. EnvoyFilters are breakglass, so its acceptable that a user will break some proxy configs, but it should not crash pilot, which will impact the entire cluster.

Some ways it can crash:

  • Generate invalid config
    // Generating invalid listeners is a bug.
    // Panic instead of trying to recover from that, since we can't
    // assume anything about the state.
    panic(retErr.Error())
  • Type mismatch
panic: proto.Merge(*v2alpha1.FilterConfig, *v2.HttpConnectionManager) type mismatch

goroutine 94 [running]:
github.com/gogo/protobuf/proto.Merge(0x2208c20, 0xc000746040, 0x2201860, 0xc0007c5b80)
        github.com/gogo/protobuf@v1.3.0/proto/clone.go:89 +0x6ff
istio.io/istio/pilot/pkg/networking/util.MergeAnyWithAny(0xc0011f0820, 0xc000f08be0, 0x1, 0xc00072a5e0, 0x1f96adb)
        istio.io/istio@/pilot/pkg/networking/util/util.go:514 +0xf3
istio.io/istio/pilot/pkg/networking/core/v1alpha3/envoyfilter.doHTTPFilterOperation(0xc000436180, 0x1, 0xc000f86150, 0xc0008a8a80, 0xc0010d80c0, 0xc0000fa9c0, 0xc0001c7d80, 0xc0011defd7)
        istio.io/istio@/pilot/pkg/networking/core/v1alpha3/envoyfilter/listener_patch.go:427 +0x3ef
istio.io/istio/pilot/pkg/networking/core/v1alpha3/envoyfilter.doHTTPFilterListOperation(0xc000436180, 0x1, 0xc000f86150, 0xc0008a8a80, 0xc0010d80c0, 0xc0000fa9c0)
        istio.io/istio@/pilot/pkg/networking/core/v1alpha3/envoyfilter/listener_patch.go:310 +0x17a
istio.io/istio/pilot/pkg/networking/core/v1alpha3/envoyfilter.doNetworkFilterOperation(0xc000436180, 0xc000000001, 0xc000f86150, 0xc0008a8a80, 0xc0010d80c0, 0xc0000fa9c0, 0xc0011df15f)
        istio.io/istio@/pilot/pkg/networking/core/v1alpha3/envoyfilter/listener_patch.go:286 +0x6d5
  • Probably others
@hzxuzhonghu

This comment has been minimized.

Copy link
Member

@hzxuzhonghu hzxuzhonghu commented Sep 21, 2019

/assign

@hzxuzhonghu

This comment has been minimized.

Copy link
Member

@hzxuzhonghu hzxuzhonghu commented Sep 21, 2019

We should simply warning and ignore the invalid envoyfilters but not panic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.