Updates to Envoy 1.17 and Istio 1.8 #177

codefromthecrypt · 2021-04-13T10:30:40Z

Istio 1.7 is EOL. This migrates to floor versions implied in Istio 1.8.

As Envoy 1.16.3 doesn't pass tests, this moves to latest Envoy 1.17.1

codefromthecrypt · 2021-04-13T23:20:27Z

istio test (TestConnectsToMockPilotAsAGateway) on macOS flaked twice, so I'll look into it prior to merge

codefromthecrypt · 2021-04-14T07:27:02Z

As Envoy 1.16.3 doesn't pass istio (XDS) tests, I had to move this to latest Envoy 1.17.1

codefromthecrypt · 2021-04-14T07:28:03Z

pkg/binary/envoy/controlplane/testdata/cert-chain.pem

@@ -1,21 +0,0 @@
-----BEGIN CERTIFICATE-----


these files were dead (never read). when we get to doing cert tests, we can add relevant ones but don't need to store dead files meanwhile

codefromthecrypt · 2021-04-14T07:28:44Z

pkg/binary/envoytest/util.go

+//   https://github.com/istio/istio/blob/1.8.4/istio.deps ->
+//   https://github.com/istio/proxy/blob/4cc266a75a84435b26613da6df6c32b4a2df4f3e/WORKSPACE ->
+//   https://github.com/istio/envoy/commit/5b0c5f7b21f84b6ab86e5f416bdaf6bb0fbc2a32 -> ~ 1.16
+// However, 1.16 fails to complete startup when XDS is configured (it blocks on 503/PRE_INITIALIZING).


here's the note about test failure

codefromthecrypt · 2021-04-14T07:31:58Z

pkg/binary/envoy/controlplane/istio.go

@@ -102,6 +103,7 @@ func generateIstioConfig(e *envoy.Runtime) meshconfig.ProxyConfig {
 	cfg.EnvoyAccessLogService = &meshconfig.RemoteService{Address: e.Config.ALSAddresss}
 	// Required: Defaults to MUTUAL_TLS, but we don't configure auth, yet, so it has to be set to NONE
 	cfg.ControlPlaneAuthPolicy = meshconfig.AuthenticationPolicy_NONE
+	cfg.Tracing = nil // Prevent server from hanging on unavailable Zipkin cluster


there were two reasons why the istio test hung, one was some bug in envoy with XDS, the other was this. Envoy wouldn't become available because there was no zipkin host ready.

codefromthecrypt · 2021-04-14T07:37:03Z

pkg/binary/envoy/controlplane/istio_test.go

+	// First, ensure r.Config.XDSAddress ended up written to config as the discoveryAddress
+	require.Contains(t, string(serverInfo), fmt.Sprintf(`"discoveryAddress": "%s"`, pilotGrpc))
+
+	// Next, ensure a connection discoveryAddress was made (Envoy -> Pilot's XDS)


While I can't commit to this now, you can probably see how this test could now be refactored into a unit test and an e2e test, where only the latter needs to run istio. We can mock the XDS service here in other words and doing so will massively clean up the dependency tree.

pkg/binary/envoy/controlplane/istio_test.go

codefromthecrypt · 2021-04-14T07:52:42Z

locally on OS/X I get a failure like this. I wonder if that means anything to anyone...

[2021-04-14 15:50:52.613][8367287][info][main] [external/envoy/source/server/server.cc:731] starting main dispatch loop
[2021-04-14T07:50:52.643Z] "GET /ready HTTP/1.1" 503 - 0 17 0 - "192.168.0.112" "Go-http-client/1.1" "-" "127.0.0.1:52913" "-"
[2021-04-14 15:50:57.615][8367287][critical][assert] [external/envoy/source/common/network/apple_dns_impl.cc:304] assert failure: interface_index == 0. Details: unexpected interface_index=4294967295
[2021-04-14 15:50:57.615][8367287][critical][backtrace] [bazel-out/darwin-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] Caught Abort trap: 6, suspect faulting address 0x7fff20435462
[2021-04-14 15:50:57.615][8367287][critical][backtrace] [bazel-out/darwin-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2021-04-14 15:50:57.615][8367287][critical][backtrace] [bazel-out/darwin-opt/bin/external/envoy/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:92] Envoy version: d6a4496e712d7a2335b26e2f76210d5904002c26/1.17.1/clean-getenvoy-bb8060d-envoy/RELEASE/BoringSSL
2021-04-14T07:50:57.616318Z	info	Envoy process (PID=41669) terminated via signal: abort trap
2021-04-14T07:50:57.616413Z	info	No Envoy processes remaining, terminating GetEnvoy process (PID=41668)
2021-04-14T07:50:57.616420Z	info	Envoy process (PID=41669) terminated prematurely
    util.go:143: 
        	Error Trace:	util.go:143
        	            				admin_test.go:37
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 4
        	Test:       	TestEnableEnvoyAdminDataCollection
        	Messages:   	never achieved status(3)

codefromthecrypt · 2021-04-14T10:49:49Z

there's no more failure, because I changed the test config to not try to resolve. however, the panic probably should be fixed upstream

Istio 1.7 is EOL. This migrates to floor versions implied in Istio 1.8. As Envoy 1.16.3 doesn't pass tests, this moves to latest Envoy 1.17.1 Signed-off-by: Adrian Cole <adrian@tetrate.io>

mathetake · 2021-04-14T13:21:27Z

pkg/binary/envoy/controlplane/istio_test.go

+// Technically what happens is bootstrap results in a "xds-grpc" cluster pointed at Pilot's gRPC address. During
+// bootstrap, Envoy invokes "/envoy.service.discovery.v3.AggregatedDiscoveryService/StreamAggregatedResources", which
+// reads configuration in pilot (from testdata/configs.yaml). This test passes when that configuration merges with the
+// bootstrap configuration generated by Istio.


sounds like you are an Istio/Envoy expert 😄 Thanks for the comment here!

pkg/binary/envoy/controlplane/istio_test.go

mathetake

Thank you!!

Signed-off-by: Adrian Cole <adrian@tetrate.io>

…stio Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt · 2021-04-15T01:18:39Z

I moved the disable zipkin hack to test code so that the default template still works as advertised. Also, added a comment because it seems hard-coding might end in istio 1.10 istio/istio#31553 (comment)

mathetake · 2021-04-15T01:36:34Z

Given that we have only been supported Istio 1.2 before you worked on the refactoring work here, I am thinking that we could eliminate entire Istio stuff from GetEnvoy considering the maintenance work..

codefromthecrypt · 2021-04-15T01:43:41Z

@mathetake I'm game for removal and will raise a PR post merge (since this works now, at least it will be in source history in case someone wants to try again). I agree this istio topic alone has cost days of effort not including weekend time, and it is likely not in use.

mathetake · 2021-04-15T01:51:12Z

sg

codefromthecrypt · 2021-04-15T02:55:09Z

#178 on removing istio

codefromthecrypt requested review from lizan and mathetake April 13, 2021 10:30

codefromthecrypt mentioned this pull request Apr 13, 2021

Clarify version support policy #169

Closed

mathetake approved these changes Apr 13, 2021

View reviewed changes

codefromthecrypt changed the title ~~Updates to Envoy 1.16 and Istio 1.8~~ Updates to Envoy 1.17 and Istio 1.8 Apr 14, 2021

codefromthecrypt commented Apr 14, 2021

View reviewed changes

codefromthecrypt requested a review from mathetake April 14, 2021 07:40

codefromthecrypt commented Apr 14, 2021

View reviewed changes

pkg/binary/envoy/controlplane/istio_test.go Show resolved Hide resolved

Updates to Envoy 1.17 and Istio 1.8

0e4652a

Istio 1.7 is EOL. This migrates to floor versions implied in Istio 1.8. As Envoy 1.16.3 doesn't pass tests, this moves to latest Envoy 1.17.1 Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt force-pushed the envoy1.16-istio1.8 branch from ff3cfe7 to 0e4652a Compare April 14, 2021 10:51

mathetake reviewed Apr 14, 2021

View reviewed changes

pkg/binary/envoy/controlplane/istio_test.go Outdated Show resolved Hide resolved

mathetake approved these changes Apr 14, 2021

View reviewed changes

Adrian Cole added 3 commits April 15, 2021 07:46

leftover from debugging

cd8c889

Signed-off-by: Adrian Cole <adrian@tetrate.io>

Adds follow-up issue

d873a93

Signed-off-by: Adrian Cole <adrian@tetrate.io>

Removes main-path zipkin hack and renames to avoid clashing on word I…

bd91370

…stio Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt merged commit cec572f into master Apr 15, 2021

codefromthecrypt deleted the envoy1.16-istio1.8 branch April 15, 2021 02:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates to Envoy 1.17 and Istio 1.8 #177

Updates to Envoy 1.17 and Istio 1.8 #177

codefromthecrypt commented Apr 13, 2021 •

edited

Loading

codefromthecrypt commented Apr 13, 2021

codefromthecrypt commented Apr 14, 2021

codefromthecrypt Apr 14, 2021

codefromthecrypt Apr 14, 2021

codefromthecrypt Apr 14, 2021

codefromthecrypt Apr 14, 2021

codefromthecrypt commented Apr 14, 2021

codefromthecrypt commented Apr 14, 2021

mathetake Apr 14, 2021

mathetake left a comment

codefromthecrypt commented Apr 15, 2021

mathetake commented Apr 15, 2021

codefromthecrypt commented Apr 15, 2021

mathetake commented Apr 15, 2021

codefromthecrypt commented Apr 15, 2021

Updates to Envoy 1.17 and Istio 1.8 #177

Updates to Envoy 1.17 and Istio 1.8 #177

Conversation

codefromthecrypt commented Apr 13, 2021 • edited Loading

codefromthecrypt commented Apr 13, 2021

codefromthecrypt commented Apr 14, 2021

codefromthecrypt Apr 14, 2021

Choose a reason for hiding this comment

codefromthecrypt Apr 14, 2021

Choose a reason for hiding this comment

codefromthecrypt Apr 14, 2021

Choose a reason for hiding this comment

codefromthecrypt Apr 14, 2021

Choose a reason for hiding this comment

codefromthecrypt commented Apr 14, 2021

codefromthecrypt commented Apr 14, 2021

mathetake Apr 14, 2021

Choose a reason for hiding this comment

mathetake left a comment

Choose a reason for hiding this comment

codefromthecrypt commented Apr 15, 2021

mathetake commented Apr 15, 2021

codefromthecrypt commented Apr 15, 2021

mathetake commented Apr 15, 2021

codefromthecrypt commented Apr 15, 2021

codefromthecrypt commented Apr 13, 2021 •

edited

Loading