-
Notifications
You must be signed in to change notification settings - Fork 7.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
xds: improve incremental logging #39889
xds: improve incremental logging #39889
Conversation
Currently, we do not log incremental pushes at Info level. The intent behind this is to avoid spam when we have large endpoint churn. However, because we also do incremental pushes for Full pushes now, we are also hiding these logs. These logs are both critical to debugging (things like istio#39720, etc) and not spammy -- while the `Full=false` pushes may add thousands of messages, this change only adds at most 1 log per push/proxy. For these types of pushes I don't see a benefit to excluding only EDS. Additionally, fix SDS to correctly assert it is incremental (when it is).
🤔 🐛 You appear to be fixing a bug in Go code, yet your PR doesn't include updates to any test files. Did you forget to add a test? Courtesy of your friendly test nag. |
@@ -148,7 +148,10 @@ func (s *SecretGen) Generate(proxy *model.Proxy, w *model.WatchedResource, req * | |||
results = append(results, res) | |||
} | |||
} | |||
return results, model.XdsLogDetails{AdditionalInfo: fmt.Sprintf("cached:%v/%v", cached, cached+regenerated)}, nil | |||
return results, model.XdsLogDetails{ | |||
Incremental: updatedSecrets != nil, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this fixing a bug?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes but just one impacting logging so not critical at all
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not immediately obvious that it only affects logging. Are you saying that the Incremental
flag isn't used by any other istiod logic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, its part of the XdsLogDetails
struct
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oof missed that, sorry for the noise :)
LGTM |
* Automator: update istio/api@master dependency in istio/istio@master (istio#39723) * Allow running TestSDS concurrently (istio#39715) * echo: avoid filling defaults twice (istio#39735) We already do this at a higher level in `(i *Instance) ForwardEcho` * tf: prevent blocking eastwest gateway with test policies (istio#39742) * xds: respond to requests previously miscategorized as ACKs (istio#39746) * xds: respond to requests previously miscategorized as ACKs See envoyproxy/envoy#13009 for details Fixes istio#38709 (previously 'fixed', but really the fix was a workaround) Fixes istio#39720 * fix tests * Make integ tests more aggressive * mod: bump quic-go dependency (istio#39744) * Automator: update proxy@master in istio/istio@master (istio#39749) * updated the pullsecret logic (istio#39750) * Improve tests and logs around meshconfig update (istio#39748) * Improve tests and logs around meshconfig update To help debug istio#39747. The test, I thought, would reproduce it -- but it didn't * fix lint * use mesh config defaults when log formatter is not specified (istio#39606) * use mesh config defaults when log formatter is not specified Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * only honour mesh config values for default provider Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * update comment Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39722) * do not cache dependent configs in xds cache store (istio#39688) * Make xds cache not cache the dependentConfigs, instead start up a thread which will cleanup in background * update xds cache to uee channel notify instead of periodically evict * rename call back evict to onEvict * Added test * update * Fix * fix flake test * address comments * address comments * remove mis added pprof files * add metrics for dependent config size (istio#39755) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39753) * upgrade go control plane (istio#39756) * upgrade go control plane Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * filter gen Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * go sum Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Not really evict a key that is added later after LRU evict bt before the evict handler run by istio (istio#39764) * Donot evict keys that are added again after evict * Add test * grafana: update to v9.x.x (istio#39670) * use config hash for dependent configs (istio#39665) * use config hash for dependent configs Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix compile Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix vet Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Revert "do not cache dependent configs in xds cache store (istio#39688)" (istio#39782) This reverts commit 898d167. * clean telemetryv2 files (istio#39781) * clean telemetryv2 files * fix gen * cluster: fix NACK when using STATIC Service with PASSTRHROUGH (istio#39745) * cluster: fix NACK when using STATIC Service with PASSTRHROUGH Fixes istio#39736 (cherry picked from commit f1e4947dff7b0f8534950c1009e6b1278abb0a2f) * lint * Automator: update common-files@master in istio/istio@master (istio#39784) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39785) * Automator: update proxy@master in istio/istio@master (istio#39790) * minor refactor in cluster builder (istio#39757) * fix cluster nack for strict dns clusters Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add comments and tests Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add additional condition Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * rearrange Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/api@master dependency in istio/istio@master (istio#39803) * Run `make gen` (istio#39807) * Tls cert cacerts secret format (istio#39732) * added the ability to read tls type secrets * formatting * release notes * removed log line * fixed notes * fixed tests * fixed tests again * added autodetection * formatting * added logs and bug fix * moved to ca file * fixed tests * removed unused code * fixed missing file issue * fixed tests * make gen * formatting * [tf] Refactoring top-level security tests (istio#39453) (istio#39743) This refactors all of the top-level security tests to use the common echo deployment and the new echotest framework. Also moves the authz tests back to the top-level, since it can now share the same TestMain. This is a roll-forward of the original PR. Reverts commit fd35962. * Update BASE_VERSION to master-2022-07-06T19-01-15 (istio#39818) * minor comment changes in xds (istio#39769) * minor optimiztion in repeated nonce case handling Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * minor comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * make gen Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * revert make gen Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * implement max connection duration (istio#39765) * implement max connection duration Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * added release notes Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix it Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add max connection duration validation Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update proxy@master in istio/istio@master (istio#39821) * add split DRs to rds/cds/eds dependent configs (istio#39730) * Agent: Unified use of NewSecretManager func (istio#39824) * Agent: Unified use of NewSecretManager func * update * fi (istio#39760) * fix injection check in x injector list (istio#39523) * ignore ignored namespaces (istio#39569) * [tf] flag to only deploy specific apps istio.test.onlyWorkloads (istio#39338) * [tf] flag to skip deployment of specific apps Change-Id: I7f1dbf6c7ec5bbd26fe2deb195efc0ee5c5f26e2 * Revert "[tf] flag to skip deployment of specific apps" This reverts commit 18ef8a5. * [tf] flag to require specific workload classes Change-Id: Ief5450e256008ee8bd1ac1b0e1a50843a2e11dbb * Fix bug(istio#38077) that causes a stale SA token is being used in istio-cni (istio#39801) * Fix bug(istio#38077) that causes a stale SA token is being used in istio-cni The BoundServiceAccountTokenVolume Kubernetes feature (graduated to stable in 1.22) improves security of service account tokens by requiring a one hour expiry time, over the previous default of no expiration. This means that install-cni should refetch service account tokens periodically. This PR integrates periodic SA token checks into already existing sleepCheckInstall function. * Fix goroutine leak in watchSAToken * Fixed unused params in cni tests * Add a release note * Fix data race in watchSAToken Co-authored-by: Sergei Gavrilov <sergei@gavrilov.work> * Automator: update proxy@master in istio/istio@master (istio#39832) * add description to admin log (istio#39553) * Respect GA topology label first (istio#39823) * Fix analyze conflicting mesh gateway with exportTo (istio#39729) * fix exportto not work in analyze * fix * add releasenotes * Update releasenotes/notes/39729.yaml Co-authored-by: Eric Van Norman <ericvn@us.ibm.com> Co-authored-by: Eric Van Norman <ericvn@us.ibm.com> * Manual update of istio/api including update for StatPrefix (istio#39831) * Manual update of istio/api including update for StatPrefix * Run make gen * pull latest api * Forgot make gen * Update to latest api * Additional test updates from Rama * [tf] load templates outside of init func (istio#39840) * [tf] load templates outside of init func This makes sure that the flags are parsed and the new values mentioned in the flags are used. * remove tmplMap as it can cause concurrent map writes * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39804) * Automator: update proxy@master in istio/istio@master (istio#39846) * Automator: update proxy@master in istio/istio@master (istio#39850) * hbone: initial echo server/client implementation (istio#39645) * hbone: initial echo server/client implementation * some fixes * cleanup * echo: avoid filling defaults twice We already do this at a higher level in `(i *Instance) ForwardEcho` (cherry picked from commit d99cb3c) * Fix: allow enableNamespacesByDefault when revision tag is set (istio#39674) * Respect enableNamespacesByDefault when revision tag is set * Fix lint * Reorder default revision control * Check for the installed revision in default mutating webhook configuration * Fix istiod remote * Fix istiod remote * Fix istiod remote * Add release note * fix: update the typo for pkg spiffe (istio#39768) * add type to xds cache evictions stat (istio#39853) * add type to xds cache evictions stat Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * revert unnecessary change Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/api@master dependency in istio/istio@master (istio#39859) * gateway-api: bump dependency to fix flakes (istio#39862) * Refactoring Security Tests (istio#39648) * refactored test to support new test framework * refactored ca_custom_root test pick f814c68094 refactored test to support new test framework pick ccad9ac7b2 refactored ca_custom_root test * added custom setup * refactor custom echo deployment * remove external custom setup * Fix KubernetesResources builder for fuzzer (istio#39861) * Add testcases for mockclient_PodsForSelector (istio#39847) * Add TestMockClient_PodsForSelector test * check lint_go.sh * fix: update the typo code comment for pkg istioctl/cmd (istio#39767) * fix istioctl ps eds (istio#39849) * possible dependent config leak (istio#39854) * possible dependent config leak Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * call delete in clear Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * disable sending unhealthy endpoints by default (istio#39834) * disable panic threshold by default Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * disable send healthy endpoints by default Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix release notes Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix ut Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update proxy@master in istio/istio@master (istio#39867) * Fix x injector ns pod info not accurate when use default (istio#39525) * fix pod revision compare * add releasenotes * revise revision extraction method * revise to only use annotation * Add protocol check for empty address in Service Entry (istio#39495) * Add protocol check for empty address in Service Entry Signed-off-by: xiaoxu <lexuscyborg103@gmail.com> * Add release-notes of issue 27990 Signed-off-by: xiaoxu <lexuscyborg103@gmail.com> * Update pkg/config/validation/validation.go Co-authored-by: John Howard <howardjohn@google.com> Co-authored-by: John Howard <howardjohn@google.com> * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39860) * Fix operator test on 1.25 (istio#39883) * XDS Cache Tests and minor improvements (istio#39713) * minor cache improvements * add xds cache tests * add copyright and license * goimports * use write lock on add * Create topology flag to proxy only kubectl traffic through HTTPProxy (istio#39865) * validation: do not warn about ECDS types (istio#39881) * xds: improve incremental logging (istio#39889) Currently, we do not log incremental pushes at Info level. The intent behind this is to avoid spam when we have large endpoint churn. However, because we also do incremental pushes for Full pushes now, we are also hiding these logs. These logs are both critical to debugging (things like istio#39720, etc) and not spammy -- while the `Full=false` pushes may add thousands of messages, this change only adds at most 1 log per push/proxy. For these types of pushes I don't see a benefit to excluding only EDS. Additionally, fix SDS to correctly assert it is incremental (when it is). * tf: fast mesh config update for Istiodless (istio#39878) This is cherrypicked from release-1.8 branch. Fixes istio#39747 * Automator: update common-files@master in istio/istio@master (istio#39891) * fix: update the typo code comment (istio#39866) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39894) * initialize commonLbConfig in default cluster (istio#39877) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Enable TestTunnelingOutboundTraffic and configure access log in forward proxy (istio#39774) * Enable TestTunnelingOutboundTraffic and configure access log in forward proxy Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> * Add remoteAddr to the HTTP server log Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> * Add test for SetXForTest (istio#39887) * Add test for SetXForTest * banner * Fix bug blocking JSON marshal of endpointShardz (istio#39893) * fix bug for the istio-agent wait command to make sure that it is really timeout (istio#39875) * fix bug for the istio-agent wait command to make sure that it is really timeout * change code based on comments * fix lint * fix integration test failed * for sleep time periodMillis should be Milliseconds * for sleep time periodMillis should be Millisecond * improve telemetry bench test (istio#39899) * Virtual service direct response (istio#39776) * virtual service direct response * remove port * split line * add size warning and error * fix locality indexes * fix lint * xds: improve req log consistency (istio#39892) * xds: improve req log consistency This mirrors the PUSH log syntax to make it easier to read * fmt * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39909) * tf: make requests a bit more leniant (istio#39910) We have seen substantial flakes recently. These are partially due to loaded systems (for example, we have seen issues where XDS takes >20s to update. This is not a bug, Envoy is just slow (likely CPU overloaded)). Other legitimate bugs don't typically recover at all, so I don't think we will ignore many legitimate issues (other than istio#38982, which we already know about and cannot do much about). This bumps the timeout a big, increases retry interval a bit as well to reduce load, and reduces the total requests sent. 3 should still be sufficient to get cross-X load balancing, as we send `requests*clusters` already so we have 9 requests. * Prevent illegal logs in goroutines in hbone test (istio#39918) * tf: add explicit loggs when stuck in warming state (istio#39890) Lately a bunch of issues have been caused by things stuck warming. This makes these obvious to see without poking through 10000s of lines of artifacts * Fix typo of InsertDataToConfigMap func description (istio#39902) * Automator: update proxy@master in istio/istio@master (istio#39908) * Automator: update common-files@master in istio/istio@master (istio#39907) * Automator: update proxy@master in istio/istio@master (istio#39920) * tf: drop unused feature (istio#39923) This flag requires the control plane to also have support, and its not enabled. This also breaks a (test only) assertion added in istio#39916. * Automator: update proxy@master in istio/istio@master (istio#39924) * remove relative operation error for envoy filter MERGE operation (istio#39904) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * remove year from copyright (istio#39900) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * xds: fix issue when skipping first request (istio#39916) The bug (example failure: https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/39896/integ-cni_istio/1546984256612339712) * ingressgateway has 1 Gateway * Gateway is removed * envoy disconnects * Envoy reconnects, requests RDS. This hits the INIT/RECONNECT flow. * RDS hits 'Gateway missing for route' path and gives no response at all * Next RDS request, we get a "stale nonce" since we have no previously sent nonce * Envoy stuck forever The fix: * Remove code path to return empty route instead of no route (matching other paths) * Add assertions to ensure that we don't send empty response to requests and that we never count a "stale nonce" if we somehow have no previously sent nonce, to ensure there aren't any other issues * gateway-api: bump to v0.5.0 (istio#39917) * gateway-api: bump to v0.5.0 We were just a couple commits behind, but this lines us up to the stable release version * make us conformant again (sort of) * lint * add xfcc authenticator (istio#39405) * add xfcc authenticator Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * go mod Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add license Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix folder Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add copying file Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * remove COPYING Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address review comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add cidr authenticator Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * lint Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * minor changes Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add more comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * update based on review comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * review comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add authentication manager Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * add test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * address comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * tf: improve dumping (istio#39915) * tf: improve dumping This was initially a bug fix but then I realized I was misguided. However, along the way I made some improvements to the dumping logic. * We were dumping istio-system 2x; fix that * We were using PodExec which is slow. Switch to port forward which is 10x+ faster. Also use a more robust mechanism to resrve ports * Dump in parallel * Add some better logging around dumping * Clear managedFields from events for size/readability * lint * rebase * Automator: update proxy@master in istio/istio@master (istio#39936) * Handle CDS and EDS case (istio#39937) (cherry picked from commit ec36e8362c34acb91c1590ac2db695156eb78d93) * dns: added DNSForwardParallel to support sending parallel queries to all nameservers (istio#39857) * Make istio-cni compatible with Talos Linux (go netns implementation + added ca cert path) (istio#39699) * cni: ca-certificates path for Talos Linux by adding the correct ca path for Talos Linux this eeffectively removes `warnOS CA Cert could not be found for agent` on cni execution, as `containernetworking/cni` version checker cannot unmarshall with this warning. part of fix for issue istio#38794 Signed-off-by: Nico Berlee <nico.berlee@on2it.net> * cni: netns implementation in go Removes by default the dependency for nsenter util on the host system. This makes istio-cni compatible with very thin osses like Talos Linux which do not ship with nsenter. `"HostNSEnterExec": true` in cni configmap reverts to old previous behavior Fixes partly istio#38794 Signed-off-by: Nico Berlee <nico.berlee@on2it.net> * tf: loosen restrictions on 'real stackdriver' testing (istio#39948) * add cache for accesslog (istio#39751) * add cache for accesslog * fix UT * reuse mutex * cache accesslog in PushContext * fix UT * revert changes * refactor cache in telemetry * make AccessLogging cachable * add tests * fix lint * fix tests * fix nit * Convert mockprom to metrics endpoint test to a subtest (istio#39941) Previous to this PR, the TestStatsFilter testcase checks the stats filter and also tests if a mocked prometheus app can call the app metrics endpoint. This PR moves the mocked prometheus test to a subtest so that it can be skipped if required. * Automator: update proxy@master in istio/istio@master (istio#39949) * Automator: update proxy@master in istio/istio@master (istio#39954) * Fix resiliience port-forward (istio#39959) * Fix resiliience port-forward * update * Bump distroless base (istio#39968) * Fixed issue with iterator variable in taint controller (istio#39977) * Refact authenticator (istio#39690) * Combine authenticator's Auth methods * fix * Address rama's comments * Mitigate TestDNS flakes (istio#39971) Two issues: * Port conflict on 15053. Pretty simple, bind to port 0 * We want to test servers supporting TCP+UDP on one port, but we cannot atomically reserve a free tcp and udp port. Currently we reserve UDP first then bind to TCP. This just swaps the order. In practice we have less UDP listeners, so much less chance of conflict. This dropped flakes from 1/10 to 1/1000 on my machine * Automator: update proxy@master in istio/istio@master (istio#39993) * Skip forward when the proxy connection is closed (istio#39956) * Skip forward when the proxy connection is closed * Address comment * handle non wildcard resources during reconnect (istio#39960) * handle non wildcard resources in resource warming duing reconnect Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * warming dependencies Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * remove test case Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * remove lds/rds dependency Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Add test for unfixed SNI overlapping bug (istio#39922) I was working on a fix but don't have time to complete for now. At least we can check in the test case to codify the bug and make it simpler when we fix it eventually. * lint: fix usages of legacy proto package (istio#39964) * lint: fix usages of legacy proto package Only valid usage is jsonpb, once https://github.com/istio/common-files/pull/633/files merges * test fix * tf: export external service to same namespace only (istio#39888) * tf: export external service to same namespace only This avoids spammy logs about overlapping hostnames. This also makes it so we actually send to the per-namespace external service I think. This shouldn't really matter much in practice. * fix telemetry * fixes * Various test framework improvements (istio#40002) * Various test framework improvements * lint * test reachability: make protocol on outer loop (istio#39972) I think this may effectively work around istio#38982 by ensuring we don't send to multiple ports in parallel anymore. Runtime and coverage should be the same, only order changed. * Add AlwaysRespond logic to Delta XDS as well (istio#40003) In testing we confirmed the same thing applies to delta, and the same fix should work. Adding it there. Also bumping the main one down to only apply if its already an ACK to avoid confusing logs. The behavior, aside from logs, is the same. * Bump api (istio#40000) * Automator: update istio/api@master dependency in istio/istio@master * Fix annotation test Co-authored-by: istio-testing <istio-testing-bot@google.com> * added imagePullSecret in custom jwt-server (istio#40020) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#39976) * Change the warning message to debug message (istio#40021) Fixes istio#40019 Signed-off-by: Tong Li <litong01@us.ibm.com> * tf: wait for CRDs to be established before Gateway tests (istio#40016) * Automator: update proxy@master in istio/istio@master (istio#40024) * Add well known credential uds socket to allow plugin external UDS SDS server (istio#39135) * use wellknown socket path for CredentialName * add make gen * [e2e test]add telemetry e2e test with default provider (istio#39457) * [e2e test]add telemetry e2e test with default provider * fix nit * fix review comments * fix lint * fix deleteTelemetryResource * remove max concurrent streams default (istio#40009) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Check AllowAny mode in RDS cache key (istio#40014) * Check AllowAny mode in RDS cache key Fixes istio#39794 (comment) * Add note * clean proxy_init.resources (istio#39833) Signed-off-by: hejianpeng <hejianpeng2@huawei.com> * improve cni logging (istio#39988) * improve cni logging * fix UT * address comments * updated image registry for custom jwt server (istio#40031) * Automator: update proxy@master in istio/istio@master (istio#40042) * gateway-api: support invalid BackendRefs (istio#40007) * bump envoyproxy/go-control-plane * support invalid BackendRefs * add unit tests * test gateway conformance * fix test * address comments * Automator: update istio/api@master dependency in istio/istio@master (istio#40045) * Refactor to remove InputParams struct (istio#40030) * Refactor to remove InputParams struct This struct is no longer used beyond tests, after we stopped using 'Plugins' concept. This fully cleans up the struct from tests that were still using it. * drop plugin * Ensure a few fields are non-null for fuzz tests (istio#40044) * istio: register init push context metric (istio#40049) Change-Id: I61825036af32dfb5efc890606708594dd687780c Reviewed-on: https://gerrit.musta.ch/c/public/istio/+/3310 Reviewed-by: Weibo He <weibo.he@airbnb.com> * Automator: update common-files@master in istio/istio@master (istio#40048) * Automator: update proxy@master in istio/istio@master (istio#40051) * Revert "Fix: allow enableNamespacesByDefault when revision tag is set (istio#39674)" (istio#40050) This commit made it so that we fail installation if a previous install exists without a default tag. In addition it makes the current revision the default even if a previous default tag exists. * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40046) * Automator: update common-files@master in istio/istio@master (istio#40055) * Move go.mod to 1.18 (istio#40028) This enables istio#36308 (comment) * Remove unused attributes of WatchedResource to reduce memory (istio#39945) * Abstract gateway context outside of mode package (istio#40035) * Automator: update proxy@master in istio/istio@master (istio#40063) * Add a flag for app container name and retry verifyTrafficMirror (istio#40053) For some workloads the container name is different and the logs can be slower. So the verification of mirroring test cases is retried for 20 seconds. * use common MessageToAny method everywhere (istio#40041) * use common MessageToAny method everywhere * add missing reference * cleanup ioutil (istio#40062) Signed-off-by: yxxhero <aiopsclub@163.com> * check DNS Proxying for headless svc (istio#40023) * Automator: update proxy@master in istio/istio@master (istio#40070) * fix lint (istio#40056) * fix lint * make gen * istioctl: set default port from webhook (istio#40069) Kubernetes doesn't require port, but ADSC does. Set a default if its not there yet. * Automator: update proxy@master in istio/istio@master (istio#40075) * Minor comments fix (istio#40079) Signed-off-by: Zhonghu Xu <xuzhonghu@huawei.com> * Proxy labels should be updated when pod/wle labels updated (istio#40036) * Proxy labels should be updated when for example pod labels updated * make pod labels not replace all node meta labels * fixlint * Adress comments * Added localityLabel to pod label so we can get proxy locality from pod labels rather than service instances, which may not exist * Refactor: abstract setTopologyLabels * Update * update * pod label change trigger proxy update * wle label change trigger proxy update * update tests * lint * fix * handle removes (#1) * handle removes * handle removes Signed-off-by: Aditya Prerepa <adiprerepa@gmail.com> * update * refresh golden files Co-authored-by: Aditya Prerepa <adiprerepa@gmail.com> * fix typo in pilot.go (istio#40084) authenication -> authentication * use same xds types every where (istio#40088) * use same types every where Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * change delta Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Update constant definitions for metrics (istio#40086) * Automator: update common-files@master in istio/istio@master (istio#40099) * Automator: update proxy@master in istio/istio@master (istio#40095) * Revert "Add a flag for app container name and retry verifyTrafficMirror (istio#40053)" (istio#40096) This reverts commit d3b1687. * move more fuzzers over to native fuzzers (istio#40029) * move more fuzzers over to native fuzzers * banner * lint * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40100) * Rewrite `interface{}` to `any` (istio#40073) * Rewrite any to anypb * Rewrite interface{} to any * regen * Remove validation of TunnelSettings.Protocol for empty string (istio#40102) Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> * Revert "Proxy labels should be updated when pod/wle labels updated (istio#40036)" (istio#40098) This reverts commit 5f90e4b. * delete multi-arch selector in the gateway templates (istio#40068) * Refactor resolution of network gateway names for more efficiency (istio#39836) * Refactor resolution of network gateway names for more efficiency * Fix lint * Fix DNS record type switch * Automator: update proxy@master in istio/istio@master (istio#40117) * Simpler injectionPath format for cluster env value with / char (istio#39979) * Simpler injectionPath format for cluster env value with / char * fix lint * update comment * more comment update * improve test * Automator: update common-files@master in istio/istio@master (istio#40119) * Automator: update proxy@master in istio/istio@master (istio#40120) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40121) * Automator: update proxy@master in istio/istio@master (istio#40129) * xdstest: use some generic functions instead of hacks (istio#40074) * xdstest: use some generic functions instead of hacks * compile * Fix nil * license * Update copyright (istio#40126) * Update copyright Signed-off-by: Xiao, Ziyang <ziyang.xiao@intel.com> * remove update in meshca.pb.go Signed-off-by: Xiao, Ziyang <ziyang.xiao@intel.com> * Update Wasm Dashboard Default Time (istio#40130) * Update Wasm Dashboard Default Time * resolve ci errors Signed-off-by: Xunzhuo <mixdeers@gmail.com> * move send unhealthy endpoint flag to atomic bool (istio#40140) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * fix multi WasmPlugin with different imagePullSecrets (istio#40093) * fix ecds secret cache * add release-notes * address comment and fix typo * Automator: update common-files@master in istio/istio@master (istio#40150) * Add default validator template to istio-remote (istio#40149) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40152) * Automator: update proxy@master in istio/istio@master (istio#40161) * Support list type in analyze (istio#40085) * support list type in analyze * revise parseChunk func * refactor response checker (istio#40156) * improve build_push_update_images.sh (istio#40170) Signed-off-by: xin.li <xin.li@daocloud.io> * workload instance cause stale CDS clusters of type STRICT_DNS (istio#39947) * workload instance cause stale CDS clusters with of type STRICT_DNS * added release note * fewer full push triggers * code review comments for release note * extend logic for DNS_ROUND_ROBIN * update trigger reason to EndpointUpdate * add unit tests * minor refactor in gateway api (istio#40171) Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40175) * test: use `T.Setenv` to set env vars in tests (istio#40176) This commit replaces `os.Setenv` with `t.Setenv` in tests. The environment variable is automatically restored to its original value when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.Setenv Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> * Update BASE_VERSION to master-2022-07-29T19-01-38 (istio#40179) * Automator: update common-files@master in istio/istio@master (istio#40180) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40181) * Automator: update proxy@master in istio/istio@master (istio#40183) * Fix ist0103 msg incorrect displaying when injection label is set to false (istio#40164) * fix ist0103 incorrect fireing * fix unit test * Automator: update proxy@master in istio/istio@master (istio#40191) * Prevent calling json.MarshalIndent and handle err, instead using writeJSON (istio#40197) * Automator: update common-files@master in istio/istio@master (istio#40200) * fuzz: attempt to fix build by working around fuzzing limitation (istio#40203) * fuzz: attempt to fix build by working around fuzzing limitation * Also add to owners * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40202) * Automated branching step 1 (istio#40206) * Automator: update proxy@master in istio/istio@master (istio#40208) * Automator: update common-files@master in istio/istio@master (istio#40212) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40214) * Automator: update proxy@master in istio/istio@master (istio#40215) * pilot: fix issue with TLS and TCP order dependency (istio#40072) * pilot: fix issue with TLS and TCP order dependency Currently, tls_inspector is only added if its the first Service. If its after TCP, it is missed. example config: ```yaml apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: tcp spec: addresses: - 10.10.10.10/24 exportTo: - . hosts: - '*.tcp' ports: - name: tcp-443 number: 443 protocol: TCP --- apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: tls spec: exportTo: - . hosts: - tls.example.com location: MESH_EXTERNAL ports: - name: https-443 number: 443 protocol: HTTPS resolution: DNS ``` depending on the order of creation it will work/not work. * note * Make it more generic and better tested * optimize handleStats temporary big byte slice (istio#40109) * add global stats buffer on handleStats * remove agent errors counter * remove process metrics function * app metrics must go last * scrape agent first * process metrics without copy slice * replace with # * use multi buffer * fix build * fix comment * reader director write to response * fix problem * fix lint * fix cancel func * add benchmark Signed-off-by: Patrick <patrickjiang0530@gmail.com> * modify benchmark Signed-off-by: Patrick <patrickjiang0530@gmail.com> * fix ci lint Signed-off-by: Patrick <patrickjiang0530@gmail.com> * add copyAndProcessMetrics Signed-off-by: Patrick <patrickjiang0530@gmail.com> * fix lint and nr <= 0 Signed-off-by: Patrick <patrickjiang0530@gmail.com> * use bufPool * fix lint and problem * add copyAndProcessMetrics unit tests Signed-off-by: Patrick <patrickjiang0530@gmail.com> * fix unit test * revert server for benchmark Signed-off-by: Patrick <patrickjiang0530@gmail.com> * revert server_test for benchmark Signed-off-by: Patrick <patrickjiang0530@gmail.com> * revert server_test for benchmark Signed-off-by: Patrick <patrickjiang0530@gmail.com> * Revert "revert server_test for benchmark" This reverts commit b3551c0. * Revert "revert server_test for benchmark" This reverts commit 63be4af. * Revert "revert server for benchmark" This reverts commit 5a3fb73. * fix unit test Signed-off-by: Patrick <patrickjiang0530@gmail.com> * remove proccess metrics * remove proccess metrics * fix imports * modify benchmark * modify benchmark * revert to test benchmark * fix tests * fix unit test * fix benchmark tests * fix benchmark * Revert "revert to test benchmark" This reverts commit a2851e2. * optimize tests * remove sync pool * Use absolute path when adding file watcher (istio#40137) * Use absolute path when adding file watcher * add test case which checks for absolute path * add testdata for TestTryAddFileWatcher test * Automator: update common-files@master in istio/istio@master (istio#40228) * Fix new linter version (istio#40204) The new linter fixes `go vet` to properly detect these, so update the code to match * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40233) * cleanup tunneling code (istio#40226) * cleanup tunneling code Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * network test Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Fix export to logic lead to not found logic (istio#40244) * fix export to logic * releasenotes * gen * fix eds comments (istio#40242) * fix eds comments Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * minor comment Signed-off-by: Rama Chavali <rama.rao@salesforce.com> * Automator: update istio/api@master dependency in istio/istio@master (istio#40248) * Automator: update common-files@master in istio/istio@master (istio#40247) * Automator: update istio/pkg@master dependency in istio/istio@master (istio#40251) * Run `make gen` (istio#40255) * Cleanup makefile env vars (istio#40252) * Cleanup some more legacy fields * fix packaging.mk * Automator: update proxy@master in istio/istio@master (istio#40253) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40249) * Automator: update proxy@master in istio/istio@master (istio#40261) * Documentation updates for comments in istio#40255 (istio#40258) * Support pulling multi-arch envoy binaries (istio#39483) * Allow no cpuinfo * allow fail * Bump kind image * debug * workaround env var * fix docker build * log * arch in build * set arch again * fix arch type * more logs * more env * Make single image architecture aware * VM per-arch * Opt out when requiring emulation * Fix jwt server * fmt * Revert env var hacks * cleanup * minor fixes * new release-builder * multi-arch * lint * fix fake v1beta1 conversion (istio#40240) * Remove unused fields of Agent (istio#40262) * Bump master to 1.16 (istio#40263) * Automator: update istio/api@master dependency in istio/istio@master (istio#40271) * Automator: update istio/client-go@master dependency in istio/istio@master (istio#40272) * Add qemulation to release builder (istio#40279) * Add qemulation to release builder VMs currently require emulation to cross compile. We previously just skipped them, but then you cannot simply run `go test ./tests/integration/...` with `gcr.io/istio-testing` since we don't have the VM images. Instead, try emulation. * Use our image * Merge fixes * Support debug building mode (cherry picked from commit 1381ef1) * minor fixes * fix endpoint_builder regression * Add back reachability test cases * fix tests * format * More fixes to align with oss * fix lint * gen * minor fix Signed-off-by: Rama Chavali <rama.rao@salesforce.com> Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> Signed-off-by: Nico Berlee <nico.berlee@on2it.net> Signed-off-by: Tong Li <litong01@us.ibm.com> Signed-off-by: hejianpeng <hejianpeng2@huawei.com> Signed-off-by: yxxhero <aiopsclub@163.com> Signed-off-by: Zhonghu Xu <xuzhonghu@huawei.com> Signed-off-by: Xiao, Ziyang <ziyang.xiao@intel.com> Signed-off-by: Xunzhuo <mixdeers@gmail.com> Signed-off-by: xin.li <xin.li@daocloud.io> Signed-off-by: Eng Zer Jun <engzerjun@gmail.com> Co-authored-by: Istio Automation <istio-testing-bot@google.com> Co-authored-by: Aryan Gupta <garyan@google.com> Co-authored-by: Rama Chavali <rama.rao@salesforce.com> Co-authored-by: Zhonghu Xu <xuzhonghu@huawei.com> Co-authored-by: zirain <hejianpeng2@huawei.com> Co-authored-by: Eric Van Norman <ericvn@us.ibm.com> Co-authored-by: Nick <nick.nellis@solo.io> Co-authored-by: Nathan Mittler <nmittler@gmail.com> Co-authored-by: dwq <41563853+dddddai@users.noreply.github.com> Co-authored-by: Chen Xintong <xintong.chen@intel.com> Co-authored-by: Xiaopeng Han <hanxiaop8@outlook.com> Co-authored-by: Steven Landow <landow@google.com> Co-authored-by: Sergei Gavrilov <12760709+srggavrilov@users.noreply.github.com> Co-authored-by: Sergei Gavrilov <sergei@gavrilov.work> Co-authored-by: Akshay J Nambiar <akshayjnambiar@users.noreply.github.com> Co-authored-by: Karol Szwaj <karol.szwaj@gmail.com> Co-authored-by: Ambor <saltbo@foxmail.com> Co-authored-by: Wongyu Lee <kyu21@outlook.com> Co-authored-by: Lexus Lee <lexuscyborg103@gmail.com> Co-authored-by: sschepens <sebastian.schepens@mercadolibre.com> Co-authored-by: stewartbutler <stewartbutler@google.com> Co-authored-by: Jacek Ewertowski <jewertow@redhat.com> Co-authored-by: Steve Zhang <huailong.zhang@intel.com> Co-authored-by: PlatformLC <lichun823@gmail.com> Co-authored-by: fatedier <fatedier@gmail.com> Co-authored-by: Nico Berlee <nico@notabigtruck.net> Co-authored-by: Douglas Reid <douglas-reid@users.noreply.github.com> Co-authored-by: sergii-ssh <83605538+sergii-ssh@users.noreply.github.com> Co-authored-by: Roman <11049859+RomanSerikov@users.noreply.github.com> Co-authored-by: Tong Li <litong01@users.noreply.github.com> Co-authored-by: Iris <irisdingbj@gmail.com> Co-authored-by: Ying Zhu <ying.zhu@airbnb.com> Co-authored-by: Sam Naser <samnaser@google.com> Co-authored-by: Greg Hanson <gregory.hanson@solo.io> Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com> Co-authored-by: Aditya Prerepa <adiprerepa@gmail.com> Co-authored-by: Ikko Ashimine <eltociear@gmail.com> Co-authored-by: xiaomudk <xiaomudk@gmail.com> Co-authored-by: Zhengzhe Yang <zhengzhey@google.com> Co-authored-by: Yaroslav Zhavoronkov <yaroslav.zh@gmail.com> Co-authored-by: Frank Budinsky <frankb@ca.ibm.com> Co-authored-by: ZiyangXiao <ziyang.xiao@intel.com> Co-authored-by: Xunzhuo <mixdeers@gmail.com> Co-authored-by: my-git9 <xin.li@daocloud.io> Co-authored-by: Eng Zer Jun <engzerjun@gmail.com> Co-authored-by: 白泽 <patrickjiang0530@gmail.com> Co-authored-by: Anubhav <anubhavaeron@gmail.com> Co-authored-by: Kebe <kebe.liu@daocloud.io>
Currently, we do not log incremental pushes at Info level. The intent
behind this is to avoid spam when we have large endpoint churn. However,
because we also do incremental pushes for Full pushes now, we are also
hiding these logs. These logs are both critical to debugging (things
like #39720, etc) and not spammy --
while the
Full=false
pushes may add thousands of messages, this changeonly adds at most 1 log per push/proxy. For these types of pushes I
don't see a benefit to excluding only EDS.
Additionally, fix SDS to correctly assert it is incremental (when it
is).
Please provide a description of this PR: