* Remove test that was moved to istio/pkg repo

* Restore checks of command line typos

* Mock remote test

* Local build

* Local build

* Fix shell

* Add docker_tag

* Fix lint

* Removing some dups

* License

* License

* License

* Push images to docker hub

* Httpbin sample fixes

* fix link

* Add a Mixer integration test for testing K8s integration.

* Add the Mixer test as a presubmit gate.

* Add additional check to ensure that pods stay in ready state.

* Increase number of checks.

* Adding more resiliency to the test.

* Make linter happy.

* Re-fix the problem that is being tested.

On the bright side, the test works.

While I was here, update the common files.

* update istio.io/api

* tidy

Currently we build and push docker images for Istio components and sample apps as
part of our build process. In this PR, we have included a way to enable security
vulnerability scanning of these images using IBM's image scanning tool - ImageScanner
(imagescanner.cloud.ibm.com). The results of the image scans are put under a new folder
'vulnerability_scan_results' which will be available to view later.

Fixes Bug: #13262

* copy code coverage to artifacts directory

For the eng dashboard, we want to be able to scrape code coverage from
GCS. This change adds the coverage file to the artifacts directory,
which in turn should automatically be uploaded to GCS.

* quote variables to fix shellcheck

Also, fixed some errors in our GitHub templates.

* update istio api

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* Also build distroless images by default. Closes #14696

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Do not build distroless variant by default

* Use BUILD_VARIANTS for docker.save

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Add handling of build variants to release scripts

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Also use distroless variant for release process

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Add missing dependency

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Use correct image name in add_extra_artifacts_to_tar_images.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Fix variant check.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Address comments.

* Fix check for VARIANT_NAME.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Refactor and fix TAG issue.

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Fix set_image_vars for distroless

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

These are wrappers around the Docker go client library that
simplify the process for the creation of networks, containers, and images.

Not including unit tests here due to the fact that not all CI environments support access to the Docker daemon.

This is split out from #14614

* Add istio state metrics for some of the networking resources

Ref: https://docs.google.com/document/d/1KMUKRMtbpp-K7hvrG5WKBJgoSABydUh4KCHXxKTg8Bk/edit?ts=5ca534e3
Ref: #882

Fix based on feedback
Added test for the metrics

Fix golang error

Updated based on feedback from Oz

Updated based on feedback from Oz

* Fix native error in scenarios_test.go

* fix based on feedback

* fix golang errors

* fix based on feedback

* Fixed based on feedback

* Fix based on feedback

* Fixed golang error

* Fix based on feedback

* Fix scenarios_test.go

* Remove _total from  metric name

* Add junit report for racetest

* Increase rds wait time

Prow is really slow I guess. I was able to reproduce the failure with a
CPU constrained docker container and raising to 15s resolved the issue.

* Fix secretcontroller test race

* use loadint

* Cleanup management of Envoy binaries

The logic flow for linux vs mac is not currently obvious
and without setting GOOS beforehand, you'll end up with
mac binaries in your dockerfiles.

This PR makes more clear where binaries are used. Docker always uses linux, where tests will use the appropriate binary for the os.

* addressing comments.

* Implement /quitquitquit in pilot-agent to support k8s job exit

* lint fix

* add e2e

* fix lint

Fixes #15150

…r load (#15141)

* Relax keepalive enforcement policy to avoid dropping connections under
load.

* lint

* Add comment.

* Move pkg/features/pilot to pilot/pkg/features.

This cleans up the /pkg package, in preparation of multi-repo.

* Remove naked os.GetEnv usages.

* Fix call sites.

* Change the default values.

* jwt: add sample jwt token for e2e tests.

* add to Makefile and move to tests/common

Adding cross-compile targets for linux and always include linux images in Docker.

* Add retry for token exchange + improve tests

* Move member functions back to secretcache.go + fix lint

* Update security/pkg/nodeagent/cache/secretcache_test.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* Fix linter issue

* Change msg log and refactor  getExchangedToken

* lint

* gaurd use_remote_address by feature flag

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add tests

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* fix comment

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* change the config name

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* Resolve rebase conflict

* Fix flaky upgrade test

* Address comments

* Run simple test with distroless images

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Add comment to set_image_vars

* Use e2e-simpleTest.sh to run distroless test

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Use mini-kube for distroless tests

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Add distroless test to circle-ci

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Make tests for distroless work

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Migrate e2e_simple distroless test from circle ci to prow

Co-authored-by: Ulrich Kramer <u.kramer@sap.com>

* Rename test_tag to app_tag and test_hub to app_hub

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* Fix building docker images for distroless variant

Co-authored-by: Ulrich Kramer <u.kramer@sap.com>

* Update to go-control-plane v0.8.1

* update rbac from v2alpha --> v2

* Correct the nodeport for 80

Signed-off-by: clyang82 <clyang@cn.ibm.com>

address comments

* drop 0 before var

This is split out from #14614

Also moving the existing Dockerfile.app to a more sane location.

This is split out from #14614 and is needed for dynamically building Docker images for the Echo component.

* refactor(pilot metrics): convert to OpenCensus from Prometheus

* fix(lint issues)

* fix(e2e_test): remove timeout queries from dashboard e2e test

* fix(racetest failure): push_context.go

* fix(data race): workaround freeze access to global in listener.go

* fix(metrics): context init after tag creation

* refactor(monitoring): add monitoring pkg to improve OC ux

* cleanup(monitoring package): refactor monitoring API

* fix(docs): add copyright to file

* fix(data race): copy tags slice before append in WithTags and remove MetricOpts

* refactor(monitoring pkg): adds Tag and TagValue types

* refactor(monitoring): WithTags -> With

* fix(metric): k8sErrors had wrong tag

…fixes (#15138)

* Add test target for new installer and fix issues

This PR accomplishes two things:
* Gets the same tests running on istio/installer running on istio/istio
so changes in istio/istio are less likely to come in that break the
installer repo
* Makes some minor modifications to get the tests passing

In the long term, we will have the test framework actually do the
installer, but there are still some open questions on how that will be
done. In the short term getting this test enabled will help the
installer progress.

* Run format

Signed-off-by: clyang82 <clyang@cn.ibm.com>

* Local build

* Local build

* Fix shell

* Add docker_tag

* Fix lint

* Removing some dups

* License

* License

* License

* Push images to docker hub

* Local 5

* branch

* Finish local build with creating helm charts

* shellcheck

* fixes

* Fixes

* right code

* apps/v1 for tests

* fix typo

* Refactor server.Args out to its own package.

- Move all settings into Args struct.
- Make handling of defaults in command-line uniform.
- Minor code analysis fixes.

* Add missing edit.

* Fix lint errors

* fix server_test.go

* suppress lint check

* Fix imports.

* Align service resolution with Pilot's validation

- This change makes service resolution more aligned with Pilot validation
during the converson of an instance to serviceEntry in Galley

* Use external service endpoints if K8s endpoints are empty

* Fix handler test

In order to check labels attached to a namespace, Citadel must have k8s
cluster permissions to read namespace resources. In the current
chart defaults, Citadel is not granted *any* namespace related
permissions, despite making a namespace read in the istioEnabledObject
function.

Fixes #15192

This is split out from #14614 and is needed for dynamically building docker images for the Echo component.

Overview:

- Added parsing for istio.deps
- Moved test utilities for downloading and extracting tar.gz files to pkg/test/util
- Added utility for downloading the current release of the envoy linux binary.

* add yamllint check into prow

* fix encoding issue for grafana/values.yaml

* fix encoding issue for values.yaml

Found those while looking at metrics endpoint.

* Replace zipkin test using new integration test framework

Fix golang errors

Add license headers

Fixed based on feedback

Change calls to product page to 1 from 10 for client tracing

Add ability to query for client trace id

Fix golang error

* Fix golang error

* read rootcert from configured ns to connect to citadel

* fix error nodeagent->citadel in configured ns

* rename

* desc

…15254)

…14985)

* Build distroless proxy_init image with go version of istio-iptables

Co-authored-by: Philipp Stehle <philipp.stehle@sap.com>

* Renamed istio-iptables.sh to istio-iptables for distroless

This changes the native Echo component to use Docker to address many limitations of the existing native implementation.

The native environment now creates its own Docker network, to which all Echo instance containers are attached. Since they're all on the same Docker network, they have built-in reachability.

The Echo+sidecar Dockerfile is a blend of the existing Dockerfile and the setup used for raw VMs. Going forward, I expect we'll build on this and create a common Dockerfile to support non-k8s use cases.

Limitations:

- This PR does not yet enable mTLS. The connection to Pilot has been switched to TLS, however the mesh config is not truly configurable yet in the native environment. We need to re-think how helm settings might be set for the native environment.

- The Docker images are currently built every time the tests are run. While this guarantees that we're running with the latest, it adds a bit of time to the duration of the test run (e.g. the sidecar image takes ~30s to build). Need to investigate ways of detecting when the image needs to be built to avoid this overhead, as well as the additional storage required for several duplicate Docker images.

- Currently using a copy of Go code for performing the untar of the downloaded Envoy. Should investigate alternatives or write our own version.

- Not currently using the node agent. Once we have a native citadel, we can consider enabling it.

Fixes #13177

…15262)

* Fixes #15250. Add support for HTTP1.0 for sidecar inbound listeners

* Adapt to move of pilot.HTTP10 to features.HTTP10

* Turn everything but cloudfoundry circle test

* Turn on nightly builds for some tests

* Turn off noauth

* jwt: add metric for network fetch

* fix metrix name

* fix lint

* update to use monitoring pkg

* Update pilot/pkg/model/jwks_resolver_test.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* jwt: update to use unified jwt token in e2e tests

* fix test

…15266)

* update

* check sds stats

* check sds stats

* update

* update

* format

* format

* revise

* revise

* fix the handling of empty secret

* update test

* format

* revise

* revise

* add sds stats inclustion into bootstrap config

* revise

* update test

* make toProto nonstrict when converting values

* Looks like we also test this in converter, updating there too

* add integ case for virtualservice with and without extra unsupported params

#15203)

* Add a process package to pkg/server for tracking sub-component.

- process.Host is a basic container of multiple sub-components.
- process.Component is an interface to be implemented by sub-components.

* Fix lint errors.

* Accommodate CR feedback.

This was broken by #14985

* Added section for configuring additional environment variable configs for grafana to override certain grafana.ini settings like adding SMTP settings.

* Removed redundance templating code. Moved grafana env to demo config file.

* Removed demo grafana config file. Moved the env and envSecrets values to grafana chart values file with comments on how to use.

A recent change (#14944) modified this logic which was causing it to no
longer actually poll for the ingress ip -- it would return "" and use
that rather than erroring properly.

With this change we will continue to retry if we don't get a valid IP.

* fix JWT token

* use groups instead of group

* one more file

* fix format

Currently each component has 10m cpu requests and an HPA that scales at
80% CPU usage meaning they will immediately scale up.

This turns off the HPA for the demo.

Previously we ran a new process for every yaml file. yamllint can just
take in a list of files to run, and handles this much more efficiently.
On my machine, this broguht runtime from 75s to 15s.

* Update KinD e2e test suite

* Set imagePullPolciy=Never per KinD docs
* Fix image loading -- before it was only loading the last image not all
of them
* Set ARTIFACTS_DIR so the script can be run locally

* Fix shellcheck

* Fix accesslog integration tests

The log command was only getting the last 10 results, so it was missing
the logs it was looking for. Additionally, cleaned up the error message
to expose what log was actually missing rather than "one of these 3 logs
was missing"

* Get all logs

In the flakes, test fails as if the ratelimit rule has not been applied. Adding a retry, so as to give change for the rules to sync in properly

Format the files

Send some initial traffic to boot up the system. This had helped ratelimit tests to be more stable in old framework

Dont change defaultlessthanoverride test as it is not flaky in postsubmits

* add cert expir time

* add debug log

* add debug log

* remove debug log

* revise

* revise

* format

* revise

* lint and format

* revise

* fix tests

* lint

The push command also does some installgen stuff which wastes 5+
minutes.

* add myself to owner file

* update

Fixes: #14237.

Signed-off-by: Jason Clark <jason.clark@ibm.com>

These scripts are no longer needed; the prow just just directly call the
targets.

…ure outbound traffic policy individually per cluster instead of on a cluster global scope (#15257)

* code clean

* fix ut

* decouple webhook configuration reconciliation

Signed-off-by: clyang82 <clyang@cn.ibm.com>

* remove unused param

Signed-off-by: clyang82 <clyang@cn.ibm.com>

* Address review comments