Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync prow-staging with master #15900

Merged
merged 405 commits into from
Jul 29, 2019
Merged

Sync prow-staging with master #15900

merged 405 commits into from
Jul 29, 2019

Conversation

howardjohn
Copy link
Member

We need to do some changes on prow that may break the build. I want to test this out on the prow-staging branch first as a canary, but it needs to be brought up to sync first

esnible and others added 30 commits June 22, 2019 07:35
* Remove test that was moved to istio/pkg repo

* Restore checks of command line typos

* Mock remote test
* Local build

* Local build

* Fix shell

* Add docker_tag

* Fix lint

* Removing some dups

* License

* License

* License

* Push images to docker hub
* Httpbin sample fixes

* fix link
* Add a Mixer integration test for testing K8s integration.

* Add the Mixer test as a presubmit gate.

* Add additional check to ensure that pods stay in ready state.

* Increase number of checks.

* Adding more resiliency to the test.

* Make linter happy.

* Re-fix the problem that is being tested.

On the bright side, the test works.
While I was here, update the common files.
* update istio.io/api

* tidy
Currently we build and push docker images for Istio components and sample apps as
part of our build process. In this PR, we have included a way to enable security
vulnerability scanning of these images using IBM's image scanning tool - ImageScanner
(imagescanner.cloud.ibm.com). The results of the image scans are put under a new folder
'vulnerability_scan_results' which will be available to view later.

Fixes Bug: #13262
* copy code coverage to artifacts directory

For the eng dashboard, we want to be able to scrape code coverage from
GCS. This change adds the coverage file to the artifacts directory,
which in turn should automatically be uploaded to GCS.

* quote variables to fix shellcheck
Also, fixed some errors in our GitHub templates.
* update istio api

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
* Also build distroless images by default. Closes #14696

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Do not build distroless variant by default

* Use BUILD_VARIANTS for docker.save

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Add handling of build variants to release scripts

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Also use distroless variant for release process

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Add missing dependency

Co-authored-by: Ralf Pannemans <ralf.pannemans@sap.com>

* Use correct image name in add_extra_artifacts_to_tar_images.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Fix variant check.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Address comments.

* Fix check for VARIANT_NAME.

Co-authored-by: Julia Plachetka <julia.plachetka@sap.com>

* Refactor and fix TAG issue.

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>

* Fix set_image_vars for distroless

Co-authored-by: Jakob Schmid <jakob.schmid@sap.com>
These are wrappers around the Docker go client library that
simplify the process for the creation of networks, containers, and images.

Not including unit tests here due to the fact that not all CI environments support access to the Docker daemon.

This is split out from #14614
* Add istio state metrics for some of the networking resources

Ref: https://docs.google.com/document/d/1KMUKRMtbpp-K7hvrG5WKBJgoSABydUh4KCHXxKTg8Bk/edit?ts=5ca534e3
Ref: #882

Fix based on feedback
Added test for the metrics

Fix golang error

Updated based on feedback from Oz

Updated based on feedback from Oz

* Fix native error in scenarios_test.go

* fix based on feedback

* fix golang errors

* fix based on feedback

* Fixed based on feedback

* Fix based on feedback

* Fixed golang error

* Fix based on feedback

* Fix scenarios_test.go

* Remove _total from  metric name
* Add junit report for racetest

* Increase rds wait time

Prow is really slow I guess. I was able to reproduce the failure with a
CPU constrained docker container and raising to 15s resolved the issue.

* Fix secretcontroller test race

* use loadint
* Cleanup management of Envoy binaries

The logic flow for linux vs mac is not currently obvious
and without setting GOOS beforehand, you'll end up with
mac binaries in your dockerfiles.

This PR makes more clear where binaries are used. Docker always uses linux, where tests will use the appropriate binary for the os.

* addressing comments.
* Implement /quitquitquit in pilot-agent to support k8s job exit

* lint fix

* add e2e

* fix lint
…r load (#15141)

* Relax keepalive enforcement policy to avoid dropping connections under
load.

* lint

* Add comment.
* Move pkg/features/pilot to pilot/pkg/features.

This cleans up the /pkg package, in preparation of multi-repo.

* Remove naked os.GetEnv usages.

* Fix call sites.

* Change the default values.
* jwt: add sample jwt token for e2e tests.

* add to Makefile and move to tests/common
Adding cross-compile targets for linux and always include linux images in Docker.
* Add retry for token exchange + improve tests

* Move member functions back to secretcache.go + fix lint

* Update security/pkg/nodeagent/cache/secretcache_test.go

Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>

* Fix linter issue

* Change msg log and refactor  getExchangedToken

* lint
* gaurd use_remote_address by feature flag

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* add tests

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* fix comment

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* change the config name

Signed-off-by: Rama Chavali <rama.rao@salesforce.com>

* Resolve rebase conflict
* Fix flaky upgrade test

* Address comments
howardjohn and others added 18 commits July 25, 2019 17:46
* Cleanup and properly document pilot env vars

Currently most of our environment variables are undocumented, and some
also use the wrong types. This makes it very confusing because you can
set FOO=false and it actually turns on FOO. This change cleans up these
cases, and adds documentation to most of the variables used in pilot.

* Fix errors
* upgrade kiali

Signed-off-by: clyang82 <clyang@cn.ibm.com>

* upgrade kiali to 1.1
Signed-off-by: clyang82 <clyang@cn.ibm.com>
* Change hostname resolution to follow Sidecar

See the design doc for more details
https://docs.google.com/document/d/15-PU9O22Pb0qTzCfwK2hjwNTcb-CqFuw8JnIgwK3EIM/

This PR changes the behavior of Pilot when the same hostname is found in
multiple namespaces (due to ServiceEntries). Previously, the behavior
was undefined -- in some cases we selected an arbitrary service while in
others we selected all services. The new behavior will always select a
single namespace for a given hostname. If a hostname exists in multiple
namespaces, one will be determined by the Sidecar scope. If the sidecar
imports multiple namespaces with the same hostname, an arbitrary one
will be chosen, favoring the proxies namespace if possible.

* Get rid of dummy sidecar

* Make InstancesByPort take a Service instead of host

This is primarily meant to enable
#13631, which will require the full
Service

* Make instances by port use proper namespace selection

* Clean up dead code

* Fix rebase errors
* Make listener_test use open port

* Make appprobe test poll until ready

* Make pilot-agent role tests not depend on global state

* Enable skipped test with resolved issue
* Integ test for webhook behavior when scaling galley

* integ test to verify webhook config deletion when galley uninstalled

* PR review fixups

* Move galley webhook tests into their own suite

* Use subtests

* Increase delay to wait for webhook reconciliation to act
#15672)

* Try fetch secret directly in case a secret is requested but cache doesn't have it somehow

* Don't put secret directly fetched from API call to cache
* add test for direct HTTP response

- status code
- body setting
- header manipulation

* gofmt changes
* Add POD_IP match to prevent infinite traffic loops

* fix lint

* Disable for cloudfoundry test

* Precompute filter

* Add integration test
* Also report number of virtual services known to pilot.

Only count virtual services for ISTIO_MESH_GATEWAY.

Fixes #14932

* Move gauge for total virtual service count to initVirtualServices
* Match http/network filters

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* insert before or after

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* split into smaller files

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* tests and lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* test

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lots of tests

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint1

* test fixes

* lint

* disable until resolution

* lint

* update gogo

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* unskip tests

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* integration test

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* config fixes

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* bug fix

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>

* lint

Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
* detect arm architecture

* remove hardcoded value

* only normalize arm with variant
* Support kube-uninject for istioctl command

Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com>

* Address review comments

* avoid duplicated method

Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com>

* add more cases: handle enable-core-dump container/dnsConfig/annotations

* add sidecar.istio.io/inject:false always

* avoid panics

Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com>

* correct the comment for exported method
Signed-off-by: Chun Lin Yang <clyang@cn.ibm.com>
@istio-testing istio-testing added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jul 29, 2019
@istio-testing
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: howardjohn
To complete the pull request process, please assign geeknoid, utka
You can assign the PR to them by writing /assign @geeknoid @utka in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov
Copy link

codecov bot commented Jul 29, 2019

Codecov Report

Merging #15900 into prow-staging will increase coverage by 1%.
The diff coverage is 69%.

@@              Coverage Diff               @@
##           prow-staging   #15900    +/-   ##
==============================================
+ Coverage            77%      77%    +1%     
==============================================
  Files               497      501     +4     
  Lines             48785    49291   +506     
==============================================
+ Hits              37354    37836   +482     
- Misses             9489     9491     +2     
- Partials           1942     1964    +22
Impacted Files Coverage Δ
mixer/pkg/lang/cel/provider.go 75% <ø> (ø) ⬆️
pilot/pkg/config/kube/crd/types.go 9% <ø> (ø) ⬆️
mixer/pkg/il/types.go 100% <ø> (+19%) ⬆️
mixer/adapter/kubernetesenv/kubernetesenv.go 90% <ø> (ø) ⬆️
mixer/pkg/runtime/lang/lang.go 69% <ø> (ø) ⬆️
mixer/pkg/protobuf/yaml/dynamic/auth.go 83% <ø> (ø) ⬆️
istioctl/cmd/register.go 37% <0%> (ø) ⬆️
istioctl/cmd/auth.go 54% <0%> (ø) ⬆️
istioctl/pkg/kubernetes/client.go 0% <0%> (ø) ⬆️
pilot/pkg/config/kube/crd/controller/controller.go 2% <0%> (ø)
... and 102 more

@googlebot
Copy link
Collaborator

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@googlebot googlebot added the cla: no Set by the Google CLA bot to indicate the author of a PR has not signed the Google CLA. label Jul 29, 2019
Previously the done channel was unbuffered. This means that if a timeout
occured, there would be nothing trying to read from `done`, which would
cause it to block indefinitely. Because of this, every timeout resulted
in a goroutine to be leaked. Now it is buffered so the send can occur
even after the timer completes.
@istio-testing
Copy link
Collaborator

The following users are mentioned in OWNERS file(s) but are not members of the istio org.

  • ijsnellf
    • tests/e2e/tests/pilot/OWNERS

@istio-testing
Copy link
Collaborator

@howardjohn: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
prow/release-test.sh 2be5863 link /test release-test-prow-staging

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid geeknoid merged commit ff1f032 into prow-staging Jul 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: no Set by the Google CLA bot to indicate the author of a PR has not signed the Google CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet