Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: use multiple stage #13557

Merged
merged 9 commits into from
Oct 15, 2020
Merged

ci: use multiple stage #13557

merged 9 commits into from
Oct 15, 2020

Conversation

lizan
Copy link
Member

@lizan lizan commented Oct 14, 2020

To be merged along with #13550. Mostly cosmetic change to use multiple stage so it is easier to see / retest in AZP UI. Also made docs as part of precheck so it blocks CI.

Signed-off-by: Lizan Zhou lizan@tetrate.io

@lizan
ci: use azp for api and go-control-plane sync
79c2292 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan
use main branch
0b5bceb 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan
comment
b459461 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan
ci: use multiple stage
27bf2d7 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@phlax
Copy link
Member

phlax commented Oct 14, 2020

@lizan one thing i just realized is that some of the docs/READMEs/scripts refer to CircleCI - eg

https://github.com/envoyproxy/envoy/blob/master/docs/README.md

doing git grep -i circleci it seems like there are a few more

@lizan
Copy link
Member Author

lizan commented Oct 14, 2020

@phlax #13558

Base automatically changed from azp_api to master October 14, 2020 15:16
@mattklein123
Copy link
Member

OK when this merges I will need to change all the required checks. Can you merge main?

/wait

@lizan
cosmetic
f3392df 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan
Copy link
Member Author

lizan commented Oct 14, 2020

@mattklein123 this is good to go, but lmk if you want do any cosmetic change (since it affects required checks name).

mattklein123
mattklein123 reviewed Oct 14, 2020
View reviewed changes
.azure-pipelines/pipelines.yml
artifactSuffix: ".arm64"
bazelBuildExtraOptions: "--sandbox_base=/tmp/sandbox_base"

- stage: check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to remove "check" from the job name so they match what we have today? I don't think it adds much and it would be nice to not have to change the required jobs?

/wait-any

@mattklein123
Copy link
Member

Yeah I think my main comment is can we avoid changing the job names or is that impossible?

If we have to change the job names, can we satisfy my OCD by making them all lower case or all properly capitalized? Right now it's all over the place. Thank you!

/wait

@lizan
Copy link
Member Author

lizan commented Oct 15, 2020 •
edited
Loading

Yeah I think my main comment is can we avoid changing the job names or is that impossible?

That will mess up the stage view if we made empty:
image

If we have to change the job names, can we satisfy my OCD by making them all lower case or all properly capitalized? Right now it's all over the place. Thank you!

totally agreed and that it will satisfy my OCD as well 😂

@lizan
OCD
80078f0 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@mattklein123
Copy link
Member

Thanks this LGTM. Do we want to sort out why OSX is still perma failing it seems before merging this?

/wait-any

@mattklein123 mattklein123 merged commit 73d78f8 into master Oct 15, 2020
@mattklein123 mattklein123 deleted the azp_stage branch October 15, 2020 15:15
mpuncel added a commit to mpuncel/envoy that referenced this pull request Oct 15, 2020
@mpuncel
Merge branch 'master' into mpuncel/secret-hc-sequence
76014af 
* master:
  ci: use multiple stage (envoyproxy#13557)
  tls: update BoringSSL to 2192bbc8 (4240). (envoyproxy#13567)
  fix macos v8 build (envoyproxy#13572)
  Fixed Health Check Fuzz corpus syntax (envoyproxy#13576)
  ci: Remove shellcheck diff (envoyproxy#13560)
  ci: Increate brew retry interval (envoyproxy#13565)
  dependencies: fix some of the fallout from Wasm merge. (envoyproxy#13569)
  hds: add support for delta updates in specifier (envoyproxy#13067)
  ci: workaround for actions/runner-images#1811 (envoyproxy#13577)
  ratelimit: be able to disable x-envoy-ratelimited response header sent (envoyproxy#13270)
  Update opencensus library (envoyproxy#13549)
  ci: use azp for api and go-control-plane sync (envoyproxy#13550)
  docs: Remove/make generic lyft references in docs (envoyproxy#13559)
  check_format: adding 2 more release note checks (envoyproxy#13444)
  [Wasm] Add cluster metadata fallback and upstream host metadata (envoyproxy#13477)
  [fuzz] Added validation for secrets (envoyproxy#13543)
  Add Platform Specific Feature guidance to PR template (envoyproxy#13547)

Signed-off-by: Michael Puncel <mpuncel@squareup.com>
lizan added a commit to lizan/envoy that referenced this pull request Oct 21, 2020
@lizan
ci: use multiple stage (envoyproxy#13557)
b25abcb 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan lizan mentioned this pull request Oct 21, 2020
Merged
lizan added a commit that referenced this pull request Oct 21, 2020
@lizan @asraa
1.16: CI fixes backport (#13665)
8620920 
Backport following commits to 1.16:
748b2ab (mac ci: try ignoring update failure (#13658), 2020-10-20)
f95f539 (ci: various improvements (#13660), 2020-10-20)
6481a27 (ci: stop building alpine-debug images in favor of ubuntu-based debug image (#13598), 2020-10-15)
73d78f8 (ci: use multiple stage (#13557), 2020-10-15)
b7a4756 (ci: use azp for api and go-control-plane sync (#13550), 2020-10-14)
876a6bb (ci use azp to sync filter example (#13501), 2020-10-12)
a0f31ee (ci: use azp to generate docs (#13481), 2020-10-12)
7e5d854 (ci: bring BAZEL_BUILD_OPTIONS back for format and docs (#13480), 2020-10-11)
7af2b2b (ci: use same flaky test process script on macOS (#13485), 2020-10-10)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: asraa <asraa@google.com>
lizan added a commit to lizan/envoy that referenced this pull request Oct 22, 2020
@lizan
ci: use multiple stage (envoyproxy#13557)
fdf2b83 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan lizan mentioned this pull request Oct 22, 2020
Merged
lizan added a commit to lizan/envoy that referenced this pull request Oct 22, 2020
@lizan
ci: use azp to generate docs (envoyproxy#13481)
3d37587 
Signed-off-by: Lizan Zhou <lizan@tetrate.io>

ci use azp to sync filter example (envoyproxy#13501)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

ci: use azp for api and go-control-plane sync (envoyproxy#13550)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

ci: use multiple stage (envoyproxy#13557)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

ci: various improvements (envoyproxy#13660)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

mac ci: try ignoring update failure (envoyproxy#13658)

Signed-off-by: Asra Ali <asraa@google.com>

revert run_docker.sh

fix

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

fix

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
@lizan lizan mentioned this pull request Oct 22, 2020
Merged
lizan added a commit that referenced this pull request Oct 22, 2020
@lizan @asraa
1.15: CI fixes backport (#13697)
2b9eeed 
Backport following commits to 1.15:
748b2ab (mac ci: try ignoring update failure (#13658), 2020-10-20)
f95f539 (ci: various improvements (#13660), 2020-10-20)
73d78f8 (ci: use multiple stage (#13557), 2020-10-15)
b7a4756 (ci: use azp for api and go-control-plane sync (#13550), 2020-10-14)
876a6bb (ci use azp to sync filter example (#13501), 2020-10-12)
a0f31ee (ci: use azp to generate docs (#13481), 2020-10-12)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: asraa <asraa@google.com>
lizan added a commit that referenced this pull request Nov 1, 2020
@lizan
1.14: CI fix backport (#13703)
a06598b 
Backport following commits to 1.14:
748b2ab (mac ci: try ignoring update failure (#13658), 2020-10-20)
f95f539 (ci: various improvements (#13660), 2020-10-20)
73d78f8 (ci: use multiple stage (#13557), 2020-10-15)
b7a4756 (ci: use azp for api and go-control-plane sync (#13550), 2020-10-14)
876a6bb (ci use azp to sync filter example (#13501), 2020-10-12)
a0f31ee (ci: use azp to generate docs (#13481), 2020-10-12)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
lizan added a commit that referenced this pull request Nov 13, 2020
@lizan
1.13: backport ci fixes (#13958)
14863ad 
    748b2ab (mac ci: try ignoring update failure (#13658), 2020-10-20)
    f95f539 (ci: various improvements (#13660), 2020-10-20)
    73d78f8 (ci: use multiple stage (#13557), 2020-10-15)
    b7a4756 (ci: use azp for api and go-control-plane sync (#13550), 2020-10-14)
    876a6bb (ci use azp to sync filter example (#13501), 2020-10-12)
    a0f31ee (ci: use azp to generate docs (#13481), 2020-10-12)
    a103da9 (test: fixing an integration test flake (#11845), 2020-07-01)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
istio-testing pushed a commit to istio/envoy that referenced this pull request Jan 8, 2021
@bianpengyuan @PiotrSikora
@antoniovicente @mattklein123 @lizan @cpakulski @asraa @jmarantz @Shikugawa @alyssawilk
[release-1.7] Backport Envoy 1.15 fixes (#290)
a0eff54 
* docs: kick-off 1.15.1 release. (envoyproxy#12166)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* tls: update BoringSSL-FIPS to 20190808. (envoyproxy#12170)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* test: Exclude wasm_vm_test from CI by making it a "manual" test. (#207)

The wee v8 build times out in CI under --config=asan because the machine the job is scheduled on is too small.

Signed-off-by: Antonio Vicente <avd@google.com>

* [v1.15] http: header map security fixes for duplicate headers (#197) (#200)

Previously header matching did not match on all headers for
non-inline headers. This patch changes the default behavior to
always logically match on all headers. Multiple individual
headers will be logically concatenated with ',' similar to what
is done with inline headers. This makes the behavior effectively
consistent. This behavior can be temporary reverted by setting
the runtime value "envoy.reloadable_features.header_match_on_all_headers"
to "false".

Targeted fixes have been additionally performed on the following
extensions which make them consider all duplicate headers by default as
a comma concatenated list:
1) Any extension using CEL matching on headers.
2) The header to metadata filter.
3) The JWT filter.
4) The Lua filter.
Like primary header matching used in routing, RBAC, etc. this behavior
can be disabled by setting the runtime value
"envoy.reloadable_features.header_match_on_all_headers" to false.

Finally, the setCopy() header map API previously only set the first
header in the case of duplicate non-inline headers. setCopy() now
behaves similiarly to the other set*() APIs and replaces all found
headers with a single value. This may have had security implications
in the extauth filter which uses this API. This behavior can be disabled
by setting the runtime value
"envoy.reloadable_features.http_set_copy_replace_all_headers" to false.

Fixes https://github.com/envoyproxy/envoy-setec/issues/188

Signed-off-by: Matt Klein <mklein@lyft.com>

* backport to v1.15: Fix Kafka Repository Location (#223)

Update mirror used to fetch kafka dependency to a valid, working mirror.

Based on envoyproxy#13025
Resolves envoyproxy#13011

Signed-off-by: Antonio Vicente <avd@google.com>

* release: cutting 1.15.1 (#217)

Signed-off-by: Antonio Vicente <avd@google.com>

* docs: Fix release notes for v1.15.1 release. (envoyproxy#13318)

Signed-off-by: Antonio Vicente <avd@google.com>

* Backport flaky test and tsan fixes to releases/v1.15 branch (envoyproxy#13337)

* hds: fix integration test flakes (envoyproxy#12214)

Part of envoyproxy#12184

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Antonio Vicente <avd@google.com>

* Switch to a tsan-instrumented libc++ for tsan tests (envoyproxy#12134)

This fixes envoyproxy#9784 and re-enables vhds_integration_test

Risk Level: Low, but will most likely increase memory usage

Signed-off-by: Dmitri Dolguikh <ddolguik@redhat.com>

Signed-off-by: Antonio Vicente <avd@google.com>

* test: shard hds_integration_test (envoyproxy#12482)

This should avoid TSAN timeout flakes.

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Antonio Vicente <avd@google.com>

* test: shard http2_integration_test (envoyproxy#11939)

This should mitigate TSAN timeout.

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Signed-off-by: Antonio Vicente <avd@google.com>

* test: fix http2_integration_test flake (envoyproxy#12450)

Fixes envoyproxy#12442

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Antonio Vicente <avd@google.com>

* Kick CI

Signed-off-by: Antonio Vicente <avd@google.com>

Co-authored-by: Matt Klein <mklein@lyft.com>
Co-authored-by: Dmitri Dolguikh <ddolguik@redhat.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>

* docs: kick off v1.15.3-dev (envoyproxy#13695)

Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* 1.15: CI fixes backport (envoyproxy#13697)

Backport following commits to 1.15:
748b2ab (mac ci: try ignoring update failure (envoyproxy#13658), 2020-10-20)
f95f539 (ci: various improvements (envoyproxy#13660), 2020-10-20)
73d78f8 (ci: use multiple stage (envoyproxy#13557), 2020-10-15)
b7a4756 (ci: use azp for api and go-control-plane sync (envoyproxy#13550), 2020-10-14)
876a6bb (ci use azp to sync filter example (envoyproxy#13501), 2020-10-12)
a0f31ee (ci: use azp to generate docs (envoyproxy#13481), 2020-10-12)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: asraa <asraa@google.com>

* 1.15: fix CI script (envoyproxy#13724)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* Prevent SEGFAULT when disabling listener (envoyproxy#13515) (envoyproxy#13903)

This prevents the stop_listening overload action from causing
segmentation faults that can occur if the action is enabled after the
listener has already shut down.

Signed-off-by: Alex Konradi <akonradi@google.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* proxy protocol: set downstreamRemoteAddress on StreamInfo (envoyproxy#14131) (envoyproxy#14169)

This fixes a regression which resulted in the downstreamRemoteAddress
on the StreamInfo for a connection not having the address supplied by
the proxy protocol filter, but instead having the address of the
directly connected peer.

This issue does not affect HTTP filters.

Fixes envoyproxy#14087

Signed-off-by: Greg Greenway <ggreenway@apple.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* ci: temproray disable vhds_integration_test in TSAN (envoyproxy#12067) (envoyproxy#14217)

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* tcmalloc changed and the data coming out of tcmalloc::MallocExtension::GetNumericProperty("generic.current_allocated_bytes") (envoyproxy#14165)

Commit Message: tcmalloc changed and the data coming out of tcmalloc::MallocExtension::GetNumericProperty("generic.current_allocated_bytes") no longer appears to be deterministic, even in unthreaded tests. So disable exact mem checks till we sort that out
Additional Description:
Risk Level: low
Testing: just thread_local_store_test
Docs Changes: n/a
Release Notes: n/a

no longer appears to be deterministic, even in unthreaded tests. So disable exact mem checks till we sort that out

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

Co-authored-by: Joshua Marantz <jmarantz@google.com>

* backport to v1.15: connection: Remember transport socket read resumption requests and replay them when re-enabling read. (envoyproxy#13772) (envoyproxy#14173)

* connection: Remember transport socket read resumption requests and replay them when re-enabling read. (envoyproxy#13772)

Fixes SslSocket read resumption after readDisable when processing the SSL record that contains the last bytes of the HTTP message

Signed-off-by: Antonio Vicente <avd@google.com>

* backport to 1.15: udp: properly handle truncated/dropped datagrams (envoyproxy#14122) (envoyproxy#14166)

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Co-authored-by: Matt Klein <mklein@lyft.com>
Co-authored-by: Christoph Pakulski <christoph@tetrate.io>

* backport to 1.15: vrp: allow supervisord to open its log file (envoyproxy#14066) (envoyproxy#14280)

Commit Message: Allow supervisord to open its log file
Additional Description:
Change the default location of the log file and give supervisord
permissions to write to it.

Risk Level: low
Testing: built image locally
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

Signed-off-by: Alex Konradi <akonradi@google.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* rel 1.15: close release 1.15.3 (envoyproxy#14303)

Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* Kick off rel 1.15.4. (envoyproxy#14323)

Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

* backport to 1.15: http: fix datadog and squash handling of responses without body (envoyproxy#13328) (envoyproxy#14458)

Commit Message: Fixing bugs in datadog and sqaush where unexpected bodyless responses would crash Envoy
Risk Level: low
Testing: new unit tests, updated certs
Docs Changes: n/a
Release Notes: inline
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Co-authored-by: alyssawilk <alyssar@chromium.org>

* backport 1.15: http: fixing a bug with IPv6 hosts (envoyproxy#14273)

Fixing a bug where HTTP parser offsets for IPv6 hosts did not include [] and Envoy assumed it did.
This results in mis-parsing addresses for IPv6 CONNECT requests and IPv6 hosts in fully URLs over HTTP/1.1

Risk Level: low
Testing: new unit, integration tests
Docs Changes: n/a
Release Notes: inline

Signed-off-by: Shikugawa <rei@tetrate.io>
Co-authored-by: alyssawilk <alyssar@chromium.org>

* backport to 1.15: tls: fix detection of the upstream connection close event. (envoyproxy#13858) (envoyproxy#14568)

Fixes envoyproxy#13856.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

Co-authored-by: Piotr Sikora <piotrsikora@google.com>
Co-authored-by: antonio <avd@google.com>
Co-authored-by: Matt Klein <mklein@lyft.com>
Co-authored-by: Dmitri Dolguikh <ddolguik@redhat.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: Christoph Pakulski <christoph@tetrate.io>
Co-authored-by: asraa <asraa@google.com>
Co-authored-by: Joshua Marantz <jmarantz@google.com>
Co-authored-by: Rei Shimizu <Shikugawa@gmail.com>
Co-authored-by: alyssawilk <alyssar@chromium.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattklein123 mattklein123 mattklein123 approved these changes

Assignees

@mattklein123 mattklein123

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lizan @phlax @mattklein123