Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-11227] Try reverting #14295: Moving from vendored gRPC 1.26 to 1.36 #14466

Merged
merged 1 commit into from Apr 8, 2021

Conversation

suztomo
Copy link
Contributor

@suztomo suztomo commented Apr 7, 2021

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang SDK ULR Dataflow Flink Samza Spark Twister2
Go Build Status --- Build Status Build Status --- Build Status ---
Java Build Status Build Status Build Status
Build Status
Build Status
Build Status
Build Status Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status
Build Status Build Status
Build Status
Build Status
Build Status
Python Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status
Build Status
Build Status
--- Build Status ---
XLang Build Status --- Build Status Build Status --- Build Status ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website Whitespace Typescript
Non-portable Build Status
Build Status
Build Status
Build Status
Build Status
Build Status Build Status Build Status Build Status
Portable --- Build Status Build Status --- --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests

See CI.md for more information about GitHub Actions CI.

@suztomo suztomo marked this pull request as draft April 7, 2021 22:44
@codecov
Copy link

codecov bot commented Apr 7, 2021

Codecov Report

Merging #14466 (f1086fc) into master (31efffe) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14466      +/-   ##
==========================================
- Coverage   83.47%   83.46%   -0.01%     
==========================================
  Files         450      450              
  Lines       58832    58832              
==========================================
- Hits        49110    49107       -3     
- Misses       9722     9725       +3     
Impacted Files Coverage Δ
...hon/apache_beam/runners/direct/test_stream_impl.py 91.91% <0.00%> (-2.21%) ⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py 89.54% <0.00%> (-0.16%) ⬇️
...st-suites/tox/py38/build/srcs/sdks/python/setup.py 0.00% <0.00%> (ø)
...teractive/testing/integration/notebook_executor.py 29.59% <0.00%> (ø)
...sdks/python/apache_beam/utils/interactive_utils.py 90.47% <0.00%> (+2.38%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68a8625...f1086fc. Read the comment docs.

Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeouts did begin at https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/3858/ and we did notice them on #14295. I think it is very likely the culprit unfortunately.

@kennknowles
Copy link
Member

There remains a CVE but I do not believe it actually affects Beam in normal situations.

@suztomo
Copy link
Contributor Author

suztomo commented Apr 7, 2021

Right. The gRPC upgrade is the main culprit.

Screen Shot 2021-04-07 at 19 48 19

@suztomo
Copy link
Contributor Author

suztomo commented Apr 7, 2021

Run Java_Examples_Dataflow PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 7, 2021

Java Precommit failed: https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/16971/console

19:45:43 org.apache.beam.runners.flink.FlinkSavepointTest > testSavepointRestoreLegacy FAILED
19:45:43     java.util.concurrent.ExecutionException at FlinkSavepointTest.java:255
19:45:43         Caused by: org.apache.flink.runtime.messages.FlinkJobNotFoundException at Dispatcher.java:894
19:28:32 org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests > testAllAttemptedMetrics FAILED
19:28:32     java.lang.AssertionError at MetricsTest.java:346

@suztomo
Copy link
Contributor Author

suztomo commented Apr 7, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Flink again. https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/3355/console

20:46:52 org.apache.beam.runners.flink.FlinkSavepointTest > testSavepointRestoreLegacy FAILED
20:46:52     java.util.concurrent.ExecutionException at FlinkSavepointTest.java:255
20:46:52         Caused by: org.apache.flink.runtime.messages.FlinkJobNotFoundException at Dispatcher.java:894

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles failed: https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/3357/

java.lang.AssertionError: FileIO.MatchAll/Reshuffle.ViaRandomKey/Values/Values/Map/ParMultiDo(Anonymous).output: 
Expected: iterable with items [<Metadata{resourceId=/tmp/junit1162320239170804688/watch/first, sizeBytes=42, isReadSeekEfficient=true, checksum=null, lastModifiedMillis=0}>, <Metadata{resourceId=/tmp/junit1162320239170804688/watch/second, sizeBytes=37, isReadSeekEfficient=true, checksum=null, lastModifiedMillis=0}>, <Metadata{resourceId=/tmp/junit1162320239170804688/watch/third, sizeBytes=99, isReadSeekEfficient=true, checksum=null, lastModifiedMillis=0}>] in any order
     but: not matched: <Metadata{resourceId=/tmp/junit1162320239170804688/watch/first, sizeBytes=0, isReadSeekEfficient=true, checksum=null, lastModifiedMillis=0}>

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

6 similar comments
@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo suztomo marked this pull request as ready for review April 8, 2021 11:59
@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Run Java PreCommit

@suztomo suztomo merged commit 9601bde into apache:master Apr 8, 2021
suztomo added a commit to suztomo/beam that referenced this pull request Apr 8, 2021
@suztomo
Copy link
Contributor Author

suztomo commented Apr 8, 2021

Memo: next time I need to revert something, use git revert -m 0 <commit>, where 0 is parent number in commit log's "Merge" section.

suztomo added a commit to suztomo/beam that referenced this pull request Apr 9, 2021
suztomo added a commit to suztomo/beam that referenced this pull request Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants