Skip to content

[BEAM-7814] Make BigqueryMatcher wait for results.#9154

Merged
pabloem merged 2 commits intoapache:masterfrom
udim:bq-test-client-wait
Jul 29, 2019
Merged

[BEAM-7814] Make BigqueryMatcher wait for results.#9154
pabloem merged 2 commits intoapache:masterfrom
udim:bq-test-client-wait

Conversation

@udim
Copy link
Member

@udim udim commented Jul 25, 2019

Also merges BigqueryMatcher and BigqueryFullResultMatcher shared code.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status --- --- Build Status

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

Also merges BigqueryMatcher and BigqueryFullResultMatcher shared code.
@udim
Copy link
Member Author

udim commented Jul 25, 2019

run python 2 postcommit

@udim
Copy link
Member Author

udim commented Jul 25, 2019

R: @pabloem

@udim
Copy link
Member Author

udim commented Jul 25, 2019

Run Python_PVR_Flink PreCommit

@pabloem
Copy link
Member

pabloem commented Jul 25, 2019

Run Python PreCommit

@pabloem
Copy link
Member

pabloem commented Jul 25, 2019

Run Python 2 PostCommit

@udim
Copy link
Member Author

udim commented Jul 25, 2019

I have a bug or two:

14:09:37 ======================================================================
14:09:37 ERROR: test__get_query_result_timeout (apache_beam.io.gcp.tests.bigquery_matcher_test.BigqueryFullResultStreamingMatcher)
14:09:37 ----------------------------------------------------------------------
14:09:37 Traceback (most recent call last):
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/target/.tox-py37-gcp/py37-gcp/lib/python3.7/site-packages/mock/mock.py", line 1305, in patched
14:09:37     return func(*args, **keywargs)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher_test.py", line 132, in test__get_query_result_timeout
14:09:37     matcher._get_query_result(None)
14:09:37 TypeError: _get_query_result() takes 1 positional argument but 2 were given
14:09:37 
14:09:37 ======================================================================
14:09:37 ERROR: test_bigquery_matcher_success (apache_beam.io.gcp.tests.bigquery_matcher_test.BigqueryMatcherTest)
14:09:37 ----------------------------------------------------------------------
14:09:37 Traceback (most recent call last):
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/target/.tox-py37-gcp/py37-gcp/lib/python3.7/site-packages/mock/mock.py", line 1305, in patched
14:09:37     return func(*args, **keywargs)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher_test.py", line 65, in test_bigquery_matcher_success
14:09:37     hc_assert_that(self._mock_result, matcher)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/target/.tox-py37-gcp/py37-gcp/lib/python3.7/site-packages/hamcrest/core/assert_that.py", line 43, in assert_that
14:09:37     _assert_match(actual=arg1, matcher=arg2, reason=arg3)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/target/.tox-py37-gcp/py37-gcp/lib/python3.7/site-packages/hamcrest/core/assert_that.py", line 49, in _assert_match
14:09:37     if not matcher.matches(actual):
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/target/.tox-py37-gcp/py37-gcp/lib/python3.7/site-packages/hamcrest/core/base_matcher.py", line 28, in matches
14:09:37     match_result = self._matches(item)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py", line 85, in _matches
14:09:37     response = self._query_with_retry()
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/utils/retry.py", line 197, in wrapper
14:09:37     return fun(*args, **kwargs)
14:09:37   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/io/gcp/tests/bigquery_matcher.py", line 102, in _query_with_retry
14:09:37     rows = query_job.result(timeout=60)
14:09:37 AttributeError: 'list' object has no attribute 'result'

@udim
Copy link
Member Author

udim commented Jul 26, 2019

run python 2 postcommit

@udim udim force-pushed the bq-test-client-wait branch from d4255a7 to 7646fd8 Compare July 26, 2019 22:57
@udim
Copy link
Member Author

udim commented Jul 26, 2019

run python 2 postcommit

@pabloem
Copy link
Member

pabloem commented Jul 27, 2019

Run Python_PVR_Flink PreCommit

@pabloem
Copy link
Member

pabloem commented Jul 29, 2019

LGTM. Thanks Udi!

@pabloem pabloem merged commit f9e0969 into apache:master Jul 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants