[BEAM-3713] Moving integration tests from nose to pytest #14481

benWize · 2021-04-08T19:42:57Z

Move integration tests from nose to pytest on the following jobs:

job_PostCommit_Python
job_PostCommit_Python_ValidatesContainer_Dataflow
job_PostCommit_Python_ValidatesRunner_Dataflow
job_PostCommit_Python_ValidatesRunner_Flink

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	ULR	Samza	Twister2
Go	---	---	---
Java
Python	---	---	---
XLang	---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

codecov · 2021-04-08T20:35:12Z

Codecov Report

Merging #14481 (3395efa) into master (325471d) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #14481      +/-   ##
==========================================
+ Coverage   83.64%   83.67%   +0.03%     
==========================================
  Files         443      443              
  Lines       59007    59011       +4     
==========================================
+ Hits        49354    49376      +22     
+ Misses       9653     9635      -18

Impacted Files	Coverage Δ
...dks/python/apache_beam/options/pipeline_options.py
...ython/apache_beam/io/gcp/tests/bigquery_matcher.py
...on/apache_beam/runners/portability/spark_runner.py
...e_beam/examples/complete/top_wikipedia_sessions.py
.../examples/snippets/transforms/aggregation/count.py
...am/testing/benchmarks/chicago_taxi/process_tfma.py
...apache_beam/runners/portability/portable_runner.py
...he_beam/io/flink/flink_streaming_impulse_source.py
...ython/apache_beam/examples/complete/estimate_pi.py
.../srcs/sdks/python/apache_beam/typehints/schemas.py
... and 876 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f61d100...3395efa. Read the comment docs.

benWize · 2021-04-08T20:38:12Z

Run Python 3.7 PostCommit

benWize · 2021-04-08T22:24:34Z

Run Python Dataflow ValidatesContainer

benWize · 2021-04-08T22:25:02Z

Run Python Dataflow ValidatesRunner

…opts

benWize · 2021-04-09T17:27:42Z

R: @udim, could you take a look at this? Thanks in advance!

udim · 2021-04-09T20:20:03Z

Run Python 3.7 PostCommit

udim · 2021-04-09T20:21:47Z

Run PythonDocs PreCommit

udim · 2021-04-09T20:30:08Z

In https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_PR/176/console I see this:

18:15:51 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?

I see that you updated the archiveJunit paths, so it's probably because the Jenkins job config is not updated (needs a "run seed job").

udim · 2021-04-09T20:30:30Z

rnu seed job

udim · 2021-04-09T20:31:52Z

run seed job

benWize · 2021-04-12T14:44:42Z

In https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_PR/176/console I see this:
18:15:51 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
I see that you updated the archiveJunit paths, so it's probably because the Jenkins job config is not updated (needs a "run seed job").

Thanks Udi, I think the "Seed Job" is not triggering on the comment (https://ci-beam.apache.org/job/beam_SeedJob/), I was thinking maybe because I'm not a committer and that job probably only runs on committer's PRs.

aaltay · 2021-04-23T02:39:44Z

In ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_PR/176/console I see this:
18:15:51 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
I see that you updated the archiveJunit paths, so it's probably because the Jenkins job config is not updated (needs a "run seed job").
Thanks Udi, I think the "Seed Job" is not triggering on the comment (ci-beam.apache.org/job/beam_SeedJob), I was thinking maybe because I'm not a committer and that job probably only runs on committer's PRs.

I believe this is true. And @tvalentyn had a work around for this. Maybe he could share, @udim could help with running tests.

udim · 2021-04-23T20:30:42Z

Running a seed job from a copy of this PR: https://ci-beam.apache.org/job/beam_SeedJob/7972/ (#14629)

udim · 2021-04-23T20:40:47Z

Run Python 3.7 PostCommit

udim · 2021-04-23T20:41:48Z

Run Python Dataflow ValidatesContainer

udim · 2021-04-23T22:28:25Z

https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont_PR/182/ passed and has test results correctly reported. Overall an improvement since it now has stderr/out captured.

https://ci-beam.apache.org/job/beam_PostCommit_Python37_PR/257/ timed out after 1h40m. I think the above result is enough to verify the .groovy changes.

udim

Some comments and lint failures, but otherwise LGTM

sdks/python/scripts/run_integration_test.sh

sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py

udim · 2021-04-23T23:12:20Z

sdks/python/apache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py

-  @attr('IT', 'ValidatesRunner', 'sickbay-batch')
+  @pytest.mark.no_sickbay_batch
+  @pytest.mark.it_validatesrunner
+  @pytest.mark.it_postcommit


This is the only test that runs in both ValidatesRunner and post-commit. I think that's a bug but let's leave it for this PR.

…tover import

udim · 2021-04-26T17:27:42Z

Lint failure:

09:32:57 apache_beam/testing/test_pipeline.py:149:0: C0301: Line too long (100/80) (line-too-long)

You can run lint checks locally as well using ./gradlew lint: https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-LintandFormattingChecks

udim

Thanks!

benWize · 2021-04-26T21:20:33Z

Thanks!

Thanks to you

TheNeuralBit · 2021-04-28T15:32:16Z

Is it possible this broke Python PostCommits? They've been failing since this went in: https://ci-beam.apache.org/job/beam_PostCommit_Python38/1146/

…che#14481)" This reverts commit 38e879b

…ts from nose to pytest (#14481)" Revert "[BEAM-3713] Moving integration tests from nose to pytest (#14481)"

benWize added 4 commits April 8, 2021 13:23

[BEAM-3713] Move from nose to pytest in integration tests

e7f77aa

[BEAM-3713] Remove unused import

4428a6c

[BEAM-3713] Format with yapf

afa7b3c

[BEAM-3713] Format with yapf

dedd6b5

[BEAM-3713] Change suite-name and add --capture=no to default pytest …

0651f30

…opts

benWize marked this pull request as ready for review April 9, 2021 17:26

[BEAM-3713] Solve merge conflicts

c1fda95

udim reviewed Apr 23, 2021

View reviewed changes

[BEAM-3713] Fix indentation in run_integration_test.sh and remove lef…

87c5765

…tover import

benWize requested a review from udim April 26, 2021 16:01

benWize added 2 commits April 26, 2021 11:28

Merge branch 'master' into BEAM-3713

b3465b2

[BEAM-3713] Change attr to pytest marker on translation_test

3fdcfb5

[BEAM-3713] Fixes on lint formatting

3395efa

udim approved these changes Apr 26, 2021

View reviewed changes

udim merged commit 38e879b into apache:master Apr 26, 2021

youngoli added a commit to youngoli/beam that referenced this pull request Apr 28, 2021

Revert "[BEAM-3713] Moving integration tests from nose to pytest (apa…

1a5f90e

…che#14481)" This reverts commit 38e879b

youngoli mentioned this pull request Apr 28, 2021

Revert "[BEAM-3713] Moving integration tests from nose to pytest (#14481)" #14676

Merged

4 tasks

youngoli added a commit that referenced this pull request Apr 29, 2021

Merge pull request #14676: Revert "[BEAM-3713] Moving integration tes…

87e1164

…ts from nose to pytest (#14481)" Revert "[BEAM-3713] Moving integration tests from nose to pytest (#14481)"

benWize mentioned this pull request May 4, 2021

[BEAM-3713] Move dataflow:validatesContainerTests from nosetest to pytest #14716

Merged

4 tasks

benWize mentioned this pull request May 11, 2021

[BEAM-3713] Move validatesRunnerBatchTests and validatesRunnerStreaming tests from nose to pytest #14788

Merged

4 tasks

benWize mentioned this pull request May 21, 2021

[BEAM-3713] Move PostCommit_Python from nosetest to pytest #14859

Merged

4 tasks

damccorm mentioned this pull request Jun 4, 2022

Python Postcommits timing out on postCommitIT #20930

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-3713] Moving integration tests from nose to pytest #14481

[BEAM-3713] Moving integration tests from nose to pytest #14481

benWize commented Apr 8, 2021

codecov bot commented Apr 8, 2021 •

edited

benWize commented Apr 8, 2021

benWize commented Apr 8, 2021

benWize commented Apr 8, 2021

benWize commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

benWize commented Apr 12, 2021

aaltay commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim left a comment

udim Apr 23, 2021

udim commented Apr 26, 2021

udim left a comment

benWize commented Apr 26, 2021

TheNeuralBit commented Apr 28, 2021

[BEAM-3713] Moving integration tests from nose to pytest #14481

[BEAM-3713] Moving integration tests from nose to pytest #14481

Conversation

benWize commented Apr 8, 2021

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

codecov bot commented Apr 8, 2021 • edited

Codecov Report

benWize commented Apr 8, 2021

benWize commented Apr 8, 2021

benWize commented Apr 8, 2021

benWize commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

udim commented Apr 9, 2021

benWize commented Apr 12, 2021

aaltay commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim commented Apr 23, 2021

udim left a comment

Choose a reason for hiding this comment

udim Apr 23, 2021

Choose a reason for hiding this comment

udim commented Apr 26, 2021

udim left a comment

Choose a reason for hiding this comment

benWize commented Apr 26, 2021

TheNeuralBit commented Apr 28, 2021

codecov bot commented Apr 8, 2021 •

edited