[BEAM-3713] Add pytest testing infrastructure #7949

udim · 2019-02-26T19:21:35Z

Initial step towards migrating off of nosetests.

Runs integration tests via tox: tox -e py27-gcp-pytest,py36-pytest,etc.
@pytest.mark.{pre,post}commit replaces @attr('IT')
- Also can mark tests as: no_dataflow, no_direct, etc. to exclude
  tests that don't pass on certain runners.
conftest.py replaces BeamTestPlugin
Jenkins reports for all tests run (not just the last nosetests
invocation), using multiple XML outputs.
Allows specifying test module in tox cmd line. Example:

tox -e py27-pytest apache_beam.transforms.window_test

Still needs:

Verification that all the tests that run on nose also run
on pytest.

Nice to have:

Parallelization, but this requires fixing pickling issues with the
pytest plugins used for this.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go	---	---	---	---	---	---
Java
Python	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

udim · 2019-02-26T19:21:56Z

run seed job

udim · 2019-02-26T19:26:41Z

run python precommit_pytest

udim · 2019-02-27T00:04:00Z

run seed job

udim · 2019-02-27T00:23:50Z

run seed job

udim · 2019-02-27T17:49:56Z

run seed job

udim · 2019-02-27T18:37:04Z

run seed job

udim · 2019-02-27T19:08:55Z

run python precommit_pytest

udim · 2019-02-27T19:09:27Z

run python precommit_pytest

udim · 2019-02-27T19:10:47Z

run python_pytest precommit

udim · 2019-02-28T00:49:50Z

run seed job

udim · 2019-02-28T00:59:11Z

run python_pytest precommit

markflyhigh · 2019-02-28T01:27:50Z

sdks/python/setup.py

@@ -201,6 +202,7 @@ def run(self):
    install_requires=REQUIRED_PACKAGES,
    python_requires=python_requires,
    test_suite='nose.collector',


we probably can also use pytest here.

I don't know what to put here for pytest. The instructions said to add the setup_requires line below.

markflyhigh · 2019-02-28T01:37:02Z

Thank you for initiating this work! I have this nose to pytest thought long time ago, but the main reason to push my back is that python framework deeply integrated with nose. Except BeamTestPlugin and @attr you mentioned in top description, I think we should also take care of:

integration test execution command
proper logging in unit/integration test
parallelization

udim · 2019-04-13T02:38:30Z

run seed job

udim · 2019-04-15T15:48:28Z

run seed job

udim · 2019-04-15T15:52:36Z

run python precommit

udim · 2019-04-15T16:14:04Z

run python_pytest precommit

udim · 2019-04-15T18:30:59Z

run python_pytest precommit

stale · 2019-06-14T19:06:06Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

udim · 2019-06-14T20:06:16Z

still working on this

lukecwik · 2019-09-17T00:20:33Z

sdks/python/tox.ini

 [testenv:py36]
 setenv =
  RUN_SKIPPED_PY3_TESTS=0
 commands =
  python apache_beam/examples/complete/autocomplete_test.py
  python setup.py nosetests

+[testenv:py36-pytest]
+setenv =


From @robertwb on #9592:
We should probably be using https://tox.readthedocs.io/en/latest/example/basic.html#compressing-dependency-matrix to avoid these kinds of omissions.

This is a temporary solution while waiting for full solution in apache#7949

This is apache#7949 without IT support. - Runs unit tests using pytest - tox: `tox -e py27-gcp-pytest,py36-pytest,etc.` - gradle: `../../gradlew pythonPreCommitPytest` - github PR phrase: `run python_pytest precommit` - Tests run in parallel (still single-threaded but on multiple worker processes). - no_xdist marker used for tests that don't work the xdist plugin. - Allows specifying test module in tox cmd line. Example: ```sh tox -e py27-pytest apache_beam.transforms.window_test ```

This is #7949 without IT support. - Runs unit tests using pytest - tox: `tox -e py27-gcp-pytest,py36-pytest,etc.` - gradle: `../../gradlew pythonPreCommitPytest` - github PR phrase: `run python_pytest precommit` - Tests run in parallel (still single-threaded but on multiple worker processes). - no_xdist marker used for tests that don't work the xdist plugin. - Allows specifying test module in tox cmd line. Example: ```sh tox -e py27-pytest apache_beam.transforms.window_test ```

This is apache#7949 without IT support. - Runs unit tests using pytest - tox: `tox -e py27-gcp-pytest,py36-pytest,etc.` - gradle: `../../gradlew pythonPreCommitPytest` - github PR phrase: `run python_pytest precommit` - Tests run in parallel (still single-threaded but on multiple worker processes). - no_xdist marker used for tests that don't work the xdist plugin. - Allows specifying test module in tox cmd line. Example: ```sh tox -e py27-pytest apache_beam.transforms.window_test ```

stale · 2019-11-16T00:52:56Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

udim · 2019-11-20T00:27:02Z

still working on this

robertwb · 2020-01-10T23:54:32Z

Is this still being worked on?

udim · 2020-03-04T18:09:54Z

Picking this up again

stale · 2020-05-03T18:43:49Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

udim · 2020-05-08T21:01:35Z

I haven't had time to work on this, but it's still relevant.

stale · 2020-07-11T01:33:59Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

udim · 2020-07-11T01:50:45Z

bump

stale · 2020-09-11T02:42:29Z

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

stale · 2020-09-19T19:13:51Z

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

udim force-pushed the python-tests-option branch from 650f78f to a76ec7f Compare February 26, 2019 23:56

udim force-pushed the python-tests-option branch from a76ec7f to 6fe353e Compare February 27, 2019 00:23

udim mentioned this pull request Feb 27, 2019

[BEAM-1251] Fix Python 3 compatibility bug in pickler.py #7953

Merged

3 tasks

udim force-pushed the python-tests-option branch from 5137e56 to e30e8b8 Compare February 27, 2019 17:48

udim force-pushed the python-tests-option branch from e30e8b8 to 33dde25 Compare February 27, 2019 18:36

udim force-pushed the python-tests-option branch from 33dde25 to 2706735 Compare February 28, 2019 00:47

markflyhigh reviewed Feb 28, 2019

View reviewed changes

udim force-pushed the python-tests-option branch from 2706735 to 2c94f4c Compare April 13, 2019 02:33

udim force-pushed the python-tests-option branch from 2c94f4c to 8c1fc1a Compare April 15, 2019 18:30

stale bot added the stale label Jun 14, 2019

stale bot removed the stale label Jun 14, 2019

lukecwik reviewed Sep 17, 2019

View reviewed changes

soyrice pushed a commit to soyrice/beam that referenced this pull request Sep 19, 2019

[BEAM-8093] Commit change to ensure py36-gcp runs tests

8285052

This is a temporary solution while waiting for full solution in apache#7949

chadrik mentioned this pull request Sep 25, 2019

[BEAM-8213] Split up monolithic python preCommit tests on jenkins #9642

Closed

3 tasks

udim mentioned this pull request Oct 9, 2019

[BEAM-3713] Add pytest for unit tests #9756

Merged

3 tasks

stale bot added the stale label Nov 16, 2019

stale bot removed the stale label Nov 20, 2019

tvalentyn mentioned this pull request Jan 7, 2020

[BEAM-8951] Stop using nose in load tests #10435

Merged

3 tasks

iemejia added python build infra labels Feb 12, 2020

stale bot added the stale label May 3, 2020

stale bot removed the stale label May 8, 2020

stale bot added the stale label Jul 11, 2020

stale bot removed the stale label Jul 11, 2020

stale bot added the stale label Sep 11, 2020

stale bot closed this Sep 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-3713] Add pytest testing infrastructure #7949

[BEAM-3713] Add pytest testing infrastructure #7949

udim commented Feb 26, 2019 •

edited

udim commented Feb 26, 2019

udim commented Feb 26, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 28, 2019

udim commented Feb 28, 2019

markflyhigh Feb 28, 2019

udim Aug 26, 2019

markflyhigh commented Feb 28, 2019

udim commented Apr 13, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

stale bot commented Jun 14, 2019

udim commented Jun 14, 2019

lukecwik Sep 17, 2019

stale bot commented Nov 16, 2019

udim commented Nov 20, 2019

robertwb commented Jan 10, 2020

udim commented Mar 4, 2020

stale bot commented May 3, 2020

udim commented May 8, 2020

stale bot commented Jul 11, 2020

udim commented Jul 11, 2020

stale bot commented Sep 11, 2020

stale bot commented Sep 19, 2020

[BEAM-3713] Add pytest testing infrastructure #7949

[BEAM-3713] Add pytest testing infrastructure #7949

Conversation

udim commented Feb 26, 2019 • edited

Post-Commit Tests Status (on master branch)

udim commented Feb 26, 2019

udim commented Feb 26, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 27, 2019

udim commented Feb 28, 2019

udim commented Feb 28, 2019

markflyhigh Feb 28, 2019

Choose a reason for hiding this comment

udim Aug 26, 2019

Choose a reason for hiding this comment

markflyhigh commented Feb 28, 2019

udim commented Apr 13, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

udim commented Apr 15, 2019

stale bot commented Jun 14, 2019

udim commented Jun 14, 2019

lukecwik Sep 17, 2019

Choose a reason for hiding this comment

stale bot commented Nov 16, 2019

udim commented Nov 20, 2019

robertwb commented Jan 10, 2020

udim commented Mar 4, 2020

stale bot commented May 3, 2020

udim commented May 8, 2020

stale bot commented Jul 11, 2020

udim commented Jul 11, 2020

stale bot commented Sep 11, 2020

stale bot commented Sep 19, 2020

udim commented Feb 26, 2019 •

edited