Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BF(TST): use higher level asyncio.run instead of asyncio.get_event_loop in test_inside_async #6808

Merged
merged 1 commit into from Jul 8, 2022

Conversation

yarikoptic
Copy link
Member

Initially discovered in #6803 while actually making our travis CI test against python 3.10, and then reconfirmed locally in a 3.10 venv that

(git)lena:~datalad/datalad-master[master]git
$> DATALAD_TESTS_SSH=1 python -m pytest -s -v -k test_inside_async datalad
====================================================== test session starts ======================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0 -- /home/yoh/proj/datalad/datalad-master/venvs/dev3/bin/python
cachedir: .pytest_cache
rootdir: /home/yoh/proj/datalad/datalad-master, configfile: tox.ini
plugins: fail-slow-0.2.0, cov-3.0.0
collected 1297 items / 1296 deselected / 1 selected

datalad/runner/tests/test_nonasyncrunner.py::test_inside_async FAILEDVersions: annexremote=1.6.0 boto=2.49.0 cmd:7z=16.02 cmd:annex=8.20210223 cmd:bundled-git=UNKNOWN cmd:git=2.35.1 cmd:ssh=8.4p1 cmd:system-git=2.35.1 cmd:system-ssh=8.4p1 datalad=0.16.6+256.gb52a10f13 exifread=3.0.0 humanize=4.2.3 iso8601=1.0.2 keyring=23.6.0 keyrings.alt=UNKNOWN msgpack=1.0.4 mutagen=1.45.1 patoolib=1.12 platformdirs=2.5.2 requests=2.28.1 tqdm=4.64.0
Obscure filename: str=b' |;&%b5{}\'"\xce\x94\xd0\x99\xd7\xa7\xd9\x85\xe0\xb9\x97\xe3\x81\x82 .datc ' repr=' |;&%b5{}\'"ΔЙקم๗あ .datc '
Encodings: default='utf-8' filesystem='utf-8' locale.prefered='UTF-8'
Environment: PATH='/home/yoh/proj/datalad/datalad-master/venvs/dev3/bin:/home/yoh/gocode/bin:/home/yoh/gocode/bin:/home/yoh/bin:/home/yoh/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/usr/local/sbin' LANG='en_US.UTF-8' GIT_PAGER='less --no-init --quit-if-one-screen' GIT_CONFIG_PARAMETERS="'init.defaultBranch=dl-test-branch' 'clone.defaultRemoteName=dl-test-remote'" GIT_CONFIG_GLOBAL='/home/yoh/.tmp/datalad_temp_gu801k9f/.gitconfig' GIT_ASKPASS='true'

=========================================================== FAILURES ============================================================
_______________________________________________________ test_inside_async _______________________________________________________

	def test_inside_async():
		async def main():
			runner = Runner()
			return runner.run(
				(["cmd.exe", "/c"] if on_windows else []) + ["echo", "abc"],
				StdOutCapture)

>       loop = asyncio.get_event_loop()
E       DeprecationWarning: There is no current event loop

datalad/runner/tests/test_nonasyncrunner.py:378: DeprecationWarning
======================================================= warnings summary ========================================================
venvs/dev3/lib/python3.10/site-packages/duecredit/versions.py:15
  /home/yoh/proj/datalad/datalad-master/venvs/dev3/lib/python3.10/site-packages/duecredit/versions.py:15: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
	from distutils.version import StrictVersion, LooseVersion

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================== short test summary info ====================================================
FAILED datalad/runner/tests/test_nonasyncrunner.py::test_inside_async - DeprecationWarning: There is no current event loop
========================================= 1 failed, 1296 deselected, 1 warning in 1.96s ========================================

note that it is solely due to DeprecationWarning being issued and treated as
error, which is the problem only for the master ATM but even there we do not
test really against 3.10 (problem which #6803 is trying to address).

This fix is completely replacing use of low level get_event_loop and uses
recommended by Python docs use of run. See
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop

So, although this fix is against maint, without it we would not fail anywhere
ATM, unless manually running/testing on master against master and using python 3.10.

…op in test_inside_async

Initially discovered in datalad#6803 while actually making our travis CI test against python 3.10, and then reconfirmed locally in a 3.10 venv that

	(git)lena:~datalad/datalad-master[master]git
	$> DATALAD_TESTS_SSH=1 python -m pytest -s -v -k test_inside_async datalad
	====================================================== test session starts ======================================================
	platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0 -- /home/yoh/proj/datalad/datalad-master/venvs/dev3/bin/python
	cachedir: .pytest_cache
	rootdir: /home/yoh/proj/datalad/datalad-master, configfile: tox.ini
	plugins: fail-slow-0.2.0, cov-3.0.0
	collected 1297 items / 1296 deselected / 1 selected

	datalad/runner/tests/test_nonasyncrunner.py::test_inside_async FAILEDVersions: annexremote=1.6.0 boto=2.49.0 cmd:7z=16.02 cmd:annex=8.20210223 cmd:bundled-git=UNKNOWN cmd:git=2.35.1 cmd:ssh=8.4p1 cmd:system-git=2.35.1 cmd:system-ssh=8.4p1 datalad=0.16.6+256.gb52a10f13 exifread=3.0.0 humanize=4.2.3 iso8601=1.0.2 keyring=23.6.0 keyrings.alt=UNKNOWN msgpack=1.0.4 mutagen=1.45.1 patoolib=1.12 platformdirs=2.5.2 requests=2.28.1 tqdm=4.64.0
	Obscure filename: str=b' |;&%b5{}\'"\xce\x94\xd0\x99\xd7\xa7\xd9\x85\xe0\xb9\x97\xe3\x81\x82 .datc ' repr=' |;&%b5{}\'"ΔЙקم๗あ .datc '
	Encodings: default='utf-8' filesystem='utf-8' locale.prefered='UTF-8'
	Environment: PATH='/home/yoh/proj/datalad/datalad-master/venvs/dev3/bin:/home/yoh/gocode/bin:/home/yoh/gocode/bin:/home/yoh/bin:/home/yoh/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/usr/local/sbin' LANG='en_US.UTF-8' GIT_PAGER='less --no-init --quit-if-one-screen' GIT_CONFIG_PARAMETERS="'init.defaultBranch=dl-test-branch' 'clone.defaultRemoteName=dl-test-remote'" GIT_CONFIG_GLOBAL='/home/yoh/.tmp/datalad_temp_gu801k9f/.gitconfig' GIT_ASKPASS='true'

	=========================================================== FAILURES ============================================================
	_______________________________________________________ test_inside_async _______________________________________________________

		def test_inside_async():
			async def main():
				runner = Runner()
				return runner.run(
					(["cmd.exe", "/c"] if on_windows else []) + ["echo", "abc"],
					StdOutCapture)

	>       loop = asyncio.get_event_loop()
	E       DeprecationWarning: There is no current event loop

	datalad/runner/tests/test_nonasyncrunner.py:378: DeprecationWarning
	======================================================= warnings summary ========================================================
	venvs/dev3/lib/python3.10/site-packages/duecredit/versions.py:15
	  /home/yoh/proj/datalad/datalad-master/venvs/dev3/lib/python3.10/site-packages/duecredit/versions.py:15: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
		from distutils.version import StrictVersion, LooseVersion

	-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
	==================================================== short test summary info ====================================================
	FAILED datalad/runner/tests/test_nonasyncrunner.py::test_inside_async - DeprecationWarning: There is no current event loop
	========================================= 1 failed, 1296 deselected, 1 warning in 1.96s ========================================

note that it is solely due to DeprecationWarning being issued and treated as
error, which is the problem only for the master ATM but even there we do not
test really against 3.10 (problem which datalad#6803 is trying to address).

This fix is completely replacing use of low level get_event_loop and uses
recommended by Python docs use of run. See
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop

So, although this fix is against maint, without it we would not fail anywhere
ATM, unless manually running/testing on master against master and using python 3.10.
@bpoldrack
Copy link
Member

Appears to hang again (not happening on master): https://ci.appveyor.com/project/mih/datalad/builds/44094400/job/f72royuoojvswqtf

Otherwise looks plausible to me.

@yarikoptic
Copy link
Member Author

yarikoptic commented Jul 7, 2022

Seems to hang at

datalad.core.local.tests.test_create.test_create_sub_gh3463

which is different test from modified here

edit: hm -- search for test_create_sub_gh3463 ends up with no hits, including no hits for this comment. odd

@yarikoptic
Copy link
Member Author

d'oh -- it is not "hanging" - it was killed after reaching a 1h running time! uff. I requested rerun of incomplete for clarity...

Copy link
Contributor

@christian-monch christian-monch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. LGTM

@yarikoptic yarikoptic merged commit 2dea515 into datalad:maint Jul 8, 2022
@yarikoptic yarikoptic deleted the bf-async-run branch July 11, 2022 19:52
@github-actions
Copy link

🚀 PR was released in 0.17.1 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released semver-tests Changes only affect tests, no impact on version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants