Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compatibility issue when asyncio loop policy is not the default one #139

Closed
nlgranger opened this issue Aug 26, 2021 · 7 comments
Closed

Comments

@nlgranger
Copy link

get_session blocks when the loop policy is not the default one, for instance when using asyncio-glib:

import asyncio_glib
import asyncio

asyncio.set_event_loop_policy(asyncio_glib.GLibEventLoopPolicy())
from arsenic import services, browsers, get_session


async def loop():
    while True:
        await asyncio.sleep(0.5)
        print('.', end='', flush=True)


async def main():
    service = services.Geckodriver()
    browser = browsers.Firefox()
    print('ready?')
    async with get_session(service, browser) as d:
        await d.get('https://www.google.fr')
        print("done")


asyncio.run(asyncio.wait([main(), loop()]))
@dimaqq
Copy link
Contributor

dimaqq commented Aug 26, 2021

Interesting... any idea what's actually blocking?
Is it trying to spawn geckodriver?
Or trying to connect to its port?
Or is it the .get(...) operation, which, in essence is aiohttp request?

@nlgranger
Copy link
Author

I do not see any geckodriver process and I just noticed the python process uses 100% of a CPU while blocking on get_session.

@nlgranger
Copy link
Author

It block when checking the version of geckodriver, here is the call stack:

run_process (.../venv/lib/python3.9/site-packages/arsenic/subprocess.py:46)
_check_version (.../venv/lib/python3.9/site-packages/arsenic/services.py:73)
start (.../venv/lib/python3.9/site-packages/arsenic/services.py:92)
start_session (.../venv/lib/python3.9/site-packages/arsenic/__init__.py:28)
__aenter__ (.../venv/lib/python3.9/site-packages/arsenic/__init__.py:16)
main (.../test.py:18)
_run (/usr/lib/python3.9/asyncio/events.py:80)
_run_once (/usr/lib/python3.9/asyncio/base_events.py:1882)
run_forever (/usr/lib/python3.9/asyncio/base_events.py:596)
run_until_complete (/usr/lib/python3.9/asyncio/base_events.py:629)
run (/usr/lib/python3.9/asyncio/runners.py:44)
<module> (.../test.py:24)

@dimaqq
Copy link
Contributor

dimaqq commented Aug 26, 2021

https://github.com/HDE/arsenic/blob/7323c550a46b94aa0dcda6659b9c3eff11bec3f0/src/arsenic/subprocess.py#L46
code in question...

I wonder if asyncio.create_subprocess_exec and process.communicate are supported in this custom event loop 🤔

@dimaqq
Copy link
Contributor

dimaqq commented Aug 26, 2021

It could be smth like jhenstridge/asyncio-glib#5 🤔

@dimaqq
Copy link
Contributor

dimaqq commented Aug 26, 2021

Given that the last update to asyncio_glib was almost 2 years ago, I'm tempted to point my finger there.

@nlgranger
Copy link
Author

Indeed, I will try something else for glib event loop interaction. Thanks for looking into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants