Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception ignored in: <function _ProactorBasePipeTransport.__del__ #6635

Closed
1 task done
5iddy opened this issue Feb 25, 2022 · 2 comments
Closed
1 task done

Exception ignored in: <function _ProactorBasePipeTransport.__del__ #6635

5iddy opened this issue Feb 25, 2022 · 2 comments
Labels

Comments

@5iddy
Copy link

5iddy commented Feb 25, 2022

Describe the bug

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000280FF592C20>
Traceback (most recent call last):
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 745, in call_soon
    self._check_closed()
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Encountered this issue in Windows 10, with python version 3.10.1 and aiohttp version 3.8.1.

This error occurred after the script is finished and before exiting.

Wrote my script in Linux environment and faced no issues but encountered this error when the same script is executed in windows 10.

To Reproduce

import aiohttp
import asyncio
from bs4 import BeautifulSoup
from aiofile import async_open
import threading
from time import perf_counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import uvloop


async def get_image_urls(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            html = await response.text()

    soup = BeautifulSoup(html, 'html.parser')
    image_urls = []
    for img in soup.find_all("img"):
        try:
            if img["class"][0].startswith("wp-image-"):
                img_url = img["src"].split("?")[0]
                image_urls.append(img_url)
        except KeyError as exc:
            pass
    return image_urls


async def download_image(url, filename):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            async with async_open(filename, 'wb+') as afp:
                async for chunk in response.content.iter_chunked(16*1024):
                    await afp.write(chunk)
    thread = str(threading.current_thread())
    print(f"{filename} downloaded successfully from {url} on {thread}")


async def async_download(urls):
    tasks = []
    for url in urls:
        task = asyncio.create_task(download_image(url, url.split("/")[-1]))
        tasks.append(task)

    await asyncio.gather(*tasks)


def main():
    image_urls = asyncio.run(get_image_urls(
        "https://triponzy.com/blog/most-beautiful-tourist-places-in-india/"))

    max_workers = 6
    for i in range(0, len(image_urls), max_workers):
        with ThreadPoolExecutor(max_workers=max_workers) as executor:
            executor.submit(asyncio.run, async_download(image_urls[i:i+max_workers]))


async def amain():
    image_urls = await get_image_urls(
        "https://triponzy.com/blog/most-beautiful-tourist-places-in-india/")
    max_workers = 6
    for i in range(0, len(image_urls), max_workers):
        urls = image_urls[i:i+max_workers]
        await async_download(urls)


if __name__ == "__main__":
    asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
    start_time = perf_counter()
    #main()
    asyncio.run(amain())
    end_time = perf_counter()
    time_lapsed = (end_time - start_time)
    print(f"finished executing in {time_lapsed}")

Note: This is the code. I wrote this script for educational purposes to asyncio and basic web scraping.

Expected behavior

script to exit gracefully after its finished just like it did in my linux pc.

Logs/tracebacks

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000280FF592C20>
Traceback (most recent call last):
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 745, in call_soon
    self._check_closed()
  File "C:\Users\mahchalla\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Python Version

$ python --version
version 3.10.1

aiohttp Version

$ python -m pip show aiohttp
version 3.8.1

multidict Version

$ python -m pip show multidict
Version: 6.0.2

yarl Version

$ python -m pip show yarl
Version: 1.7.2

OS

Windows 10

Note: Didnt experience this issue in ArchLinux (5.16.10-arch1-1)

Related component

Client

Additional context

No issues in Linux. Only encountered in Windows 10.

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@5iddy 5iddy added the bug label Feb 25, 2022
@Dreamsorcerer
Copy link
Member

Dreamsorcerer commented Feb 27, 2022

What if you add an await asyncio.sleep(.25) to the end of amain, as per: https://docs.aiohttp.org/en/stable/client_advanced.html#graceful-shutdown ?

@evgfilim1
Copy link

evgfilim1 commented Mar 15, 2022

Sleep should help.
See #1925 also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants