Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does undetected chromedriver leave so many instances of chrome.exe open in memory? #1270

Open
disposablecarcass opened this issue May 17, 2023 · 26 comments

Comments

@disposablecarcass
Copy link

My script opens a new chrome every few minutes and exits with driver.quit(). With a normal chrome driver, driver.quit() kills everything. With undetected chrome driver, for some reason, I see each insteance of Google Chrome stay open in the task manager despite having the window disappeared from the screen. I have to end task manually in task manager or they will max out my cpu.

Does anyone know why this is happening or how to rectify?

@ManiMozaffar
Copy link

Because __del__ is very bad implemented
you may use #1258 , or you can use seleniumbase, which doesn't have this bug in the uc module.

@disposablecarcass
Copy link
Author

Because __del__ is very bad implemented you may use #1258 , or you can use seleniumbase, which doesn't have this bug in the uc module.

Is there any reason why this hasn't been merged into the main branch? Is there anyway I can specify a branch installing directly from pycharm? Or do I have to do dl from github and point to it manually?

@disposablecarcass
Copy link
Author

disposablecarcass commented May 20, 2023

@ManiMozaffar So I'm reading conflicting information in these threads. Does your commit itself fix this issue? Do I need to include some type of implicit wait after driver.quit() as well?

Thanks for the help

@ManiMozaffar
Copy link

Because __del__ is very bad implemented you may use #1258 , or you can use seleniumbase, which doesn't have this bug in the uc module.

Is there any reason why this hasn't been merged into the main branch? Is there anyway I can specify a branch installing directly from pycharm? Or do I have to do dl from github and point to it manually?

This isn't merged, because in both PRs, the maintainers haven't checked and provided feedback.
Yes you can, pip install git+.....

@ManiMozaffar So I'm reading conflicting information in these threads. Does your commit itself fix this issue? Do I need to include some type of implicit wait after driver.quit() as well?

Thanks for the help

no you don't need to, just use my code.
it's the issue with thread, in each thread we create new event loop, and each event loop is creating 2 new sockets with different ports, so the more chrome you open, the more thread it will have, and the more event loops it will have in parallel, causing port exhaustion.

@disposablecarcass
Copy link
Author

@ManiMozaffar
Thanks for the reply. I did try your code, but still have the same issue of instances of chrome.exe being left open when using driver.close()

I'm not sure what the best solution is to this, but I think for the time being, I may have to use the os package to kill all chrome instances between each iteration of my script.

I'll keep my eyes open here tho to see if any solution is found.

Thanks again

@ultrafunkamsterdam
Copy link
Owner

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

@disposablecarcass
Copy link
Author

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

Interesting, thanks for the update. Any rough eta on the new package? In the meantime, I'll just use os to kill the chrome that stays open in memory

@sSuHao
Copy link

sSuHao commented May 23, 2023

Did anyone try to run in incognito mode?

@pm85
Copy link

pm85 commented Jun 28, 2023

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

Whats the timeline on the new implementation?

@Mister-Stein
Copy link

Unset the --no-sandbox Chrome argument, setting no_sandbox argument of uc.Chrome to False, because it is set to True by default. Example:

uc.Chrome(driver_executable_path=driver_executable_path, options=options, user_data_dir=user_data_dir, no_sandbox=False, user_multi_procs=True, use_subprocess=False)

But some things to consider:
Users say that undetected-chromedriver doesn't work without --no-sandbox Chrome argument on Windows 7 and lower, but works on Windows 8.1 and above. I didn't tested that for myself, but I think it's true.
Also, undetected-chromedriver most of the times doesn't work on Linux without --no-sandbox Chrome argument(when running test script as root user for example). So, that's because no_sandbox argument of uc.Chrome is set to True by default. Concluding that, set no_sandbox argument of uc.Chrome to False for Windows, and set to True for Linux.
But I warn that undetected-chromedriver has breached no_sandbox functionality, so --no-sandbox is always set, whether or not you set it to True or False. I've made pull request to fix it, so we need to wait until it is applied(or download my fork, and use as custom Python module). #1542

@tromotixc
Copy link

is there any solution for quit() problem?
thank you in advance!

@iamumairayub
Copy link

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

Eagerly waiting for this.

@disposablecarcass
Copy link
Author

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

Eagerly waiting for this.

do you know if there is a development blog for this? would love to follow along and try to learn. feeling pretty useless at the moment trying to fix the old version of uc. anyway hopefully this comes out soon - looking forward to it

@Sleeeepy7
Copy link

My script opens a new chrome every few minutes and exits with driver.quit(). With a normal chrome driver, driver.quit() kills everything. With undetected chrome driver, for some reason, I see each insteance of Google Chrome stay open in the task manager despite having the window disappeared from the screen. I have to end task manually in task manager or they will max out my cpu.

Does anyone know why this is happening or how to rectify?

Have you found a solution?

@11AnJo
Copy link

11AnJo commented Jan 11, 2024

@Sleeeepy7
@disposablecarcass
I really don't know why but disabling gpu worked for me. I've tested it on multiple machines and multiple times.
options.add_argument("--disable-gpu")

@cmdlinebeep
Copy link

@Sleeeepy7 @disposablecarcass I really don't know why but disabling gpu worked for me. I've tested it on multiple machines and multiple times. options.add_argument("--disable-gpu")

I get this error and I use that flag.

@11AnJo
Copy link

11AnJo commented Jan 30, 2024

@cmdlinebeep This is so weird that its working on my computer and my friend and few vms putted for tests. Can you share version of undetected-chromedriver and your chrome version?

@Darren80
Copy link

Is there still no solution for this, I am using the pip package, is the issue fixed on this repo?

@11AnJo
Copy link

11AnJo commented Feb 13, 2024

Is there still no solution for this, I am using the pip package, is the issue fixed on this repo?

For me and my friend adding option to disable gpu helped, but its not working for everybody

options.add_argument("--disable-gpu")

@ifeldshteyn
Copy link

Because i will only fix cricital issues. Im rewriting the whole package. (Spoiler: this performs like crazy, and no more Chrome driver

Just checking if the fix to the open socket problem is available?

@xerobia-test
Copy link

I've found using an older version of Chrome like V113 works. But V113 is also less stable than the current 124

@ifeldshteyn
Copy link

I've found using an older version of Chrome like V113 works. But V113 is also less stable than the current 124

I don't mind downgrading but can you confirm that the socket issue really goes away with it? I had to add a hack that reboots the service after it has 1000 open sockets.

@josetilin37
Copy link

hello, I had the same problem about the phantom processes and I managed to solve it by using pyautogui to close the browser just before applying the driver.quit(), this way the browser is completely closed just before the driver.quit() and leaves no trace in the task manager, this solution was effective for me. I hope this helps you.

pyautogui.moveTo(x, y, duration=0.35) #coords for closing the browser
pyautogui.click()
driver.quit()

You can adjust the duration of this action to make it more instantaneous

@ifeldshteyn
Copy link

ifeldshteyn commented Apr 24, 2024

hello, I had the same problem about the phantom processes and I managed to solve it by using pyautogui to close the browser just before applying the driver.quit(), this way the browser is completely closed just before the driver.quit() and leaves no trace in the task manager, this solution was effective for me. I hope this helps you.

pyautogui.moveTo(x, y, duration=0.35) #coords for closing the browser pyautogui.click() driver.quit()

You can adjust the duration of this action to make it more instantaneous

It's a workaround for sure but it would be nice to have a working __del__ that cleans up fds so the proc folder doesn't have a heart attack. Is this something I can do with the driver itself instead of using another dependency? Thanks!

@cvtkir
Copy link

cvtkir commented May 12, 2024

For me using both driver.close() and driver.quit() one by one helped a lot

@noureddineerramy
Copy link

after driver.quit() call destroyChromeProcess (code that close any chrome process that its parent pid is not exists) like this :

import psutil
import os
import signal

def get_chrome_parent_pids():
    parent_pids = set()
    for proc in psutil.process_iter(['pid', 'name', 'ppid']):
        if 'chrome' in proc.name().lower():
            parent_pids.add(proc.ppid())
    return parent_pids

def kill_processes_with_parent_pid(parent_pid):
    try:
        for proc in psutil.process_iter(['pid', 'ppid']):
            if proc.info['ppid'] == parent_pid:
                try:
                    process = psutil.Process(proc.info['pid'])
                    process.terminate()
                except psutil.NoSuchProcess:
                    pass
    except psutil.NoSuchProcess:
        pass

def destroyChromeProcess():
    chrome_parent_pids = get_chrome_parent_pids()
    for parent_pid in chrome_parent_pids:
        if not psutil.pid_exists(parent_pid):
            kill_processes_with_parent_pid(parent_pid)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests