New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setting progress_bar=True freezes execution for parallel_apply before reaching 1% completion on all CPU's #131
Comments
progress_bar=True
freezes execution for parallel_
apply` before reaching 1% completion on all CPU'sprogress_bar=True
freezes execution for parallel_apply
before reaching 1% completion on all CPU's
progress_bar=True
freezes execution for parallel_apply
before reaching 1% completion on all CPU's
Similar issue here, except that once one process reaches 100% all others get stuck at 99.99%. Problem is completely fixed by turning off the progress bars (but I don't look quite leet enough /s). Specs:
|
Same issue ^ I'm running parallel_apply, and 2 out of 12 bars finish, the others get stuck and I'm getting a "python quit unexpectedly" error from the os |
Similar issue and i'm only working on about 12k rows. Python 3.6.9 on Ubuntu-18.04 WSL2 ** Edit** |
Same issue here, I set the number of workers to 12 but 2 of them stopped with 1% progress. |
I have the same issue, working on 111k rows, Python 3.8. |
Same here. None of the processes make any progress. I use |
Same, is there any workaround for it? |
Setting |
also experiencing this issue Python 3.8 happens both at all <1% and sometimes at most >99% the workaround |
This happens to me too, but the workaround works. |
Could you please tell me the version of pandarallel you are using? |
Name: pandarallel |
Could you please try with Pandarallel 1.5.7?
Le ven. 4 mars 2022 à 18:26, Lucas Servi ***@***.***> a
écrit :
… Name: pandarallel
Version: 1.5.5
—
Reply to this email directly, view it on GitHub
<#131 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABFW7VWOFRGOQEQFGPXSD3TU6JBS7ANCNFSM4WXWW2UA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you commented.Message ID:
***@***.***>
|
Sorry, I don't get if your issue is fixed with Pandarallel 1.5.7.
|
I made a little folder with code + 2 dataframes used. Hope it helps! Thanks for Pandarallel, it's amazing :)! |
Hello, I do reproduce your issue with To know the current version of import pandarallel
pandarallel.__version__ To be sure you install the last version of pip install pandallel --upgrade (I guess you are not using |
I'm assuming this has been fixed. |
I got it to work. Couple of observations:
|
can you consistently reproduce the problem like this? If yes, can you send me an SSCCE? |
I am facing a similar issue of parallel_apply() freezing when running my code on an EC2 cluster. It was working fine up till a few days back, everday on a schedule, but suddenly it has stopped working. On running the same code on my local machine it is working alright though. I have also kept |
When
progress_bar=True
, I noticed that the execution of myparallel_apply
task stopped right before all parallel processes reached 1% progress mark.Here are some further details of what I was encountering -
logging
withDEBUG
messages, but no messages were displayed when the execution stopped. There were no error messages either. The dataframe rows simply stopped processing further and the process seemed to be frozen.progress_bar=False
.The text was updated successfully, but these errors were encountered: