Python quit unexpectedlly

[hector-orkestro]

Hi guys,

I got python crashing on my MacBook, even with only one process. Has it happened to anyone before?

Thanks,
Hector

[nalepae]

Hi,

Can you please provide a minimal piece of code to let me reproduce your issue?

[longinteger017]

I tried running .parallel_apply() on a DataFrame of 1m entries and it worked fine until I accidentally started two instances in my code in a one run and since then I got that error.

This is what I run:

from pandarallel import pandarallel
import pandas as pd


def func(x):
    return x * 2

def run():
    pandarallel.initialize(progress_bar=True)
    df = pd.DataFrame(range(1, 10))
    df = df.parallel_apply(func, axis=1)
    print(df.tail())

run()

Output

If the range is set to range(1,10) it works but if it's set higher like range(1,100), it starts and immediately stops with the following output in the terminal:

INFO: Pandarallel will run on 16 workers.
INFO: Pandarallel will use standard multiprocessing data transfer (pipe) to transfer data between the main process and workers.
0.00% | 0 / 7 |
0.00% | 0 / 7 |
0.00% | 0 / 7 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
16.67% :::::: | 1 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |
0.00% | 0 / 6 |

and a pop up window saying Python quit unexpectedly.

Configuration

Python Version: 3.7.3
OS: macOS Catalina 10.15.7

pip freeze after installing pandas and pandarallels:

dill==0.3.4
numpy==1.21.4
pandarallel==1.5.4
pandas==1.3.4
python-dateutil==2.8.2
pytz==2021.3
six==1.16.0

EDIT:
It seems like it is a similar issue to this one and it works again if I set the progress_bar parameter to False.

replace pandarallel.initialize(progress_bar=True)
to pandarallel.initialize(progress_bar=False)

[shermansiu]

Seems like the fix for Python 3.7 is to upgrade your version of Python (at least to 3.7.6).

The aforementioned code snippet works fine for me on Python 3.10.13.