Connection to IPC socket failed for pathname

[lsircc]

Hello, ask, pandarallel.initialize () appears warning how is it? Thank you: WARNING: Logging before InitGoogleLogging() is written to STDERR
E0812 19:11:57.484051 2409853824 http://io.cc:168] Connection to IPC socket failed for pathname /var/folders/sp/vz74h1tx3jlb3jqrq__bjwh00000gp/T/pandarallel-32ts0h6r/plasma_sock, retrying 20 more times
please help me,thank

[lsircc]

In addition, an error occurred after more than 54 cycles:
Traceback (most recent call last):
File "/Users/lsir/.local/lib/python3.7/site-packages/multiprocess/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/Users/lsir/.local/lib/python3.7/site-packages/multiprocess/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/lsir/.local/lib/python3.7/site-packages/pathos/helpers/mp_helper.py", line 15, in
func = lambda args: f(*args)
File "/Users/lsir/.local/lib/python3.7/site-packages/pandarallel/dataframe.py", line 19, in worker_apply
client = plasma.connect(plasma_store_name)
File "pyarrow/_plasma.pyx", line 805, in pyarrow._plasma.connect
File "pyarrow/error.pxi", line 87, in pyarrow.lib.check_status
pyarrow.lib.ArrowIOError: Encountered unexpected EOF

[lsircc]

The current workaround is to increase every loop: pandarallel.initialize()

[CookieChief]

@lsircc what do you mean by "increase every loop"? What needs to be increased?

[lsircc]

@CookieChief
Sorry. Machine translation is not accurate. My original intent is to add a line to each function: pandarallel.initialize(). That is, each function is initialized before it runs, so that only warnings are guaranteed. thank you for your reply. The biggest question now is how to deal with the warning? Thank you

WARNING: Logging before InitGoogleLogging() is written to STDERR
E0829 04:54:52.643167 3059774336 io.cc:168] Connection to IPC socket failed for pathname /var/folders/sp/vz74h1tx3jlb3jqrq__bjwh00000gp/T/pandarallel-32ts0h6r/plasma_sock, retrying 20 more times

[lsircc]

@CookieChief pandarallel.initialize(shm_size_mb=1024,progress_bar=False, verbose=0)

WARNING: Logging before InitGoogleLogging() is written to STDERR
E0829 04:54:52.643167 3059774336 io.cc:168] Connection to IPC socket failed for pathname /var/folders/sp/vz74h1tx3jlb3jqrq__bjwh00000gp/T/pandarallel-32ts0h6r/plasma_sock, retrying 20 more times

[lsircc]

@CookieChief My version:
Python 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda custom (64-bit) on darwin
MACOS 10.13.6 Thank.

[nalepae]

I'm currently developing a new version of Pandarallel without PyArrow Plasma (which seems to be the cause of your bug).

This version is not yet relased, but you can already try it by:

• Cloning this git repository on your computer
• Switching to develop branch
• Running $ pip install . (with the dot after the install)

By default, this version of Pandarallel will try to use /dev/shm to transfer data between the main process and the workers. If you don't want to use this feature but you prefer to use standard multiprocessing tranfert feature (pipe), you can disable it by passing use_memory_fs_is_available=False in the initialize method.

See the docstring of initialize for more information.

Note the shm_size_mb parameter of initialize is now deprecated since Pandarallel don't use PyArrow anymore.

In you chosoe to use /dev/shm to transfer data between the main process and the workers and you got a memory error, you can either:

• Increase the size of this partition, thanks to this link, or
• Use standard multiprocessing tranfert feature (pipe), by passing use_memory_fs_is_available=False in the initialize method.

Note that I did not re-implemented (yet) progress bar and verbosity option in this version.

To remove this develop version and retrieve the official one:

• $ pip uninstall pandarallel
• $ pip install pandarallel

Please let me know if you encounter a new bug or if it works better now.

Regards,

Manu

[nalepae]

Fixed with pandarallel 1.4.0

It seems your bug comes from the usage of pyarrow plasma.

pandarallel 1.4.0 does not use pyarrow plasma any more.

[lsircc]

Thank you! let me try.

[lsircc]

Thank you.Very Good.