Segfault with closure in setCompleteCallback

[brstgt]

Unfortunately it is very very hard to reproduce but I have a case when it consistently crashes. I am not able to post that complete case but I can provide some snippets. Moving the captured local referenced variable to a class property helped.

Use case: A bunch of images is resized in several gearman jobs that run in parallel. PHP client waits for return.

This example works:
http://pastebin.com/VCqRqd6Q

This example crashes with segfault in PHP shutdown:
http://pastebin.com/EPZ9sMgB

We could track down the segfault by core dumps to this:
#0 zend_string_release (s=0x1) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_string.h:269

269 /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_string.h: No such file or directory.
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in
from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
(gdb) bt
#0 zend_string_release (s=0x1) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_string.h:269
#1 zend_array_destroy (ht=0x7f27f5bca498) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_hash.c:1328
#2 0x00007f2811478406 in _zval_dtor_func_for_ptr (p=) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_variables.c:96
#3 0x00007f28114ad304 in i_zval_ptr_dtor (zval_ptr=0x7f27f5bbba58) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_variables.h:58
#4 zend_object_std_dtor (object=0x7f27f5bbba00) at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_objects.c:69
#5 0x00007f28114b1890 in zend_objects_store_free_object_storage (objects=0x1, objects@entry=0x7f281185a530 <executor_globals+816>)

at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_objects_API.c:103
#6 0x00007f281146b403 in shutdown_executor () at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend_execute_API.c:357
#7 0x00007f2811479cb5 in zend_deactivate () at /build/php7.0-ggjBrm/php7.0-7.0.4/Zend/zend.c:967
#8 0x00007f281141ca41 in php_request_shutdown (dummy=) at /build/php7.0-ggjBrm/php7.0-7.0.4/main/main.c:1823
#9 0x00007f281130f986 in main (argc=, argv=) at /build/php7.0-ggjBrm/php7.0-7.0.4/sapi/fpm/fpm/fpm_main.c:1996

Hope that helps :)

[brstgt]

I am sorry for these kind of confusing tickets but the situation is also confusing for me.
Now I had a different use case where the above "fix" did not work and the request segfaults again. It still segfaults somewhere in the request finishing process. The gearman work is done but the request does not finish.

I now have a workaround that seems to "fix" both cases. When I do not reuse the gearman client that lives throughout the whole request and is GC'ed at the end of the script but instead create a new one and destroy it immediately after having done the work, the request finished. This looks like this:

http://pastebin.com/ju37wZMx

As you already mentioned it is very likely that "all these crashes" belong together. This very likely relates to my other ticket but shows a different use case.

[rosmo]

@brstgt, Can you try reverting this commit and try: 4423da6

[brstgt]

Tested and works.

What else I can tell:

• Without that fix, creating (better destroying) many GearmanClient creates SEGV
• With that fix no SEGV occur by creating many GearmanClient objects

But there must still be something wrong with the callbacks.
Even if I reuse the client object and only set the complete callback for every run, that creates segfaults.
Core dump says it crashed in "call_user_function_ex" indicating it comes from complete callback (only callback that was set).
When I also reuse the complete callback and only set it once, there are no crashes any more.

[rosmo]

@brstgt, Can you try this branch: https://github.com/rosmo/pecl-gearman/tree/task-callback-fix ? (includes reverting the commit I mentioned)

[brstgt]

Hi @rosmo,
Sure I can but at the moment we had to roll back to PHP5 for a different reason.
I think I will have time to test next week.

Is that ok?

JFYI: My own patch works quite stable, that means I commented out the ZVAL_DTOR(...->zClient) in task free and the efree(intern) in client free

[wcgallego]

Hey All,

I've merged @rosmo's patches (thanks again!). The included tests are passing and any ad-hoc ones i've put together to try to suss out the issues seems solid as well. Was worried about a memory leak but I think we're ok there!

Please give the most recent build a shot and let me know if you're still experiencing issues.

[acasademont]

Going to try it asap, thank you all!

[acasademont]

@wcgallego no segfaults on our testsuite with the new master, yay!

[brstgt]

Yes, works quite well, but there is another issue. The Worker segfaults if an Exception is thrown and not caught in the worker callback. I will open another issue for that.