Segfault when unpickling expression containing user-defined formal symbolic function

[YunosukeY]

sage: loads(dumps(function('f')(x)))                                                                                                                           
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/s/sage/sage-rebasing/worktree-gcc11/src/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join(olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.

As without #32480, deepcopy goes through pickling/unpickling, this also causes a failure of deepcopy as originally reported:

When I deepcopy expression containing function, a segmentation fault occurred.

sage: var('t');
sage: x = function('x')(t);
sage: l = [[x(t) == 1]];
sage: l
[[x(t) == 1]]
sage: l2 = deepcopy(l);
sage: l2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Applications/SageMath-9.1.app/Contents/Resources/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join(olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Applications/SageMath-9.1.app/Contents/Resources/sage/src/bin/sage-python: line 2: 14997 Segmentation fault: 11  sage -python "$@"

Is this known issue?

Depends on #32386

CC: @tscrim @rburing

Component: symbolics

Issue created by migration from https://trac.sagemath.org/ticket/30018

[seblabbe]

comment:1

Doing

diff --git a/src/sage/libs/pynac/pynac.pyx b/src/sage/libs/pynac/pynac.pyx
index 0ebc8b5..b87e94e 100644
--- a/src/sage/libs/pynac/pynac.pyx
+++ b/src/sage/libs/pynac/pynac.pyx
@@ -490,7 +490,7 @@ def py_print_function_pystring(id, args, fname_paren=False):
     else:
         olist = [func._name]
     olist.extend(['(', ', '.join(map(repr, args)), ')'])
-    return ''.join(olist)
+    return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
 
 cdef stdstring* py_print_function(unsigned id, args):
     return string_from_pystr(py_print_function_pystring(id, args))

allows to fix the issue, but I don't know if this is a desirable fix. Maybe something else is better to do when the copy is done instead. But I don't know where this is done.

[YunosukeY]

comment:2

Replying to @seblabbe:

Doing

diff --git a/src/sage/libs/pynac/pynac.pyx b/src/sage/libs/pynac/pynac.pyx
index 0ebc8b5..b87e94e 100644
--- a/src/sage/libs/pynac/pynac.pyx
+++ b/src/sage/libs/pynac/pynac.pyx
@@ -490,7 +490,7 @@ def py_print_function_pystring(id, args, fname_paren=False):
     else:
         olist = [func._name]
     olist.extend(['(', ', '.join(map(repr, args)), ')'])
-    return ''.join(olist)
+    return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
 
 cdef stdstring* py_print_function(unsigned id, args):
     return string_from_pystr(py_print_function_pystring(id, args))

allows to fix the issue, but I don't know if this is a desirable fix. Maybe something else is better to do when the copy is done instead. But I don't know where this is done.

Thanks for your comment.

But it doesn't seem to work in my environment.

sage: var('t');
sage: x = function('x')(t);
sage: l = [[x(t) == 1]];
sage: l2 = deepcopy(l);
sage: l2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/Applications/SageMath-9.1.app/Contents/Resources/sage/local/lib/python3.7/site-packages/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)()
    491         olist = [func._name]
    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
--> 493     return ''.join((a.decode('utf-8') if isinstance(a,bytes) else a) for a in olist)
    494 
    495 cdef stdstring* py_print_function(unsigned id, args):

TypeError: sequence item 0: expected str instance, bytes found
Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
Traceback (most recent call last):
  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring (build/cythonized/sage/libs/pynac/pynac.cpp:7182)
TypeError: sequence item 0: expected str instance, bytes found
------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Applications/SageMath-9.1.app/Contents/Resources/sage/src/bin/sage-python: line 2: 16125 Segmentation fault: 11  sage -python "$@"

[YunosukeY]

comment:3

Replying to @seblabbe:

I noticed that the file is in Cython and tried to make it, but it failed.

...(I omitted)...
[sagelib-9.1] build/cythonized/sage/tests/stl_vector.cpp: In function 'PyObject* __pyx_pf_4sage_5tests_10stl_vector_14stl_int_vector_4__getitem__(__pyx_obj_4sage_5tests_10stl_vector_stl_int_vector*, int)':
[sagelib-9.1] build/cythonized/sage/tests/stl_vector.cpp:2892:30: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<int>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
[sagelib-9.1]  2892 |       __pyx_t_1 = (__pyx_v_i < __pyx_v_self->data->size());
[sagelib-9.1]       |                    ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
[sagelib-9.1] error: command 'gcc' failed with exit status 1
[sagelib-9.1] make[4]: *** [sage] Error 1
[sagelib-9.1] 
[sagelib-9.1] real      3m48.105s
[sagelib-9.1] user      2m58.864s
[sagelib-9.1] sys       0m23.285s
make[3]: *** [sagelib] Error 2
make[2]: *** [all-start] Error 2

real    3m48.678s
user    2m59.280s
sys     0m23.488s
***************************************************************
Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make all-start'):

It is safe to delete any log files and build directories, but they
contain information that is helpful for debugging build problems.
WARNING: If you now run 'make' again, the build directory of the
same version of the package will, by default, be deleted. Set the
environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this.

make[1]: *** [all-start] Error 1
make: *** [all] Error 2

[seblabbe]

comment:4

Maybe it depends on the way you installed sage. For example, if you installed sage from a binary download, you may not have all the tools to compile such changes... The README of Sage gives information on the prerequisite necessary to compile sage.

[mkoeppe]

comment:7

Moving this ticket to 9.4, as it seems unlikely that it will be merged in 9.3, which is in the release candidate stage

[mkoeppe]

comment:9

still broken in 9.5.beta0; #32480 will fix it

[mkoeppe]

Dependencies: #32480

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,31 @@
+
+```
+sage: var('t')                                                                                                                                                 
+t
+sage: x = function('x')(t)                                                                                                                                     
+sage: l = [[x == 1]]                                                                                                                                           
+sage: loads(dumps(l))                                                                                                                                          
+TypeError                                 Traceback (most recent call last)
+~/s/sage/sage-rebasing/worktree-gcc11/src/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring()
+    491         olist = [func._name]
+    492     olist.extend(['(', ', '.join(map(repr, args)), ')'])
+--> 493     return ''.join(olist)
+    494 
+    495 cdef stdstring* py_print_function(unsigned id, args):
+
+TypeError: sequence item 0: expected str instance, bytes found
+Exception ignored in: 'sage.libs.pynac.pynac.py_print_function'
+Traceback (most recent call last):
+  File "sage/libs/pynac/pynac.pyx", line 493, in sage.libs.pynac.pynac.py_print_function_pystring
+TypeError: sequence item 0: expected str instance, bytes found
+------------------------------------------------------------------------
+(no backtrace available)
+------------------------------------------------------------------------
+Unhandled SIGSEGV: A segmentation fault occurred.
+```
+
+As without #32480, `deepcopy` goes through pickling/unpickling, this also causes a failure of `deepcopy` as originally reported:
+
 When I deepcopy expression containing function, a segmentation fault occurred.
 
 ```

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,10 +1,7 @@
 
 ```
-sage: var('t')                                                                                                                                                 
-t
-sage: x = function('x')(t)                                                                                                                                     
-sage: l = [[x == 1]]                                                                                                                                           
-sage: loads(dumps(l))                                                                                                                                          
+sage: loads(dumps(function('f')(x)))                                                                                                                           
+---------------------------------------------------------------------------
 TypeError                                 Traceback (most recent call last)
 ~/s/sage/sage-rebasing/worktree-gcc11/src/sage/libs/pynac/pynac.pyx in sage.libs.pynac.pynac.py_print_function_pystring()
     491         olist = [func._name]

[mkoeppe]

comment:12

However, the underlying issue is that unpickling gives a segfault -- this needs to be fixed.

[mkoeppe]

Changed dependencies from #32480 to #32386