Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

10.0.2 seg fault with rpy2 #434

Closed
yang opened this issue May 13, 2011 · 8 comments
Closed

10.0.2 seg fault with rpy2 #434

yang opened this issue May 13, 2011 · 8 comments
Milestone
old

Comments

@yang
Copy link

yang commented May 13, 2011

When I tried out RPy2 (either 2.1 or 2.2) from ipython 10.0.2, I get a seg fault, but this doesn't occur with ipython 10.0. Details below. Any ideas what's going on? Thanks in advance.

$ R --version
R version 2.13.0 (2011-04-13)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License version 2.
For more information about these matters see
http://www.gnu.org/licenses/.

$ pip install 'rpy2==2.1.9'
/home/yang/env/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/req.py:641:
UserWarning: Unbuilt egg for mining [unknown version]
(/home/yang/pod/mining)
 self.satisfied_by = pkg_resources.get_distribution(self.req)
Downloading/unpacking rpy2==2.1.9
 Downloading rpy2-2.1.9.tar.gz (129Kb): 129Kb downloaded
 Running setup.py egg_info for package rpy2
   Configuration for R as a library:
     include_dirs: ('/usr/share/R/include',)
     libraries: ('lapack', 'blas', 'R')
     library_dirs: ('/usr/lib64/R/lib',)
     extra_link_args: ()
    # OSX-specific (included in extra_link_args)
     framework_dirs: ()
     frameworks: ()
Installing collected packages: rpy2
 Running setup.py install for rpy2
   Configuration for R as a library:
     include_dirs: ('/usr/share/R/include',)
     libraries: ('lapack', 'blas', 'R')
     library_dirs: ('/usr/lib64/R/lib',)
     extra_link_args: ()
    # OSX-specific (included in extra_link_args)
     framework_dirs: ()
     frameworks: ()
   building 'rpy2.rinterface.rinterface' extension
   gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -DR_INTERFACE_PTRS=1 -DHAVE_POSIX_SIGJMP=1
-DCSTACK_DEFNS=1 -DRIF_HAS_RSIGHAND=1 -Irpy/rinterface
-I/usr/include/python2.6 -I/usr/share/R/include -c
rpy/rinterface/rinterface.c -o
build/temp.linux-x86_64-2.6/rpy/rinterface/rinterface.o
   In file included from /usr/include/python2.6/Python.h:8,
                    from rpy/rinterface/rinterface.c:55:
   /usr/include/python2.6/pyconfig.h:1031:1: warning:
"_POSIX_C_SOURCE" redefined
   In file included from /usr/include/signal.h:29,
                    from rpy/rinterface/rinterface.c:51:
   /usr/include/features.h:210:1: warning: this is the location of
the previous definition
   In file included from rpy/rinterface/rinterface.c:62:
   /usr/share/R/include/Rinternals.h:657: warning: function
declaration isn’t a prototype
   In file included from rpy/rinterface/rinterface.c:74:
   /usr/share/R/include/R_ext/Rdynload.h:26: warning: function
declaration isn’t a prototype
   In file included from rpy/rinterface/rinterface.c:105:
   rpy/rinterface/sequence.c: In function ‘VectorSexp_ass_slice’:
   rpy/rinterface/sequence.c:447: warning: unused variable ‘sexp_item’
   rpy/rinterface/sequence.c:447: warning: unused variable ‘tmp’
   rpy/rinterface/sequence.c:446: warning: unused variable ‘vs’
   rpy/rinterface/sequence.c:386: warning: unused variable ‘self_typeof’
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_end’:
   rpy/rinterface/rinterface.c:1112: warning: unused variable ‘str’
   rpy/rinterface/rinterface.c: In function ‘EnvironmentSexp_ass_subscript’:
   rpy/rinterface/rinterface.c:2001: warning: unused variable ‘sexp_copy’
   rpy/rinterface/rinterface.c: At top level:
   rpy/rinterface/rinterface.c:2775: warning: ‘externalMethods’
defined but not used
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_ShowFiles’:
   rpy/rinterface/rinterface.c:654: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_FlushConsole’:
   rpy/rinterface/rinterface.c:524: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_ChooseFile’:
   rpy/rinterface/rinterface.c:568: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_ReadConsole’:
   rpy/rinterface/rinterface.c:424: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_WriteConsole’:
   rpy/rinterface/rinterface.c:288: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_ShowMessage’:
   rpy/rinterface/rinterface.c:356: warning: ‘gstate’ may be used
uninitialized in this function
   rpy/rinterface/rinterface.c: In function ‘EmbeddedR_CleanUp’:
   rpy/rinterface/rinterface.c:787: warning: ‘gstate’ may be used
uninitialized in this function
   gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/rpy/rinterface/rinterface.o
-L/usr/lib64/R/lib -L/usr/lib64/R/modules -lR -llapack -lblas -lR -o
build/lib.linux-x86_64-2.6/rpy2/rinterface/rinterface.so
   building 'rpy2.rinterface.rpy_device' extension
   gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -DR_INTERFACE_PTRS=1 -DHAVE_POSIX_SIGJMP=1
-DCSTACK_DEFNS=1 -DRIF_HAS_RSIGHAND=1 -Irpy/rinterface
-I/usr/include/python2.6 -I/usr/share/R/include -c
rpy/rinterface/rpy_device.c -o
build/temp.linux-x86_64-2.6/rpy/rinterface/rpy_device.o
   In file included from rpy/rinterface/rpy_device.c:6:
   /usr/share/R/include/Rinternals.h:657: warning: function
declaration isn’t a prototype
   rpy/rinterface/rpy_device.c:690: warning: ‘GrDev_clear’ defined but not used
   gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/rpy/rinterface/rpy_device.o
-L/usr/lib64/R/lib -L/usr/lib64/R/modules -lR -llapack -lblas -lR -o
build/lib.linux-x86_64-2.6/rpy2/rinterface/rpy_device.so
Successfully installed rpy2

$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rpy2.robjects
>>>

$ ipython
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import rpy2.robjects
Segmentation fault

$ pip install --upgrade 'rpy2'
Downloading/unpacking rpy2
 Downloading rpy2-2.2.0beta3.tar.gz (164Kb): 164Kb downloaded
 Running setup.py egg_info for package rpy2
   Configuration for R as a library:
     include_dirs: ('/usr/share/R/include',)
     libraries: ('lapack', 'blas', 'R')
     library_dirs: ('/usr/lib64/R/lib',)
     extra_link_args: ()
    # OSX-specific (included in extra_link_args)
     framework_dirs: ()
     frameworks: ()
Installing collected packages: rpy2
 Found existing installation: rpy2 2.1.9-20110510
   Uninstalling rpy2:
     Successfully uninstalled rpy2
 Running setup.py install for rpy2
   Configuration for R as a library:
     include_dirs: ('/usr/share/R/include',)
     libraries: ('lapack', 'blas', 'R')
     library_dirs: ('/usr/lib64/R/lib',)
     extra_link_args: ()
    # OSX-specific (included in extra_link_args)
     framework_dirs: ()
     frameworks: ()
   building 'rpy2.rinterface._rinterface' extension
   gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -DR_INTERFACE_PTRS=1 -DHAVE_POSIX_SIGJMP=1
-DCSTACK_DEFNS=1 -DRIF_HAS_RSIGHAND=1 -I./rpy/rinterface
-I/usr/include/python2.6 -I/usr/share/R/include -c
./rpy/rinterface/_rinterface.c -o
build/temp.linux-x86_64-2.6/./rpy/rinterface/_rinterface.o
   In file included from /usr/include/python2.6/Python.h:8,
                    from ./rpy/rinterface/_rinterface.c:55:
   /usr/include/python2.6/pyconfig.h:1031:1: warning:
"_POSIX_C_SOURCE" redefined
   In file included from /usr/include/signal.h:29,
                    from ./rpy/rinterface/_rinterface.c:51:
   /usr/include/features.h:210:1: warning: this is the location of
the previous definition
   In file included from ./rpy/rinterface/_rinterface.c:63:
   /usr/share/R/include/Rinternals.h:657: warning: function
declaration isn’t a prototype
   In file included from ./rpy/rinterface/_rinterface.c:74:
   /usr/share/R/include/R_ext/Rdynload.h:26: warning: function
declaration isn’t a prototype
   In file included from ./rpy/rinterface/_rinterface.c:122:
   ./rpy/rinterface/sequence.c: In function ‘VectorSexp_ass_slice’:
   ./rpy/rinterface/sequence.c:542: warning: unused variable ‘sexp_item’
   ./rpy/rinterface/sequence.c:542: warning: unused variable ‘tmp’
   ./rpy/rinterface/sequence.c:541: warning: unused variable ‘vs’
   ./rpy/rinterface/sequence.c:478: warning: unused variable ‘self_typeof’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToINTSXP’:
   ./rpy/rinterface/sequence.c:1193: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1202: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_IterToINTSXP’:
   ./rpy/rinterface/sequence.c:1241: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1261: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1270: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToREALSXP’:
   ./rpy/rinterface/sequence.c:1425: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_IterToREALSXP’:
   ./rpy/rinterface/sequence.c:1461: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1474: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToSTRSXP’:
   ./rpy/rinterface/sequence.c:1631: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1661: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToLGLSXP’:
   ./rpy/rinterface/sequence.c:1838: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToRAWSXP’:
   ./rpy/rinterface/sequence.c:1991: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c:1997: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToCPLXSXP’:
   ./rpy/rinterface/sequence.c:2133: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/sequence.c: In function ‘RPy_SeqToVECSXP’:
   ./rpy/rinterface/sequence.c:2330: warning: format ‘%i’ expects
type ‘int’, but argument 3 has type ‘Py_ssize_t’
   ./rpy/rinterface/_rinterface.c:2172:11: warning: extra tokens at
end of #endif directive
   ./rpy/rinterface/_rinterface.c: At top level:
   ./rpy/rinterface/sequence.c:2144: warning:
‘ComplexVectorSexp_AsSexp’ defined but not used
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_ShowFiles’:
   ./rpy/rinterface/_rinterface.c:738: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_FlushConsole’:
   ./rpy/rinterface/_rinterface.c:592: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_ChooseFile’:
   ./rpy/rinterface/_rinterface.c:636: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_ReadConsole’:
   ./rpy/rinterface/_rinterface.c:460: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_WriteConsole’:
   ./rpy/rinterface/_rinterface.c:318: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_ShowMessage’:
   ./rpy/rinterface/_rinterface.c:392: warning: ‘gstate’ may be used
uninitialized in this function
   ./rpy/rinterface/_rinterface.c: In function ‘EmbeddedR_CleanUp’:
   ./rpy/rinterface/_rinterface.c:886: warning: ‘gstate’ may be used
uninitialized in this function
   gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/./rpy/rinterface/_rinterface.o
-L/usr/lib64/R/lib -L/usr/lib64/R/modules -lR -llapack -lblas -lR -o
build/lib.linux-x86_64-2.6/rpy2/rinterface/_rinterface.so
   building 'rpy2.rinterface.rpy_device' extension
   gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -DR_INTERFACE_PTRS=1 -DHAVE_POSIX_SIGJMP=1
-DCSTACK_DEFNS=1 -DRIF_HAS_RSIGHAND=1 -Irpy/rinterface
-I/usr/include/python2.6 -I/usr/share/R/include -c
rpy/rinterface/rpy_device.c -o
build/temp.linux-x86_64-2.6/rpy/rinterface/rpy_device.o
   In file included from rpy/rinterface/rpy_device.c:33:
   /usr/share/R/include/Rinternals.h:657: warning: function
declaration isn’t a prototype
   rpy/rinterface/rpy_device.c:789: warning: ‘GrDev_clear’ defined but not used
   gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.6/rpy/rinterface/rpy_device.o
-L/usr/lib64/R/lib -L/usr/lib64/R/modules -lR -llapack -lblas -lR -o
build/lib.linux-x86_64-2.6/rpy2/rinterface/rpy_device.so
Successfully installed rpy2
Cleaning up...

yang@partycat Tue May 10 16:20:42 ~/pod/sales git(master)
$ ipython
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import rpy2.robjects
Segmentation fault
@fperez
Copy link
Member

fperez commented May 14, 2011

On Fri, May 13, 2011 at 5:44 PM, yang
reply@reply.github.com
wrote:

In [1]: import rpy2.robjects
Segmentation fault

It's very strange: I was able to reproduce the segfault once and
never again. But the execution of the import statement in ipython is
no different from python itself, and we have no C code in ipython
capable of segfaulting directly. So I suspect that this is simply a
problem (likely a race condition) in rpy2 that happens to be triggered
by IPython and not as much by plain python.

Can you reproduce the problem reliably, every time you start ipython?
Try it a few times in succession, both with python and with ipython.
If it's 100% reproducible in ipython and not in plain python we may be
able to make some progress. I suspect it's still an rpy2 bug, but we
might know how to trigger it.

Cheers,

f

@takluyver
Copy link
Member

For info (I'm subscribed to the rpy-dev list as well), this is almost certainly a tricky problem in rpy2 - there are somewhat inconsistent segfaults that seem to trigger on some systems but not others. And where segfaults can be reproduced, they can (at least in some cases) be reproduced outside of IPython.

So I think something IPython is doing is triggering the bug for @yang, but the bug itself is in rpy2.

@fperez
Copy link
Member

fperez commented May 14, 2011

I think you're right, @takluyver, I'm closing this one. Thanks for the info.

@fperez fperez closed this as completed May 14, 2011
@yang
Copy link
Author

yang commented May 15, 2011

I retried a few times and so far it has been 100% in ipython, 0% in python.

@takluyver
Copy link
Member

Other people can trigger segfaults in various situations without IPython (see Artur's t.py script). So my money's certainly on rpy2, at least until those segfaults are fixed.

Also, IPython is 100% Python code, and any segfault indicates a bug in C code - either rpy2, or CPython itself.

@fperez
Copy link
Member

fperez commented May 15, 2011

On Sat, May 14, 2011 at 8:48 PM, yang
reply@reply.github.com
wrote:

I retried a few times and so far it has been 100% in ipython, 0% in python.

I can certainly believe that, but the point we're making is that
IPython is simply triggering the segfault in rpy, not causing it.
IPython loads more modules from the stdib than plain python, including
modules with internal C state like readline, and it's possible that
rpy2 is sensitive to those. But as Thomas said, in ipython proper
there's zero C code, so we can't cause a segfault directly (well,
and we're not doing funky ctypes stuff either). But we can easily
(inadvertently) set up the conditions for one to be triggered.

In a sense, it's great that ipython makes this segfault more likely to
happen: you can help the rpy2 developers fix their bug by having a
semi-reliable way of causing it. They could perhaps force a core dump
with a debug build of rpy2 (that has symbols enabled) to pinpoint
where it's coming from.

We're not trying to dismiss your concern, just to point you in the
direction where effort will be productive. The bug needs to be fixed
in rpy2, where it's actually coming from, so no amount of looking at
ipython is going to help you with that.

Regards,

f

@yang
Copy link
Author

yang commented May 15, 2011

Thanks for both your answers. What you say makes sense. Sorry for sounding like I was trying to blame ipython - just reporting back what I seemed to be finding (more "FWIW"/for posterity than anything). Hopefully the rpy folks can help resolve this.

@fperez
Copy link
Member

fperez commented May 15, 2011

No worries, this is indeed useful information to have on record,
especially for the rpy2 team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
old
Development

No branches or pull requests
3 participants
@yang @fperez @takluyver