rpy2 raises RecursionError on simple vector/list consumption

[sungmincs]

Describe the issue or bug
In rpy2 3.5.0, even very simple vector/list input to R magic fails to run with RecursionError
Tested the following in the same environment just with different rpy2 versions

Python 3.7.12 (default, Apr  4 2022, 22:10:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.32.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %pip show rpy2
Name: rpy2
Version: 3.5.0
Summary: Python interface to the R language (embedded R)
Home-page: https://rpy2.github.io
Author: Laurent Gautier
Author-email: lgautier@gmail.com
License: GPLv2+
Location: /apps/pyenv/versions/3.7.12/lib/python3.7/site-packages
Requires: tzlocal, pytz, jinja2, cffi, typing-extensions
Required-by:
Note: you may need to restart the kernel to use updated packages.

In [2]: %load_ext rpy2.ipython

In [3]: %R version[['version.string']]
Out[3]:
<rpy2.robjects.vectors.StrVector object at 0x7fc843696730> [RTYPES.STRSXP]
R classes: ('character',)
['R version 4.1.3 (2022-03-10)']

In [4]: num_list=[1,2]
   ...: %R -i num_list
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-4-53533f69f4d6> in <module>
      1 num_list=[1,2]
----> 2 get_ipython().run_line_magic('R', '-i num_list')

/apps/python3/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2405                 kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2406             with self.builtin_trap:
-> 2407                 result = fn(*args, **kwargs)
   2408             return result
   2409

/apps/python3/lib/python3.7/site-packages/decorator.py in fun(*args, **kw)
    230             if not kwsyntax:
    231                 args, kw = fix(args, kw, sig)
--> 232             return caller(func, *(extras + args), **kw)
    233     fun.__name__ = func.__name__
    234     fun.__doc__ = func.__doc__

/apps/python3/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188
    189         if callable(arg):

/apps/python3/lib/python3.7/site-packages/rpy2/ipython/rmagic.py in R(self, line, cell, local_ns)
    739                         raise NameError("name '%s' is not defined" % input)
    740                 with localconverter(converter) as cv:
--> 741                     ro.r.assign(input, val)
    742
    743         if args.display:

/apps/python3/lib/python3.7/site-packages/rpy2/robjects/functions.py in __call__(self, *args, **kwargs)
    200                 kwargs[r_k] = v
    201         return (super(SignatureTranslatedFunction, self)
--> 202                 .__call__(*args, **kwargs))
    203
    204

/apps/python3/lib/python3.7/site-packages/rpy2/robjects/functions.py in __call__(self, *args, **kwargs)
    123                 new_kwargs[k] = conversion.py2rpy(v)
    124         res = super(Function, self).__call__(*new_args, **new_kwargs)
--> 125         res = conversion.rpy2py(res)
    126         return res
    127

/apps/python3/lib/python3.7/functools.py in wrapper(*args, **kw)
    838                             '1 positional argument')
    839
--> 840         return dispatch(args[0].__class__)(*args, **kw)
    841
    842     funcname = getattr(func, '__name__', 'singledispatch function')

/apps/python3/lib/python3.7/site-packages/rpy2/robjects/__init__.py in _rpy2py_listsexp(obj)
    230     clsmap = conversion.converter.rpy2py_nc_name[rinterface.ListSexpVector]
    231     cls = clsmap.find(obj.rclass)
--> 232     return cls(obj)
    233
    234

/apps/python3/lib/python3.7/functools.py in wrapper(*args, **kw)
    838                             '1 positional argument')
    839
--> 840         return dispatch(args[0].__class__)(*args, **kw)
    841
    842     funcname = getattr(func, '__name__', 'singledispatch function')

/apps/python3/lib/python3.7/site-packages/rpy2/robjects/numpy2ri.py in rpy2py_sexp(obj)
    188             res[res == rinterface.NA_Character] = None
    189     else:
--> 190         res = ro.default_converter.rpy2py(obj)
    191     return res
    192

... last 4 frames repeated, from the frame below ...

/apps/python3/lib/python3.7/functools.py in wrapper(*args, **kw)
    838                             '1 positional argument')
    839
--> 840         return dispatch(args[0].__class__)(*args, **kw)
    841
    842     funcname = getattr(func, '__name__', 'singledispatch function')

RecursionError: maximum recursion depth exceeded in comparison

This works just fine in 3.4.5

Python 3.7.12 (default, Apr  4 2022, 22:10:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.32.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %pip show rpy2
Name: rpy2
Version: 3.4.5
Summary: Python interface to the R language (embedded R)
Home-page: https://rpy2.github.io
Author: Laurent Gautier
Author-email: lgautier@gmail.com
License: GPLv2+
Location: /apps/pyenv/versions/3.7.12/lib/python3.7/site-packages
Requires: tzlocal, cffi, jinja2, pytz
Required-by:
Note: you may need to restart the kernel to use updated packages.

In [2]: %load_ext rpy2.ipython

In [3]: %R version[['version.string']]
Out[3]:
<rpy2.robjects.vectors.StrVector object at 0x7fd660619b90> [RTYPES.STRSXP]
R classes: ('character',)
['R version 4.1.3 (2022-03-10)']

In [4]: num_list=[1,2]
   ...: %R -i num_list

In [5]:

To Reproduce
Try to pass any list to R magic.
or even weirdly,

%R version

yields the same failure as version output is a vector

Expected behavior
R magic should work with simple list/vector

Error
attached above.

Additional context
Add any other context about the problem here.

[bakaleks]

Looks like it has something to do with 3.5.0 changes inside pandas2ri.conversion. Here is a pure Python script reproducing the problem:

import rpy2.robjects as ro
from rpy2.robjects import pandas2ri
from rpy2.robjects.conversion import localconverter

with localconverter(ro.default_converter + pandas2ri.converter):
    vec = ro.r("list('a', 'b')")
    print(vec)

What I see from my debugging is that earlier in 3.4.5 pandas2ri converter used singledispatch mechanism to handle ListSexpVector, which in turn called numpy2ri converter that bounced back to default_converter and through _rpy2py_nc_map mechanism the ListVector implementation was used in the end.

Now in 3.5.0 pandas2ri converter uses _rpy2py_nc_map to register handler for ListSexpVector instead of singledispatch. That handler calls numpy2ri and then goes back to default_converter. However now the default_converter ListSexpVector handler sees the _rpy2py_nc_map of the overlayed pandas2ri which points again to numpy2ri, and here you have infinite recursion.

[lgautier]

Thanks for the report and looking for the source of the error. This is fixed in the dev branch and will be included in the bugfix release 3.5.1 (to happen by tomorrow).

[bakaleks]

It did not seem to solve the issue with robjects.r(). Please run my code that I posted earlier to reproduce

[lgautier]

The issue still seems to be present.

[lgautier]

After checking, it seems that the issue was fixed for the ipython converter only.

[lgautier]

This appears fixed in current dev branch (future 3.5.3). Closing, hopefully for good.