get_func_args maximum recursion

[nramirezuy]

https://github.com/scrapy/scrapy/blob/master/scrapy/utils/python.py#L149

Today I was working on a project were I have to skip the first item of a list, and then join the rest. Instead of writing the typical slice I tried something much more good looking Compose(itemgetter(slice(1, None)), Join()) but I found out this maximum recursion. I did some research and ask @dangra about it, but nothing came up.
I think the main problem is that inspect isn't able recognize itemgetter as something.

>>> inspect.getmembers(itemgetter(2))
[('__call__',
  <method-wrapper '__call__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__class__', <type 'operator.itemgetter'>),
 ('__delattr__',
  <method-wrapper '__delattr__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__doc__',
  'itemgetter(item, ...) --> itemgetter object\n\nReturn a callable object that fetches the given item(s) from its operand.\nAfter, f=itemgetter(2), the call f(r) returns r[2].\nAfter, g=itemgetter(2,5,3), the call g(r) returns (r[2], r[5], r[3])'),
 ('__format__',
  <built-in method __format__ of operator.itemgetter object at 0x7f79aeffb990>),
 ('__getattribute__',
  <method-wrapper '__getattribute__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__hash__',
  <method-wrapper '__hash__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__init__',
  <method-wrapper '__init__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__new__', <built-in method __new__ of type object at 0x8c1ec0>),
 ('__reduce__',
  <built-in method __reduce__ of operator.itemgetter object at 0x7f79aeffb990>),
 ('__reduce_ex__',
  <built-in method __reduce_ex__ of operator.itemgetter object at 0x7f79aeffb990>),
 ('__repr__',
  <method-wrapper '__repr__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__setattr__',
  <method-wrapper '__setattr__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__sizeof__',
  <built-in method __sizeof__ of operator.itemgetter object at 0x7f79aeffb990>),
 ('__str__',
  <method-wrapper '__str__' of operator.itemgetter object at 0x7f79aeffb990>),
 ('__subclasshook__',
  <built-in method __subclasshook__ of type object at 0x8c1ec0>)]
>>> inspect.getargspec(itemgetter(2).__call__)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/inspect.py", line 815, in getargspec
    raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <method-wrapper '__call__' of operator.itemgetter object at 0xb3ddd0> is not a Python function
>>> inspect.getargspec(itemgetter(slice(None, 2)).__init__)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python2.7/inspect.py", line 815, in getargspec
    raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <method-wrapper '__init__' of operator.itemgetter object at 0xb3de10> is not a Python function

EDIT: Looks like the reason was C functions weren't covered by inspect module until Python 3.4 (http://bugs.python.org/issue17481)

[dangra]

same results for python2 and python3

>>> ig = itemgetter(0)
>>> [(n, f(ig), f(ig.__call__)) for n, f in vars(inspect).items() if n.startswith('is')]
[('isgenerator', False, False),
 ('isdatadescriptor', False, False),
 ('isframe', False, False),
 ('ismemberdescriptor', False, False),
 ('isabstract', False, False),
 ('isbuiltin', False, False),
 ('isfunction', False, False),
 ('istraceback', False, False),
 ('isgeneratorfunction', False, False),
 ('ismethod', False, False),
 ('iscode', False, False),
 ('isclass', False, False),
 ('ismethoddescriptor', False, False),
 ('isroutine', False, False),
 ('ismodule', False, False),
 ('isgetsetdescriptor', False, False)]

[dangra]

at least we can detect it similar to what we did for partial

>>> isinstance(ig, itemgetter)
True

[nramirezuy]

Do we want to fall more than one time on line https://github.com/scrapy/scrapy/blob/master/scrapy/utils/python.py#L166?
I'm thinking that we can just return an empty list the second time we reach that point and avoid the recursion completely.

[dangra]

Do we want to fall more than one time on line https://github.com/scrapy/scrapy/blob/master/scrapy/utils/python.py#L166?
I'm thinking that we can just return an empty list the second time we reach that point and avoid the recursion completely.

+1 I can't imagine a better solution