TypeError when using Cython

[Alexander-0x80]

Hi , i am using Cython to write my bottle based application .
inside my application i serve static files like this :

@controller.route('/Apps/filepath:path')
def server_static(filepath):
return static_file(filepath,root='%s/Apps' %myPath)

When i run this code as Python code , static files being served as intended ,
but when i compile my project to Cython code , and try to access a static file
i get following error :

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/bottle.py", line 763, in _handle
return route.call(*_args)
File "/usr/local/lib/python2.7/dist-packages/bottle.py", line 1572, in wrapper
rv = callback(_a, **ka)
TypeError: server_static() takes no keyword arguments

Note : i am compiling only my modules to Cython , Bottle module is not touched .
Also everything else works as it should .. only accessing static files rises the exception.

Python 2.7 Linux , Bottle 11.4 , Cython 0.16

[defnull]

Bottle passes route wildcards as keyword arguments. Cython seems to be incompatible to Python in this regard as it does not allow to address positional arguments by name. You can work around this by changing the function signature to server_static(filepath=None).

[tzickel]

This is actually an incorrect assertion, and there is a quite simply fix.

By default, Cython compiles functions with 0 or 1 arguments, as special PyCFunction METH_O or METH_NOARGS. This functions do not accept keyword arguments.

You can tell Cython to disable this optimization by changing the always_allow_keywords compiler directive to True (you can do it per function, per file or globally, check cython's documentation on how to do it).

This issue happens actually in all web frameworks who use tricks like this.

[defnull]

This issue happens actually in all web frameworks who use tricks like this.

It's not a trick, its how Python is supposed to work.

>>> def test(x):
...     print(x)
>>> test(5)
5
>>> test(x=5)
5
>>> test(**{'x':5})
5

See https://docs.python.org/2/reference/expressions.html#calls (same for 3.x)
"If keyword arguments are present, they are first converted to positional arguments, as follows. [...] Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). [...] When all arguments have been processed, the slots that are still unfilled are filled with the corresponding default value from the function definition. [...]"

If Cython optimizes some functions in a way that they no longer accept keyword arguments, then it behaves differently than CPython or the official Python documentation and is in fact incompatible to standard Python. Thats not surprising, tough. Cython does not claim to be a full Python implementation.

[tzickel]

First, let me say that the reason I've commented here, is when I googled this issue (happened to me with another web framework) , this was the first hit.

Second, CPython itself does this trick a lot in the name of optimization as well (which is why Cython allows itself to do it as well).

Your statement is wrong, your are quoting the Python Language Reference, but CPython has another side to it, the Python C API Reference which allows you to do wonderful and weird things in the name of speed and C interoperability. You can read about METH_O and PyCFunction there:
https://docs.python.org/2/c-api/structures.html

>>> sum([1,2])
3
>>> sum(**{'x': [1, 2]})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: sum() takes no keyword arguments

Why is this ? Because lots of builtin functions are defined as METH_O PyCFunction:
https://github.com/python/cpython/blob/2.7/Python/bltinmodule.c#L2614

in 2.7, METH_O functions cannot be passed keyword arguments by the CPython interpreter itself:
https://github.com/python/cpython/blob/2.7/Objects/methodobject.c#L123

Cython is all about compiling code to Python C API, and by default gives you the METH_O optimization. As I've written before, it also allows you (by a flag) to easily disable that optimization:
always_allow_keywords (True / False)
"Avoid the METH_NOARGS and METH_O when constructing functions/methods which take zero or one arguments. Has no effect on special methods and functions with more than one argument. The METH_NOARGS and METH_O signatures provide faster calling conventions but disallow the use of keywords."

You would be hard pressed (asides from compiler bugs) to find python code which does not compile under Cython:
https://github.com/cython/cython/wiki/FAQ#is-cython-a-python-implementation

[defnull]

We are going off-topic now, but the original issue is resolved and closed, so why not.

https://docs.python.org/2/reference/expressions.html#calls
"CPython implementation detail: An implementation may provide built-in functions whose positional parameters do not have names, even if they are ‘named’ for the purpose of documentation, and which therefore cannot be supplied by keyword."

As you can see, the METH_O optimization is only allowed for built-ins, not for user-defined functions.

Some third party CPython modules may also violate this rule, because the C-API does not prevent it. But only because you can do something with an implementation dependent C-API, it is not automatically part of the language specification. The Python language is defined by the language reference, not by the capabilities of the CPython C-API.

My statements are: 1) Cython in its default configuration violates this specific part of the Python language specification. 2) Bottle is not relying on undefined behavior or implementation details, and is fully compatible with conforming implementations of the python language. This issue is not a bug in Bottle.

I'll say it again: This is totally fine. Cython does not claim to be 100% compatible (yet). It's a behavior-changing optimization that can be disabled. Nothing wrong with that. I just find it inappropriate to blame library authors for using well documented language features, just because Cython stumbles over them. This discussion should happen in a Cython mailing-list or issue thread, not here.