Unable to pass argument with space to cell magic

[memeplex]

Cython defines this argument for the %%cython margin:

    @magic_arguments.argument(
        '--link-args', action='append', default=[],
        help="Extra flags to pass to linker via the `extra_link_args` "
             "Extension flag (can be specified  multiple times)."
    )

But trying to pass %%cython --link-args="-L/Users/carlos/Desktop/Test LGBM API" I get:

UsageError: unrecognized arguments: LGBM API"

If I mimic what I believe this should be doing, I get:

EDIT: there is a typo in this code, please see comments below.

In [15]: l = shlex.shlex('%%cython --link-args="-L/Users/carlos/Desktop/Test LGBM API"', posix=False)

In [16]: a.whitespace_split=True

In [17]: a.commenters=''

In [18]: list(l)
Out[18]:
['%',
 '%',
 'cython',
 '-',
 '-',
 'link',
 '-',
 'args',
 '=',
 '"-L/Users/carlos/Desktop/Test LGBM API"']

which seems fine.

My system is MacOS 11.0.1. I'm using python 3.8.6 with ipython 7.19.0.

[memeplex]

Well, debugging arg_split shows that it is indeed splitting the argument:

ipdb>  p list(lex)
['--link-args="-L/Users/carlos/Desktop/Test', 'LGBM', 'API"']

I have posix = False and strict = True. The problem is with whitespace_split = True. I didn't catch it above because of a typo (used a which was in my history instead of l).

Now, this seems excessive to me, why are you splitting in every whitespace disregarding quoting? If this is on purpose, how can I introduce whitespace in my arguments?

[memeplex]

Ok, I think it's not that serious, the problem is caused because of the = before the ", the lexer somehow gets confused by this:

In [22]: s = shlex.shlex('--link-args "-L/Users/carlos/Desktop/Test LGBM API"', posix=False)

In [23]: s.whitespace_split = True

In [24]: list(s)
Out[24]: ['--link-args', '"-L/Users/carlos/Desktop/Test LGBM API"']

In [25]: s = shlex.shlex('--link-args="-L/Users/carlos/Desktop/Test LGBM API"', posix=False)

In [26]: s.whitespace_split = True

In [27]: list(s)
Out[27]: ['--link-args="-L/Users/carlos/Desktop/Test', 'LGBM', 'API"']

The documentation states

When operating in non-POSIX mode, shlex will try to obey to the following rules.

• Quote characters are not recognized within words (Do"Not"Separate is parsed as the single word Do"Not"Separate);
• Closing quotes separate words ("Do"Separate is parsed as "Do" and Separate);

Indeed in POSIX mode the behavior is more sensible:

In [28]: s = shlex.shlex('--link-args="-L/Users/carlos/Desktop/Test LGBM API"', posix=True)

In [29]: s.whitespace_split = True

In [30]: list(s)
Out[30]: ['--link-args=-L/Users/carlos/Desktop/Test LGBM API']

So I change my question: why do you set posix = False by default? But the other question remains: how can I solve my immediate problem?

Alternatives that don't work:

• --link-args "-L/Users/carlos/Desktop/Test LGBM API": passes the quotes alongside the argument.

• --link-args=-L/Users/carlos/Desktop/Test\ LGBM\ API: still splits on whitespace.

Regarding the second option, again posix = True should be preferable:

shlex.escape
Characters that will be considered as escape. This will be only used in POSIX mode, and includes just '' by default.

[MrMino]

I have a feeling that posix=True might break under Windows - haven't checked though. Anyway, you should be able to pass paths with spaces inside them here, so let's treat this as a bug.