Magics.parse_options parse problem

[louismartin]

Hi,
I think there is a bug with the parse_options method in the sense that it modifies the input statement which causes a bug in my application.

Repro

In jupyter notebook, when iterating over my dataframe, I tried to time the operation with the timeit magic:

> df = pd.DataFrame({'document': ['First doc.', 'Second doc.']})
> %timeit _ = [s.count('. ') for s in df['document'].tolist()]
KeyError: ' document'

Note the key that was transformed from 'document' in my code to ' document' at execution time.

This happens in the parse_options method

ipython/IPython/core/magics/execution.py

Line 1077 in a6082c6

opts, stmt = self.parse_options(line,'n:r:tcp:qo',

        opts, stmt = self.parse_options(line,'n:r:tcp:qo',
                                        posix=False, strict=False)

When dropping into pdb, I get the following:

> print(line)
_ = [s.count('. ') for s in df['document'].tolist()]
> print(stmt)
_ = [s.count('. ') for s in df[' document'].tolist()]

and within parse_options(), I guess it happens when splitting the line with arg_split and then joining back with spaces

> argv = arg_split(arg_str, posix, strict)
> print(arg_str)
_ = [s.count('. ') for s in df['document'].tolist()]
> print(argv)
['_', '=', "[s.count('.", "') for s in df['", "document'].tolist()]"]

And thenargs = ' '.join(args) will join with an extra space.

Also changing my command from _ = [s.count('. ') for s in df['document'].tolist()] to _ = [s for s in df['document'].tolist()] removes the problem so there might be a problem with the .count('. ').

[Carreau]

Hum, thanks, that seem legit. This code is really nasty; and it might take some time to fix.
Pull requests appreciated.

[adityausathe]

#12708
I have raised a PR to address this issue. @Carreau please review.

[ivanov]

Hey there, I'm going through old issues and it seems to me that it makes sense to close this one.

The fix for this was merged in #12708 and will make it into IPython 8.0 🎉

Thanks everyone and happy hacking!