Preparser not robust enough against typos

[tscrim]

I made a typo and sage crashed:

R.<t> = QQ{]

Note that there is one opening squigly bracket { and one closing square bracket ].

Here's the backtrace

Traceback (most recent call last):
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/IPython/core/application.py", line 175, in excepthook
    return self.crash_handler(etype, evalue, tb)
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/IPython/core/crashhandler.py", line 178, in __call__
    raw_input("Hit <Enter> to quit (your terminal may close):")
  File "c_lib.pyx", line 70, in sage.ext.c_lib.sage_python_check_interrupt (sage/ext/c_lib.c:925)
KeyboardInterrupt

Original exception was:
Traceback (most recent call last):
  File "/home/tscrim/sage-5.11.beta3/local/bin/sage-ipython", line 18, in <module>
    app.start()
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/IPython/frontend/terminal/ipapp.py", line 363, in start
    self.shell.mainloop()
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/IPython/frontend/terminal/interactiveshell.py", line 467, in mainloop
    self.interact(display_banner=display_banner)
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/IPython/frontend/terminal/interactiveshell.py", line 579, in interact
    self.input_splitter.push(line)
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/sage/misc/sage_extension.py", line 393, in push
    line = f(line, line_number)
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/sage/misc/interpreter.py", line 271, in __call__
    return preparse(line, reset=(line_number==0))
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/sage/misc/preparser.py", line 1112, in preparse
    L = preparse_generators(L)
  File "/home/tscrim/sage-5.11.beta3/local/lib/python2.7/site-packages/sage/misc/preparser.py", line 977, in preparse_generators
    opening = constructor.rindex('[')
ValueError: substring not found

Apply [trac_14961-hack_fix_preparser-ts.patch​] and attachment: 14961_doctest.patch

Upstream: Reported upstream. Developers acknowledge bug.

Component: user interface

Author: Travis Scrimshaw, Jeroen Demeyer

Reviewer: Volker Braun

Merged: sage-5.12.beta4

Issue created by migration from https://trac.sagemath.org/ticket/14961

[tscrim]

Author: Travis Scrimshaw

[tscrim]

comment:2

Here's a patch which is just a hack fix so that sage doesn't crash and it tells you that an error has occurred. It prints the error twice, but it works...

sage: R.<t> = QQ{]
SyntaxError: Mismatched ']'
SyntaxError: Mismatched ']'
sage:

[vbraun]

Upstream: Reported upstream. No feedback yet.

[vbraun]

comment:3

I've asked on IPython-dev at http://python.6.x6.nabble.com/InputSplitter-and-SyntaxError-td5025938.html

[tscrim]

comment:4

Thanks Volker. I was going to do that first thing this morning (i.e. right now).

[vbraun]

comment:6

The IPython devs acknowledged that this currently does not work. It sounds like they would accept a patch but don't plan on fixing this in the near future themselves. So fixing this properly will take some time.

In the meantime, the proposed workaround looks good. However, it has an unrelated change to in sage/algebras/lie_algebras/kac_moody.py that needs to be removed.

[vbraun]

Reviewer: Volker Braun

[vbraun]

Changed upstream from Reported upstream. No feedback yet. to Reported upstream. Developers acknowledge bug.

[tscrim]

comment:7

Eeek, how did I let that get in there. Here's the patch without that change.

[vbraun]

comment:8

Looks good to me.

[jdemeyer]

Attachment: trac_14961-hack_fix_preparser-ts.patch.gz

[jdemeyer]

comment:10

Attachment: 14961_doctest.patch.gz

[jdemeyer]

Changed author from Travis Scrimshaw to Travis Scrimshaw, Jeroen Demeyer

[jdemeyer]

comment:11

attachment: 14961_doctest.patch needs review.

[vbraun]

comment:12

Fine with me, though I think the effort should go in pushing this upstream not doctesting the temporary workaround.

[vbraun]

comment:13

Followup at #15075 for a proper fix.

[jdemeyer]

Merged: sage-5.12.beta4