preparse behaviour

[mantepse]

I noticed that apparently preparse treats a (multiline) string as comment, if the first non-whitespace character is a #:

    L = line.lstrip()
    if len(L) > 0 and L[0] in ['#', '!']:
        return line

I wonder why, because it is not unusual for a block of code to start with a comment. I cannot see where comments in later lines are treated, so I have no idea how to fix this.

The issue is that when calling preparse, if the first line is a comment (IDK why it would start with a !...) then it does not preparse anything further. Specifically, if there is a multiline statement that you want to preparse, but the first line is a comment, it doesn't do anything. For example:

# Comment
def foo(x):
    R.<y> = QQ[]
    return x * y

would not get preparsed. Given the description, the preparser works on the assumption that the input is a specific self-contained line, but it seems to have the functionality to handle multi-line input. I think we should either see if there is a line break and then preparse the rest after the comment line or very explicitly state the assumption.

Component: user interface

Keywords: preparser

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/17778

[jdemeyer]

comment:1

What exactly is the issue, I don't understand the description...

[tscrim]

comment:2

The issue is that when calling preparse, if the first line is a comment (IDK why it would start with a !...) then it does not preparse anything further. Specifically, if there is a multiline statement that you want to preparse, but the first line is a comment, it doesn't do anything. For example:

# Comment
def foo(x):
    R.<y> = QQ[]
    return x * y

would not get preparsed. Given the description, the preparser works on the assumption that the input is a specific self-contained line, but it seems to have the functionality to handle multi-line input. I think we should either see if there is a line break and then preparse the rest after the comment line or very explicitly state the assumption.

[jdemeyer]

Reviewer: Jeroen Demeyer

[jdemeyer]

comment:4

Not a bug.

preparse preparses single lines, not blocks of code.

[tscrim]

Commit: 44e5bb0

[tscrim]

Changed reviewer from Jeroen Demeyer to none

[tscrim]

Author: Travis Scrimshaw

[tscrim]

comment:6

I feel like this needs to a be little bit more explicitly documented.

---

New commits:

44e5bb0

Some slightly more explicit doc in preparse.py.

[tscrim]

Branch: public/17778

[tscrim]

comment:7

Actually, I'm not sure that is true. How are block comments with """ handled? It seems like they are passed as a group to preparse() by the interpreter after stripping out ....:. So here's what I believe happens:

I type in

sage: def foo(x):
....:     return 5 * x

which gets fed to the interpreter, which results in

def foo(x):
    return 5 * x

which then gets sent to the preparser as a complete unit (a single string) and becomes

def foo(x):
    return Integer(5) * x

Unless I am misunderstanding something?

[tscrim]

comment:8

Okay, so it does things a little differently than I thought. The interpreter decides whether or not it needs to preparse a line. So the interpreter keeps track of whether you're in a comment or not. If it has a line break with a \, then it stitches the lines together. If it has a parenthesis with a line break, then it feeds that (with the \n) into the preparser as single string.

Although I can break the preparser by doing

sage: def foo(x):
....:     R.<x> = QQ[
....:     ]

So there is an issue with bad user input, but I'm not sure what to do about that.

[slel]

comment:9

Solved by #31043.

If you agree, remove the branch, move your name
from author to reviewer, and give positive review.

[tscrim]

Reviewer: Travis Scrimshaw

[tscrim]

Changed branch from public/17778 to none

[tscrim]

Changed author from Travis Scrimshaw to none

[tscrim]

Changed commit from 44e5bb0 to none

[tscrim]

comment:10

I agree this is fixed. It would be nice to do something about the error in comment:8, but I think that is expecting too much.