New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backslash line continuation broken when defining a callable symbolic expression #30928
Comments
comment:1
The issue is already there in Sage 9.2.beta14. |
comment:2
FWIW, in Sage 9.3.beta1, we have
which is correct, while in the reported issue the second line is entirely missing after |
comment:4
I may be off base, but the following may be important
|
comment:5
I get the same if |
comment:6
Replying to @strogdon:
Yes probably; moreover |
comment:7
Some data points. I think diff --git a/src/sage/repl/interpreter.py b/src/sage/repl/interpreter.py
index b70345f940..206041e4a1 100644
--- a/src/sage/repl/interpreter.py
+++ b/src/sage/repl/interpreter.py
@@ -444,6 +444,7 @@ def SagePreparseTransformer(lines):
sage: # instead of ["'''", 'abc-Integer(1)-Integer(2)', "'''"]
"""
+ print('we are now in sage.repl.interpreter.SagePreparseTransformer')
lines_out = []
reset = True
for line in lines:
diff --git a/src/sage/repl/preparse.py b/src/sage/repl/preparse.py
index 3619fd9e54..2073caa855 100644
--- a/src/sage/repl/preparse.py
+++ b/src/sage/repl/preparse.py
@@ -1745,6 +1745,8 @@ def preparse(line, reset=True, do_time=False, ignore_prompts=False,
quote_state = None
L = line.lstrip()
+ print('result of L = line.lstrip() in sage.repl.preparse')
+ print(L)
if len(L) > 0 and L[0] in ['#', '!']:
return line I see:
Note that |
comment:9
I think this may have been introduced by the transition to IPython 7 and how its new input transformer API deals with lines. For example, commenting out the backslash substitution altogether in preparse.py still yields: sage: f(x) = x \
....: + 1
File "<ipython-input-1-bc21d2198ee4>", line 1
__tmp__=var("x"); f = symbolic_expression(x \).function(x)
^
SyntaxError: unexpected character after line continuation character In the old IPython input transformer docs, it appears there were three types of transformer: physical line, logical line, and python line. In the same docs for IPython 7, there now only appears to be two: cleanup and post. Judging by the change to ipython_extension.py for #28197, it looks like the SagePromptTransformer was moved from "physical line" to "cleanup" and SagePreparseTransformer from "python line" to "post". I'm wondering if that "post" is not quite the same as the old "python line" however and functions more like the old "logical line". That could explain why #30417 that I fixed arose in the first place. I have a couple ideas for how to patch this in the short term, but neither is likely to be particularly elegant. I'll play around with them and see if I can get something pushed in the next couple days. I agree with @strogdon that some significant changes/cleanup are needed in the preparser and interpreter in order to fix this properly. I know there was talk in #28974 about building a proper grammar and parser for Sage to remove the need for all these regex kludges. Would be a large undertaking though. |
Commit: |
Author: Joshua Campbell |
comment:10
I've pushed a potential fix.
As predicted: not the most elegant fix, and there's a fair chance I broke another edge case somewhere in the process. I'm not sure how else to fix it without tearing up a bunch of the existing preparser code. Input most welcome! :) New commits:
|
comment:11
Initial testing indicates that line continuation is now working. However, regardless the input after the Example:
|
comment:12
Hmm... perhaps this is an IPython quirk? For example, running an IPython installed completely independently of Sage:
|
comment:13
Same here with |
comment:14
Replying to @strogdon:
Looks like intended behavior:
The new I'll add a bit about that attribute in |
Branch pushed to git repo; I updated commit sha1. New commits:
|
comment:16
Replying to @jcamp0x2a:
Good spotting this. I tried to search the ipython git repo bugs for this without success. I never considered that it was related to a closed issue. I have ipython-7.18.1 here on my Gentoo laptop. Your above
I'm satisfied with this, but will wait to see if @egourgoulhon has any comments. |
comment:17
Replying to @strogdon:
Or if anyone else has comments. |
comment:18
Thanks for the fix! I made a few tests and everything is OK. So I am +1 for a positive review, once the pyflake error reported by the patchbot error has been fixed. |
comment:19
Btw, while making tests for this ticket branch, I found another error:
But this does not pertain to the current ticket: it is already here in Sage 9.1. |
comment:20
Replying to @egourgoulhon:
I think this is the same false positive that was identified in #30417 comment:8. Trying to ignore it with |
comment:21
Replying to @egourgoulhon:
The code is still fresh in my mind, and this seems like a trivial fix, so I'm going to try to knock it out here. If it's not as trivial as I thought, though, I'll spin off a new ticket and put this back into review. |
comment:22
Replying to @jcamp0x2a:
OK very good! |
comment:24
Not trivial. Forgot about nested parentheses/braces/brackets. A regular expression is really ill-suited to handling that, so On the bright side, I was able to quickly add support for breaking the parameter list onto multiple lines like so: f(a,
b,
c,
d) = a + b*2 + c*3 + d*4 ...and we'll see what pyflakes has to say now. :) |
comment:25
Replying to @jcamp0x2a:
Created #30953 to track this. |
comment:26
Not sure what the source of the Darwin failures is. Every thing is fine here with |
comment:27
Replying to @strogdon:
Yea, doesn't look like a very happy patchbot. :) Looking through it's history, I don't see a single green TestsPassed result, so I don't think it's due to anything introduced here. |
Reviewer: Steven Trogdon |
comment:28
Eric please add your name to reviewers if you agree. |
comment:29
Thank you very much for having fixed this! |
Changed reviewer from Steven Trogdon to Steven Trogdon, Eric Gourgoulhon |
comment:30
Replying to @egourgoulhon:
You are most welcome! I'm going to spend some time brainstorming ways to make the preparser more robust, potentially including building that proper grammar/parser I mentioned earlier. Might have to be a year-long goal for 2021, though. :) |
Changed branch from u/gh-jcamp0x2a/30928-backslash-continuation to |
As reported in https://groups.google.com/g/sage-devel/c/XD1VtG0TOEk/m/Lo6L8YBGCgAJ, we have currently (Sage 9.2 and 9.3.beta1):
There was no such issue with Sage 9.1 and lower.
CC: @jcamp0x2a @slel
Component: symbolics
Keywords: callable symbolic expression, backslash, preparser
Author: Joshua Campbell
Branch/Commit:
64c5757
Reviewer: Steven Trogdon, Eric Gourgoulhon
Issue created by migration from https://trac.sagemath.org/ticket/30928
The text was updated successfully, but these errors were encountered: