Bring back readline capability? #10364
Comments
|
I'm willing to do the work to get this back in time for 6.0, so I'll tag #10329 here. |
|
I think this is a particularly important issue because long time users and advocates of IPython are finding themselves frustrated by or even unwilling to use IPython 5 as it is right now. |
|
I'm happy to add that back as an option, though most of the code has been removed and that will likely be a lot of efforts to reintegrate readline. |
|
If we can figure out how to make that optional (so that it can evolve faster than than core to re-fix potential bug that were introduced by the revival, that would be great. |
|
We might want to delay #10356 if we bring RL back. |
|
See #9260 as well. |
|
And #9399 is the bulk of the removal. |
|
I am strongly -1 on this. We were able to drop a lot of code and complexity when we ditched readline, (and we're working on dropping still more), while dramatically improving the user experience for most users. If we bring back a readline option, we have all the complexity of both interfaces to maintain ~forever. We always knew a change this big would break some workflows, but I'm confident that it's a net improvement, and I do not want us to be in the business of maintaining two alternative terminal interfaces. I would rather we:
|
|
Given that there were always a number of problems with pyreadline on Windows (not least of it, the fact that it globally affected the standard Python REPL as well as IPython) I'd prefer that not requiring (py)readline remain the default, at least on Windows. If a user prefers to install pyreadline manually and enable it, that's fine, but IPython should work without pyreadline present, and not include it as a dependency of a normal install. |
I think that is reasonable, though I believe have some integration (like a flag and/or an env variable) with IPython that restore previous behavior is reasonable. The goal would be to have already well spread scripts and recipes that do rely on IPython readline still working with a simple switch. My guess is that a simple config in IPythonApp, that make the TerminalInteractiveShell class use a traitlet, plus a --readline flag that have a preset value to a known external package would be enough. The ipython as a separate package will also allow to not block IPython 6, and get a fast iteration cycle if necessary.
I doubt that is completely true. The multiline, highlighting, and now completion are still a large advantage of the PTK interface. And user that are pinning to 4.x because they can't even use bothe RL/PTK in parallel have no incentive send patch to IPython / PTK to smooth the behavior. |
|
I wouldn't integrate it with a switch, because that makes it look like a supported option, and people will file bugs about it here. |
|
Just a note on the current experience of In Bash, when you hit ctrl-r then hit escape, it will exit you out. In this one it does nothing when it should close the search area. |
This one is easy to fix, but not totally: ~/dev/ipython master [1]"!!" $ git diff
diff --git a/IPython/terminal/shortcuts.py b/IPython/terminal/shortcuts.py
index 22ad111..89713cd 100644
--- a/IPython/terminal/shortcuts.py
+++ b/IPython/terminal/shortcuts.py
@@ -54,6 +54,10 @@ def register_ipython_shortcuts(registry, shell):
registry.add_binding(Keys.ControlC, filter=HasFocus(DEFAULT_BUFFER)
)(reset_buffer)
+
+ registry.add_binding(Keys.Escape, filter=HasFocus(SEARCH_BUFFER)
+ )(reset_search_buffer)
+
registry.add_binding(Keys.ControlC, filter=HasFocus(SEARCH_BUFFER)
)(reset_search_buffer)That will work, but the |
|
Thanks for opening this, @ivanov! I'm going to go over the details a bit more carefully, I'm also going to encourage more feedback on the list on this. It's evidently a tricky balance and contentious, thus a good sign that more voices can at least inform our decision process. |
|
Just an update on the possible way forward for this effort: instead of bringing the removed code into IPython proper, @Carreau made a new project with just the old readline-based code over at Carreau/rlipython, and made #10373 which allows users to customize this piece. |
|
FYI: the issue that is described in the mailing list:
This indicates that this terminal does not support bracketed paste. It could be worth adding readline support again, but notice that multiple of the mentioned bugs are already fixed or will receive a fix. Biggest problems are probably the Emacs terminal (which is not really an xterm-compatible terminal and will never be supported), and .inputrc support which will eventually come, but due to bandwidth/priority issues on my side, will take some time. In any case, keep reporting user experience issues. This is important. |
|
@pfmoore that's a good point. I would doubt any Windows environments are hurting from the lack of pyreadline. If we bring back readline UI, it probably makes sense to do this for 'true' readline only, whether it's in IPython or the One advantage of the separate rlipython package is that it can be used with the existing IPython 5 release, whereas doing it in IPython itself would obviously only work for new IPython releases. |
|
I'd like to add #9799 as another source of discomfort with the prompt_toolkit defaults. I'd be perfectly happy with an extra external package to install. |
|
pyreadline is a pain on Windows, but prompt-toolkit has also issues. So between 2 unsatisfactory solutions, the less "technical debt" solution looks like the best solution: prompt-toolkit. |
|
I think the point is that prompt_toolkit is better for some people, and readline is better for other people. So the question is, is it necessary to cut the readline people loose to avoid maintaining the extra code? |
Note that right now the attached PR (#10373) is 4 lines, and is just a configuration option to make it a configuration option. The readline code would not even be in IPython and would be an extension. The real question is:
In both case readline will not be made a dependency of IPython again. It does influence a couple of functionalities that are simpler to keep in IPython, but the main contention right now is A or B. |
|
From the emacs side either is fine. B seems more flexible in the long run. |
|
I guess, for those of us who prefer readline, we'd want readline for IPython shelling out as well - so B. |
|
I've been thinking about this one for a while and talking to a few folks on the team I've had a chance to engage in-person. Here's my take on the matter. I'm not yet calling a final decision, but I am trying to move us closer to a resolution. BTW, I want to emphasize that I am very grateful to the Prompt Toolkit team, as it's given us in the vast majority of cases an amazing, modern and high-quality user experience. On to the matter at hand:
Given this, I think a good compromise solution is along the lines of what @Carreau has proposed in his PR:
This does mean supporting for a while this extra package, but I don't imagine it will be that much work: that code barely changed in quite a while, and we're not talking about adding lots of new features. Simply maintaining compatibility with existing, historical baseline behavior. The one cost that I see as somewhat significant is that (mentioned by @Carreau) there may be code we'd like to rip out that this would prevent us from removing. I see that as a price we should pay for the benefit of our users, at least for now. If at some future time we truly find ourselves in a situation where PT is a 100% replacement for RL in every conceivable case, we can revisit. But for now, keeping a bit of slightly stale, special-purpose code for the benefit of some subset of our users is the right thing to do. Over the years we've had multiple kinds of special-case code (windows, py2/3, etc.) and I'm sure we will again in the future. Serving the workflows of our users should, I think, take priority over a code cleanup (in this situation, not making a blanket statement). How does this sound? |
|
BTW, I think this 'fix' ought to be backported to the 5.x series: if anything, I suspect the population of users still on python 2.x has a good overlap with that of people working remotely on older servers. So my vote would be for making the external package be python 2-3 compatible, and to backport the ipython side of the code to 5.x. |
|
Thanks Fernando for taking the time to respond, and especially for reaching to several people. I've rlipython on GitHub for anyone who is interested in maintaining the readline interface. I won't maintain, and won't publish it on PyPI, but you are welcome to do it, and it might be a good things to move this to https://github.com/ipython-contrib if you need an org. Thanks ! |
|
I actually think we should host this on the main IPython org, and put it on pypi. We can also send a slightly more welcoming message to the community, along the lines of:
If @ivanov has a few cycles to put in this direction that would be great, but if not I'm willing to do it over the next couple of weeks. I think it's an important part of engaging with all corners of our user community. Thanks @Carreau for pushing the work this far! |
|
Thanks everyone, especially @Carreau for getting |
|
@ivanov, ping me on that if you'd like a hand... I'd offered above but I was more than a bit naïve about my time availability these last few weeks. Nonetheless I want to help this reach a solid solution for all cmd line users, so I'm still willing to pitch in if you need a hand (and would enjoy doing so :) |
|
Hi, with this issue closed, is there any way to disable |
yes, if you set the |
|
For anyone still subscribed on this issue: I just released |
|
Late to the party, but let me reiterate that RT at present is a no-no is to projects that have modules not designed for multithreaded initialisation, see Sage's ticket 22766. (And this is the primary reason we need to do something about it in Sage). As well, a comment on a general problem with the present RT: it triggers multi-threaded importing of Python modules while doing tab-completion. We very much doubt that it is safe. Last but not the least - given that IPython tab completion causes segfaults, how about a means for non-interactive testing of it? |
|
@jonathanslenders have you run across any issues with completion running in a separate thread? Is there an option to make it run synchronously? |
|
Hi @takluyver, Excuse me for the late reply. I got married last month, and so I've been offline for a while. ;) I'm working on prompt_toolkit 2.0 which already supports both synchronous (in the main thread) and asynchronous (in other thread) completion. I've been working on 2.0 for almost a year and hope to release that in a couple of months. For synchronous completion, you have to keep in mind that if the completer is not super fast, that this will introduce a delay after every key stroke (if you have complete-while-typing on). I believe that Jedi for instance is not super fast. I know that certain libraries have some issues with asynchronous completion. I think we should encourage the authors of these libraries to make sure that importing them in the first place can be done from any thread. In the future, I'd make it an option to run the completions sync or async, but still prefer to have async completion by default. It makes the user experience so much nicer. |
|
On Sat, Oct 21, 2017 at 2:47 PM, Jonathan Slenders ***@***.*** > wrote:
I know that certain libraries have some issues with asynchronous
completion. I think we should encourage the authors of these libraries to
make sure that importing them in the first place can be done from any
thread.
IMHO it's wishful thinking that this is always easy to achieve.
There are Python libraries that are basically wrapping very complex 3rd
party code, and changes needed for thread-safe initialisation have to be
done upstream.
A concrete example we are banging our heads
is Maxima run in an (embedded into Python) Lisp system ECL.
The latter, as all Lisps, needs a garbade collector, and it is using [Boehm
GC](https://github.com/ivmai/bdwgc). On some platforms the latter must be
initialised in the main thread---or it used to be the case, e.g. for
FreeBSD, until I pointed it out, and it was [quickly fixed](ivmai/bdwgc#180). As you can imagine, in other
cases one could be totally out of luck with such a fix, as it needed expert
knowledge of a quite complicated system.
And there are several issues to be resolved, one of which is that in a
single-threaded scenario a library can do its own signal handling, whereas
in a multi-threaded scenario the signal handling has to be centralised. The
other, even harder to resolve, is that that 3rd party code might not be
thread-safe.
(going back to the example above, we see all of this...)
It appears that the best option would be
for PT to do all the imports in the main thread---I have no clue whether
it's easy or even feasible to implement, though.
Finally, let me point out that in effect PT does a multithreaded importing.
As importing is a startup bottleneck for largish systems, it's very
tempting to allow multithreaded importing.
I would really like to hear what CPython people would say about it; I
won't be surprised if the answer were that it must not be done.
PS. I don't mean to ruin your honeymoon, congratulations :-)
|
|
Congratulations! Of course there's no need to apologise - we get your awesome library for free, so any support has to be as and when it's convenient for you. Best wishes to you and your partner. I think we'll likely switch to synchronous completions when they're available. We don't use complete-as-you-type, because we enable the history search feature which is mutually exclusive with that. |
Even if it's not what is used by default, I've experienced and heard about enough other people experiencing pains with the prompt_toolkit-based input breaking people's workflows, that I think we should seriously consider bringing that code back in.
I think it was a mistake to remove it altogether, and change the default at the same time.
The text was updated successfully, but these errors were encountered: