New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved Repl #325

Open
adriaanm opened this Issue Mar 2, 2017 · 21 comments

Comments

Projects
None yet
9 participants
@adriaanm
Member

adriaanm commented Mar 2, 2017

Here are some ideas for improving the repl:

  • syntax highlighting
  • class-based wrapping
  • take advantage of Java 9 features
  • improve embeddability of repl in worksheets for fast data platforms

Steal Incorporate the improvements already implemented in the Ammonite REPL:

  • improved multi-line editing/history
  • magic imports (ivy dependencies etc)

More ideas / requirements:

  • reload class path when changing/recompiling project (faster test cycle)
  • ctrl-c stops execution of program instead of repl itself
  • test in IntelliJ shell window

@adriaanm adriaanm added this to the 2.13 milestone Mar 2, 2017

@DavidPerezIngeniero

This comment has been minimized.

Show comment
Hide comment
@DavidPerezIngeniero

DavidPerezIngeniero Mar 10, 2017

+1 for incorporating Ammonite, this is a fantastic tool.

DavidPerezIngeniero commented Mar 10, 2017

+1 for incorporating Ammonite, this is a fantastic tool.

@kjsingh

This comment has been minimized.

Show comment
Hide comment
@kjsingh

kjsingh Mar 12, 2017

As long as we keep the performance smooth!

kjsingh commented Mar 12, 2017

As long as we keep the performance smooth!

@yinshurman

This comment has been minimized.

Show comment
Hide comment
@yinshurman

yinshurman May 5, 2017

in the REPL, enter a Tab will display all the available names imported in the scope which makes a lot troubles in some case,for example,when pasting codes with tab as indent from editors like vim even in the :paste mode ,the screen if full of unexpected names.
by the way ,the auto completion by Tab direct print all the names in the screen makes it hard read on the console.Can't you just display it another way ? For example, like IPython' way.

yinshurman commented May 5, 2017

in the REPL, enter a Tab will display all the available names imported in the scope which makes a lot troubles in some case,for example,when pasting codes with tab as indent from editors like vim even in the :paste mode ,the screen if full of unexpected names.
by the way ,the auto completion by Tab direct print all the names in the screen makes it hard read on the console.Can't you just display it another way ? For example, like IPython' way.

@yinshurman

This comment has been minimized.

Show comment
Hide comment
@yinshurman

yinshurman May 5, 2017

In Python you can investigate any object foo (type, object in a global variable, built-in function, etc.) using foo to see what the thing evaluates to, type(foo) to show its type, dir(foo) to tell you the methods you can call on it, and help(foo) to get the built-in documentation. You can even do things like help("re") to find out the documentation on the package named re (which holds regular-expression objects and methods), even though there isn't an object associated with it.Can scala REPL does things like this ?

yinshurman commented May 5, 2017

In Python you can investigate any object foo (type, object in a global variable, built-in function, etc.) using foo to see what the thing evaluates to, type(foo) to show its type, dir(foo) to tell you the methods you can call on it, and help(foo) to get the built-in documentation. You can even do things like help("re") to find out the documentation on the package named re (which holds regular-expression objects and methods), even though there isn't an object associated with it.Can scala REPL does things like this ?

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt May 5, 2017

@yinshurman about tab, are you using a recent version with scala/bug#9336 ? That enables the kludge in jline; but maybe it could do more in paste mode.

som-snytt commented May 5, 2017

@yinshurman about tab, are you using a recent version with scala/bug#9336 ? That enables the kludge in jline; but maybe it could do more in paste mode.

@yinshurman

This comment has been minimized.

Show comment
Hide comment
@yinshurman

yinshurman May 5, 2017

I'm use scala of version 2.11.7

yinshurman commented May 5, 2017

I'm use scala of version 2.11.7

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm May 5, 2017

Member
Member

adriaanm commented May 5, 2017

@yinshurman

This comment has been minimized.

Show comment
Hide comment
@yinshurman

yinshurman May 8, 2017

It's fixed by add a tip "Display all 435 possibilities? (y or n)" every time you enter a tab.
Instead of direct printing all the available options ,I think it's better to display them another way,for example,when you type foo. and enter a tab, all the available options display in another little window right behind the foo.,and when you selected the option you needed,the window disappeared too. I think it is better comfortable

yinshurman commented May 8, 2017

It's fixed by add a tip "Display all 435 possibilities? (y or n)" every time you enter a tab.
Instead of direct printing all the available options ,I think it's better to display them another way,for example,when you type foo. and enter a tab, all the available options display in another little window right behind the foo.,and when you selected the option you needed,the window disappeared too. I think it is better comfortable

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt May 8, 2017

It's a nice consequence of English pronunciation that a windowing REPL or wrepl sounds the same as repl.

som-snytt commented May 8, 2017

It's a nice consequence of English pronunciation that a windowing REPL or wrepl sounds the same as repl.

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm May 9, 2017

Member

The plan is to first split the current repl into:

  • a layer on top of the compiler (similar to the presentation compiler)
  • a shell that implements the UI (and interfaces with jline3).

The api exposed by the compiler should also support other shells, such as ammonite / spark notebooks. Eventually, maybe we can support connecting to this repl compiler instance using the language server protocol (as in lampepfl/dotty#2195)

I've just started hacking on this in my WIP branch: https://github.com/scala/scala/compare/2.13.x...adriaanm:repl-modularize?expand=1

Member

adriaanm commented May 9, 2017

The plan is to first split the current repl into:

  • a layer on top of the compiler (similar to the presentation compiler)
  • a shell that implements the UI (and interfaces with jline3).

The api exposed by the compiler should also support other shells, such as ammonite / spark notebooks. Eventually, maybe we can support connecting to this repl compiler instance using the language server protocol (as in lampepfl/dotty#2195)

I've just started hacking on this in my WIP branch: https://github.com/scala/scala/compare/2.13.x...adriaanm:repl-modularize?expand=1

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm May 19, 2017

Member

scala/scala#5903 takes the first steps towards splitting the REPL into a shell (UI) component and a compiler to support evaluating expressions and definitions + auto-completion (similar to the presentation compiler). The idea is that there could be multiple front-ends (the default, ammonite, spark worksheets,...) that all use the same underling compiler for consistent evaluation and type checking semantics.

Also, the default shell will likely use jline3, which has better support multi-line editing, highlighting, parsing as you type, etc.

Member

adriaanm commented May 19, 2017

scala/scala#5903 takes the first steps towards splitting the REPL into a shell (UI) component and a compiler to support evaluating expressions and definitions + auto-completion (similar to the presentation compiler). The idea is that there could be multiple front-ends (the default, ammonite, spark worksheets,...) that all use the same underling compiler for consistent evaluation and type checking semantics.

Also, the default shell will likely use jline3, which has better support multi-line editing, highlighting, parsing as you type, etc.

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Jun 13, 2017

Member

By the way, we're currently not planning to backport the REPL work to 2.12. Please target 2.13.x for any PR that changes the REPL (or, please fix it first in 2.13 and backport to 2.12).

We are open to discussing a backport of the rework, though -- especially if someone's interested to contribute it :-) (A backport would have to be more careful not to break anyone currently embedding the 2.12 repl).

Finally, we'd love to hear feedback on the interface proposed in scala/scala#5903 from anyone embedding the REPL.

Member

adriaanm commented Jun 13, 2017

By the way, we're currently not planning to backport the REPL work to 2.12. Please target 2.13.x for any PR that changes the REPL (or, please fix it first in 2.13 and backport to 2.12).

We are open to discussing a backport of the rework, though -- especially if someone's interested to contribute it :-) (A backport would have to be more careful not to break anyone currently embedding the 2.12 repl).

Finally, we'd love to hear feedback on the interface proposed in scala/scala#5903 from anyone embedding the REPL.

@retronym

This comment has been minimized.

Show comment
Hide comment
@retronym

retronym Aug 10, 2017

Member

Another wishlist item that was in part prototyped in REPLs of yore would some features that exploit the runtime types and reflection.

  • Autocompletion of x.asInstanceOf[<TAB>] could look at the x.getClass, figure out the corresponding ClassSymbol, and then intelligently merge that with the static type of x. Example if x.getClass == classOf[Some[_], and x is of type Option[String], offer to downcast to Some[String].
  • Another UI option for that would be to have another level of tab completion used a safe downcast as an adaptation.
  • Provide a utility method or command to do some reflection based stringification of objects
  • Cheat mode: Add the ability to call private methods and fields by generating reflective calls

An unrelated feature that I think would be beneficial would be a UI to show the expected type at the cursor. Maybe use this to filter or prioritize autocomplete suggestions. I prototyped this in my last round of REPL renovations, but didn't follow through.

Member

retronym commented Aug 10, 2017

Another wishlist item that was in part prototyped in REPLs of yore would some features that exploit the runtime types and reflection.

  • Autocompletion of x.asInstanceOf[<TAB>] could look at the x.getClass, figure out the corresponding ClassSymbol, and then intelligently merge that with the static type of x. Example if x.getClass == classOf[Some[_], and x is of type Option[String], offer to downcast to Some[String].
  • Another UI option for that would be to have another level of tab completion used a safe downcast as an adaptation.
  • Provide a utility method or command to do some reflection based stringification of objects
  • Cheat mode: Add the ability to call private methods and fields by generating reflective calls

An unrelated feature that I think would be beneficial would be a UI to show the expected type at the cursor. Maybe use this to filter or prioritize autocomplete suggestions. I prototyped this in my last round of REPL renovations, but didn't follow through.

@retronym

This comment has been minimized.

Show comment
Hide comment
@retronym

retronym Aug 18, 2017

Member

One more wishlist item for power mode: a version of reify that just funnels the actual typed argument tree back into userland.

You can simulate this at the moment with a macro and a backchannel:

⚡ scala -cp /tmp
Welcome to Scala 2.12.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.

scala> :power
Power mode enabled. :phase is at typer.
import scala.tools.nsc._, intp.global._, definitions._
Try :help or completions for vals._ and power._

scala> jz.TreeOf { class C { def foo = 42 } }
res0: $r.intp.global.Tree =
{
  class C extends scala.AnyRef {
    def <init>(): C = {
      C.super.<init>();
      ()
    };
    def foo: Int = 42
  };
  ()
}

scala> :quit

⚡ cat sandbox/treeof.scala
package jz
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context

object TreeOf {
  def apply[A](a: A): Any = macro impl
  def impl(c: Context)(a: c.Tree): c.Tree = {
    val key = "jz.treeof.backchannel"
    System.getProperties.put(key, a)
    import c.universe._
    q"_root_.java.lang.System.getProperties.remove($key).asInstanceOf[Tree]"
  }
}
Member

retronym commented Aug 18, 2017

One more wishlist item for power mode: a version of reify that just funnels the actual typed argument tree back into userland.

You can simulate this at the moment with a macro and a backchannel:

⚡ scala -cp /tmp
Welcome to Scala 2.12.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.

scala> :power
Power mode enabled. :phase is at typer.
import scala.tools.nsc._, intp.global._, definitions._
Try :help or completions for vals._ and power._

scala> jz.TreeOf { class C { def foo = 42 } }
res0: $r.intp.global.Tree =
{
  class C extends scala.AnyRef {
    def <init>(): C = {
      C.super.<init>();
      ()
    };
    def foo: Int = 42
  };
  ()
}

scala> :quit

⚡ cat sandbox/treeof.scala
package jz
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context

object TreeOf {
  def apply[A](a: A): Any = macro impl
  def impl(c: Context)(a: c.Tree): c.Tree = {
    val key = "jz.treeof.backchannel"
    System.getProperties.put(key, a)
    import c.universe._
    q"_root_.java.lang.System.getProperties.remove($key).asInstanceOf[Tree]"
  }
}
@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Aug 19, 2017

Member

that's beautiful

Member

adriaanm commented Aug 19, 2017

that's beautiful

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt commented Aug 19, 2017

jz mode.

@zaxebo1

This comment has been minimized.

Show comment
Hide comment
@zaxebo1

zaxebo1 Sep 29, 2017

is there a "plugin loading mechanism for REPL", with a stable API and proper documentation/example, "through which one can add new inbuilt commands(colon commands) to REPL" , say ":my_new_command" , just like the inbuilt commands ":help" , ":paste" etc.

zaxebo1 commented Sep 29, 2017

is there a "plugin loading mechanism for REPL", with a stable API and proper documentation/example, "through which one can add new inbuilt commands(colon commands) to REPL" , say ":my_new_command" , just like the inbuilt commands ":help" , ":paste" etc.

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt Sep 30, 2017

Welcome to Scala 2.12.3 (OpenJDK 64-Bit Server VM, Java 1.8.0_131).
Type in expressions for evaluation. Or try :help.

scala> import scala.tools.nsc._,interpreter._
import scala.tools.nsc._
import interpreter._

scala> val loop = new ILoop { override def commands = super.commands :+ LoopCommand.cmd("greet","helper","go for it", s => { println("hi") ; Result(false, None) }) }
loop: scala.tools.nsc.interpreter.ILoop = $anon$1@58f31629

scala> loop.process(new Settings)
Welcome to Scala 2.12.3 (OpenJDK 64-Bit Server VM, Java 1.8.0_131).
Type in expressions for evaluation. Or try :help.

scala> :greet
hi
res0: Boolean = true

Other than that, no. So, no.

som-snytt commented Sep 30, 2017

Welcome to Scala 2.12.3 (OpenJDK 64-Bit Server VM, Java 1.8.0_131).
Type in expressions for evaluation. Or try :help.

scala> import scala.tools.nsc._,interpreter._
import scala.tools.nsc._
import interpreter._

scala> val loop = new ILoop { override def commands = super.commands :+ LoopCommand.cmd("greet","helper","go for it", s => { println("hi") ; Result(false, None) }) }
loop: scala.tools.nsc.interpreter.ILoop = $anon$1@58f31629

scala> loop.process(new Settings)
Welcome to Scala 2.12.3 (OpenJDK 64-Bit Server VM, Java 1.8.0_131).
Type in expressions for evaluation. Or try :help.

scala> :greet
hi
res0: Boolean = true

Other than that, no. So, no.

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt Feb 22, 2018

REPL should clip my stackoverflows for me. Give me a last command to display it. And elide repeating segments:

  at .$anonfun$f$1$adapted(<console>:11)
  at .$anonfun$f$1(<console>:11)
  at .$anonfun$f$1$adapted(<console>:11)
  at .$anonfun$f$1(<console>:11)

som-snytt commented Feb 22, 2018

REPL should clip my stackoverflows for me. Give me a last command to display it. And elide repeating segments:

  at .$anonfun$f$1$adapted(<console>:11)
  at .$anonfun$f$1(<console>:11)
  at .$anonfun$f$1$adapted(<console>:11)
  at .$anonfun$f$1(<console>:11)
@hvesalai

This comment has been minimized.

Show comment
Hide comment
@hvesalai

hvesalai commented Mar 6, 2018

See also scala/scala#6382

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Aug 8, 2018

Member

jline3 upgrade split off to #535 (and scheduled for 2.13.0-RC1)

Member

dwijnand commented Aug 8, 2018

jline3 upgrade split off to #535 (and scheduled for 2.13.0-RC1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment