New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dotty REPL initial version #1082

Merged
merged 18 commits into from Feb 18, 2016

Conversation

Projects
None yet
6 participants
@odersky
Contributor

odersky commented Feb 14, 2016

An initial version of a REPL for Dotty. Basic functionality works.

Class dotty.tools.dotc.repl.Main is the main entry point. To start the REPL, before we have a dedicated command, use

 java dotty.tools.dotc.repl.Main

(when started under scala, it does not find an Execution context. Hopefully somebody else is better than me at figuring this out).

A comment below explains how the REPL code was derived, and contains a list of TODOs with suggestions to improve it. I am very much hoping for the contributions of others to move this forward.

Review by @hubertp or @DarkDimius or @smarter or anybody wishing to contribute.

@odersky

This comment has been minimized.

Show comment
Hide comment
@odersky

odersky Feb 14, 2016

Contributor

Explanations, copied from comment on repl.Main.scala:

This REPL was adapted from an old (2008-ish) version of the Scala
REPL. The original version from which the adaptation was done is found in:

 https://github.com/odersky/legacy-svn-scala/tree/spoon

The reason this version was picked instead of a more current one is that
the older version is much smaller, therefore easier to port. It is also
considerably less intertwined with nsc than later versions.

There are a number of TODOs:

  • re-enable jline support (urgent, easy, see TODO in InteractiveReader.scala)
  • figure out why we can launch REPL only with java, not with scala.
  • make a doti command (urgent, easy)
  • create or port REPL tests (urgent, intermediate)
  • make interpreter run a pseudo line on startup to pre-load compiler (somewhat urgent, easy)
  • copy improvements of current Scala REPL wrt to this version
    (somewhat urgent, intermediate)
  • re-enable bindSettings (not urgent, easy, see TODO in InterpreterLoop.scala)
  • make string generation more functional (not urgent, easy)
  • better handling of ^C (not urgent, intermediate)
  • syntax highlighting (not urgent, intermediate)
  • integrate with presentation compiler for command completion (not urgent, hard)
Contributor

odersky commented Feb 14, 2016

Explanations, copied from comment on repl.Main.scala:

This REPL was adapted from an old (2008-ish) version of the Scala
REPL. The original version from which the adaptation was done is found in:

 https://github.com/odersky/legacy-svn-scala/tree/spoon

The reason this version was picked instead of a more current one is that
the older version is much smaller, therefore easier to port. It is also
considerably less intertwined with nsc than later versions.

There are a number of TODOs:

  • re-enable jline support (urgent, easy, see TODO in InteractiveReader.scala)
  • figure out why we can launch REPL only with java, not with scala.
  • make a doti command (urgent, easy)
  • create or port REPL tests (urgent, intermediate)
  • make interpreter run a pseudo line on startup to pre-load compiler (somewhat urgent, easy)
  • copy improvements of current Scala REPL wrt to this version
    (somewhat urgent, intermediate)
  • re-enable bindSettings (not urgent, easy, see TODO in InterpreterLoop.scala)
  • make string generation more functional (not urgent, easy)
  • better handling of ^C (not urgent, intermediate)
  • syntax highlighting (not urgent, intermediate)
  • integrate with presentation compiler for command completion (not urgent, hard)
/** A GenBCode phase that uses `virtualDirectory` for its output */
private class REPLGenBCode extends GenBCode {
override def outputDir(implicit ctx: Context) = virtualDirectory

This comment has been minimized.

@smarter

smarter Feb 14, 2016

Member

Instead of using a custom phase, couldn't we just make sure that the Context instance we use in the compiler is set up so that settings.d.value is set to virtualDirectory ?

@smarter

smarter Feb 14, 2016

Member

Instead of using a custom phase, couldn't we just make sure that the Context instance we use in the compiler is set up so that settings.d.value is set to virtualDirectory ?

This comment has been minimized.

@odersky

odersky Feb 15, 2016

Contributor

No, because virtualDirectory does not have a name, so it cannot be used in the -d option.

@odersky

odersky Feb 15, 2016

Contributor

No, because virtualDirectory does not have a name, so it cannot be used in the -d option.

}
/** Dig the types of all bound variables out of the compiler run.
* TODO: Change the interface so that we typecheck, and then transform

This comment has been minimized.

@smarter

smarter Feb 14, 2016

Member

I don't really understand this comment, what interface? What transformations?

@smarter

smarter Feb 14, 2016

Member

I don't really understand this comment, what interface? What transformations?

@smarter

This comment has been minimized.

Show comment
Hide comment
@smarter

smarter Feb 14, 2016

Member

I'd be interested in hearing from @retronym or others familiar with the scalac REPL (@som-snytt ?) about pitfalls we could fall into and mistakes we are likely to repeat. (Suggestions on cool things we could do are also welcome!)

Member

smarter commented Feb 14, 2016

I'd be interested in hearing from @retronym or others familiar with the scalac REPL (@som-snytt ?) about pitfalls we could fall into and mistakes we are likely to repeat. (Suggestions on cool things we could do are also welcome!)

@som-snytt

This comment has been minimized.

Show comment
Hide comment
@som-snytt

som-snytt Feb 14, 2016

I subscribed to this PR this morning because I'm interested in contributing code. I'll make an effort to make time for it.

som-snytt commented Feb 14, 2016

I subscribed to this PR this morning because I'm interested in contributing code. I'll make an effort to make time for it.

@VladimirNik

This comment has been minimized.

Show comment
Hide comment
@VladimirNik

VladimirNik Feb 15, 2016

Contributor

@odersky To launch repl with scala I used:

scala -J-Xbootclasspath/a:\
[path]/.ivy2/cache/me.d-d/scala-compiler/jars/scala-compiler-2.11.5-[version-info].jar:\
[path-to-dotty]/dotty/bin/../target/scala-2.11/dotty_2.11-0.1-SNAPSHOT.jar \
dotty.tools.dotc.repl.Main

Fork of scala compiler from me.d-d and dotty are added to bootclasspath.

Contributor

VladimirNik commented Feb 15, 2016

@odersky To launch repl with scala I used:

scala -J-Xbootclasspath/a:\
[path]/.ivy2/cache/me.d-d/scala-compiler/jars/scala-compiler-2.11.5-[version-info].jar:\
[path-to-dotty]/dotty/bin/../target/scala-2.11/dotty_2.11-0.1-SNAPSHOT.jar \
dotty.tools.dotc.repl.Main

Fork of scala compiler from me.d-d and dotty are added to bootclasspath.

@retronym

This comment has been minimized.

Show comment
Hide comment
@retronym

retronym Feb 15, 2016

Contributor

I'd be interested in hearing from @retronym or others familiar with the scalac REPL

Adding tab completion support to the REPL will be a good way to start experimenting with ways to make the typechecker IDE friendly, without the waiting for integration into Eclipse. This will be handy to validate that the parser and typechecker are fault tolerant and don't discard trees that correspond to the textual sources.

See scala/scala#4766 for an example of such a problem in Scalac, and scala/scala#4725 for the change to use the presentation compiler to drive tab completion.

Try to avoid splicing user-written code into the synthetic wrappers as text. Ideally you want to be able to typecheck the expression as entered after programattically setting up a typechecker context that includes the imports of previously defined symbols and with previously entered imports. Once that is typechecked, it could be spliced into the synthetic wrapper via AST manipulation. This avoids awkward problems of quoting / escaping, and could probably avoid the need to have the deeply nested $iw modules polluting the owner chain of definitions.

Contributor

retronym commented Feb 15, 2016

I'd be interested in hearing from @retronym or others familiar with the scalac REPL

Adding tab completion support to the REPL will be a good way to start experimenting with ways to make the typechecker IDE friendly, without the waiting for integration into Eclipse. This will be handy to validate that the parser and typechecker are fault tolerant and don't discard trees that correspond to the textual sources.

See scala/scala#4766 for an example of such a problem in Scalac, and scala/scala#4725 for the change to use the presentation compiler to drive tab completion.

Try to avoid splicing user-written code into the synthetic wrappers as text. Ideally you want to be able to typecheck the expression as entered after programattically setting up a typechecker context that includes the imports of previously defined symbols and with previously entered imports. Once that is typechecked, it could be spliced into the synthetic wrapper via AST manipulation. This avoids awkward problems of quoting / escaping, and could probably avoid the need to have the deeply nested $iw modules polluting the owner chain of definitions.

Show outdated Hide outdated tests/run/t920.scala
@odersky

This comment has been minimized.

Show comment
Hide comment
@odersky

odersky Feb 16, 2016

Contributor

@som-snytt Contributions are very welcome!
@retronym I agree. Widening the interface to the compiler can streamline things a lot and can provide useful new functionality.

Contributor

odersky commented Feb 16, 2016

@som-snytt Contributions are very welcome!
@retronym I agree. Widening the interface to the compiler can streamline things a lot and can provide useful new functionality.

@odersky

This comment has been minimized.

Show comment
Hide comment
@odersky

odersky Feb 17, 2016

Contributor

After discussion at dotty meeting we decided to merge

Contributor

odersky commented Feb 17, 2016

After discussion at dotty meeting we decided to merge

odersky added some commits Feb 13, 2016

Make output directory overridable
The interpreter needs to install a virtual directory
as output directory. This is not supported with the -d
option in ScalaSettings. The solution is to make the
output directory overridable in the GenBCode phase.
Utility method for phase replacement
Allows to replace existing phase by sequence of new
phases.
First PoC of REPL
Adaptation of REPL by Spoon from ca 2007. Compiles OK, but
not yet tested.
newCompiler now takes a context parameter
Makes side-effecting initialization of interpreter
unnecessary.
Revisions to REPL
Changes necessary to make basic REPL functionality work.

Major refactoing: Code of Interpreter is now in CompilingInterpreter.scala.
Interpreter.scala contains just the API.
Refine validity checking
When defining a class in the interpreter we had a case where
the class was accessed at phase 46 in the backend, yet the denotation
was the initial denotation in a previous run. In that case we
have to check again at the phase where the denotation is valid.
This was not done before, and hence the owner of the denbotation
did not contain the symbol because the backend phase is after flatten.
Fix validity checking refinement
It seems some symbols are valid from NoPhase (0). In any case, we should
not check members before typerphase.
Reinstantiate t920
Got deleted by accident. Version in run has object
renamed to prevent case clashes on MacOS. Version that
exhibits the clash is in pending/run.
Remove interpreterSettings logic
Seems to be overkill for the current interpreter. The only thing that was needed
was a configrable linewidth. A plain setting works fine for this and is in
line with the way things are done elsewhere.
Narrow Interpreter.scala further.
compileString is not needed and does not what one might
expect (no wrapping). So it should not be exported.

DarkDimius added a commit that referenced this pull request Feb 18, 2016

@DarkDimius DarkDimius merged commit 7a893f5 into lampepfl:master Feb 18, 2016

4 checks passed

validate-junit [1080] SUCCESS. Took 22 min.
Details
validate-main [1092] SUCCESS. Took 38 min.
Details
validate-partest [1082] SUCCESS. Took 20 min.
Details
validate-partest-bootstrapped [174] SUCCESS. Took 20 min.
Details

@allanrenucci allanrenucci deleted the dotty-staging:add-repl branch Dec 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment