New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pretty basic language server and build client. #337

Merged
merged 45 commits into from Nov 27, 2018

Conversation

6 participants
@olafurpg
Copy link
Member

olafurpg commented Nov 21, 2018

This is a large commit that implements Pretty Basic IDE features.

  • "Import build" for sbt and Bloop projects using the Build Server Protocol.
  • File save triggers compile in the build server and errors are
    displayed in the editor.
  • Goto definition works for project sources and library dependencies.
  • Goto definition continues to work despite of syntax errors in the
    open unsaved buffer.
  • Comprehensive integration tests that are easy to write and read.
  • Documentation for how to
    • use Metals with the editors Vim, VS Code, Sublime and Atom
    • import sbt and Bloop build
    • integrate new build tools and text editors

Big thanks to the following people who made this milestone possible.

  • Jorge Vicente Cantero (@jvican) for his great work in Bloop and helping out with
    unblocking a lot of Metals issues. Most the heavy lifting in "build import"
    is provided by Bloop.
  • Eugene Burmako (@xeno-by) for close collaboration on Scalameta and SemanticDB
    that made the features in this PR possible.
  • Gabriele Petronella (@gabro) and Alexey Alekhin (@laughedelic) for constantly trying out
    bleeding edge features and providing great feedback on WIP features.
  • Justin Kaeser (@jastice from Jetbrains for a great collaboration on the Build
    Server Protocol specification and bsp4j library.
Implement pretty basic language server and build client.
This is a large commit that implements Pretty Basic IDE features.

- "Import build" for sbt and Bloop projects using the Build Server Protocol.
- File save triggers compile in the build server and errors are
  displayed in the editor.
- Goto definition works for project sources and library dependencies.
- Goto definition works OK despite of syntax and compile errors in the
  open unsaved editor buffer.
- Comprehensive integration tests that are easy to write and read.
- Documentation for how to
  - use Metals with the editors Vim, VS Code, Sublime and Atom
  - import sbt and Bloop build
  - integrate new build tools and text editors

Big thanks to the following people who made this milestone possible.
- Jorge Vicente Cantero for his great work in Bloop and helping out with
  unblocking a lot of Metals issues. Most the heavy lifting in "build import"
  is provided by Bloop.
- Eugene Burmako for close collaboration on Scalameta and SemanticDB
  that made the features in this PR possible.
- Gabriele Petronella and Alexey Alekhin for constantly trying out
  bleeding edge features and providing great feedback on WIP features.
- Justin Kaeser from Jetbrains for a great collaboration on the Build
  Server Protocol specification and bsp4j library.
@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 21, 2018

Demo of the new docs https://olafurpg.github.io/metals/

@gabro gabro self-requested a review Nov 21, 2018

Try to make Travis CI pass.
- remove unused imports
- fix file watcher to create directories that don't exist
- run actual tests on Windows
@olafurpg

This comment was marked as outdated.

Copy link
Member

olafurpg commented Nov 21, 2018

Here is pre-compiled VS Code extension if somebody wants to give it a try.

metals-vscode.zip

Remove dead code
- scripted tests are no longer needed since we stress sbt-metals
  in `unit/test`
- dead `start-server` script

@laughedelic laughedelic self-requested a review Nov 21, 2018

olafurpg added some commits Nov 22, 2018

Various cleanups
- Update docs
- Use 0.2.0-SNAPSHOT instead of SNAPSHOT to avoid picking up old
  artifacts
- Use Travis CI Windows support instead of Appveyor
Reinstate Appveyor config
- Windows job on Travis didn't start after waiting for 1hr30min.
- Created a new "Scalameta" organization account on Appveyor
  so that all collaborators can restart jobs.
@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 22, 2018

Got a working Windows VM setup and can reproduce the Windows failures without waiting for Appveyor 💪

olafurpg added some commits Nov 22, 2018

Try to make Windows happy
- embed sbt-launch.jar so that `sys.process("sbt")` succeeds
- avoid unnecessary I/O to prevent AccessDeniedException

Tests are still not passing when I try on my VM, the `build/initialize`
request is never responded. The request goes out but nothing comes back.
@gabro
Copy link
Collaborator

gabro left a comment

First round of comments (local and possibly low-level)

Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala Outdated
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/ActiveFiles.scala
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala Outdated
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala Outdated
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala Outdated
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopInstall.scala
*
* Connects to a running bloop server instance if it is installed on the user
* machine and has compatible version (+v1.1.0). Ignores the installed bloop
* server instance if it is v1.0.0.

This comment has been minimized.

@gabro

gabro Nov 22, 2018

Collaborator

Note that this causes the installed bloop client not to work anymore on the same project, because the newly written 1.1.0 JSON files are not compatible with the installed 1.0.0 client.

I don't think this is a Metals concern, but I just wanted to highlight this scenario /cc @jvican

This comment has been minimized.

@olafurpg

olafurpg Nov 25, 2018

Member

That's a good point, maybe bloop v1.1 should be v2 instead?

Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/BloopServers.scala Outdated
Show resolved Hide resolved metals/src/main/scala/scala/meta/internal/metals/DefinitionProvider.scala Outdated
Fix tests on Windows and address review feedback
It took a long time to track down the Windows issues and the resulting
diffs were small. Fortunately, now I have a decent Windows VM setup
so it's easier to reproduce bugs and validate fixes.

olafurpg added some commits Nov 23, 2018

Furter refinements
- use thread safe hashmaps in mtags (to avoid infinite loop)
- throttle status bar in vim to not spam the user with showMessageRequest
- more JDK src.zip candidates
- increase tcp retry number
- change how sbt checksum status is queried
- elaborate on comments

olafurpg added some commits Nov 27, 2018

Refactor sbt build import cancellation test
Instead of asserting against the specific state of the checkum (which is
fragile), we just assert whether the digest has been installed or not.
Rename sbt-metals package name.
This avoids the dupious `sbt` package name.
@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

Hi,

I followed manual installation steps closely and I can't get metals work with vim-lsc. sbt metalsEnable bloopInstall finishes successfully, but when I open build.sbt/*.scala file, vim gives me [lsc:Error] StdErr from metals-vim: ^[[2K.

I'd like to give you more info, but that's everything I have. There is no .metals directory and I don't know where to look for logs. Do you have any ideas on how to get diagnostics?

@Duhemm

This comment has been minimized.

Copy link
Contributor

Duhemm commented Nov 27, 2018

@lambdas, You can get more logs from vim by starting vim like this: vim -V9logfile.txt, which will write the logs in logfile.txt. Keep in mind that there's some buffering, so you may have to exit vim to have everything flushed to the log file.

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

@lambdas Thanks for reporting! Are you on Windows? I see ^[[2K in the vscode logs but it never caused problems, maybe this can be fixed by redirecting stderr to the metals logs.

@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

Thanks for such a fast answers! I'm on linux. Give me a couple of minutes to investigate vim logs and I'll report it back to you.

The single thing I can think of right now is that I was unable to get the exact sbt-bloop version specified in the manual, so I'm using 1.0.0 instead, can it cause such issues?

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Hmm, we already redirect stderr so that should not be the cause. This sounds familiar because I see the following in vscode whenever I start the plugin

�[2Klogging to file /Users/olafurpg/dev/metals/test-workspace/.metals/metals.log
Resetting Metals database /Users/olafurpg/dev/metals/test-workspace/.metals/metals.h2.db

screenshot 2018-11-27 at 12 38 19

Something somewhere is sending the clear ansi code 🤔

I can still use vim-lsc without problems..

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

The error happens before the metals server has started properly (hence no .metals/ directory). I don't think the problem is caused by sbt-bloop, and v1.0.0 should be fine btw.

@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

Ok, I've disabled all vim plugins except for vim-lsc and vim-scala to not to interfere with things. Here is the log https://pastebin.com/RBt97we5 relevant lines are around :521.

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Thank you @lambdas can you check if metals has logged to the global log file? On macos it is located in the user cache directory

tail -f ~/Library/Caches/org.scalameta.metals/global.log

On linux it should be $XDG_CACHE_HOME or $HOME/.cache

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Can you try suppressing stderr? From :help lsc

                                                *lsc-server-suppress_stderr*
`'suppress_stderr'`: Set to `v:true` to suppress stderr output which is
received after the server has been initialized. Stderr output before
initialization still is shown in an error message. Use this if the server uses
stderr for noisy status or logging.
@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Here's where the error is reported in vim-lsc https://github.com/natebosch/vim-lsc/blob/dc21e35abe7922ed239846d0459d55db227856fa/autoload/lsc/server.vim#L273-L276 and it seems guarded by that setting. If that works then I can update the vim installation instructions to suppress stderr.

@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

@olafurpg Just did a full fs search for anything related to metals log and found nothing, sorry. The option helps to get rid of the obscure error, but metals still doesn't start.

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

@lambdas Did you clone my too-large branch and run sbt publishLocal?

I think it's best to wait until this PR is merged and released so we can reproduce more easily. Thanks for trying it out so early!

@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

@olafurpg Sure, I cloned it yesterday, let me pull it and build again..

@lambdas

This comment has been minimized.

Copy link

lambdas commented Nov 27, 2018

@olafurpg Ok, I just did it again and somehow it started(did I mess up with coursier built launcher? maybe). Then I went to metals web interface(btw, it's awesome!) and clicked Import build. After that go to definition started to work perfectly(even for library types).

I'm going to spend some time exploring metals and vim-lsc now..

Thank you for your help and your awesome work!

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Glad to hear you got it working and the http interface helped @lambdas 👍

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

@laughedelic great catch, that sounds very plausible. If that is the case then I can update the instructions to use --standalone so that coursier bootstrap builds a fatjar with fully resolved and downloaded jars.

@olafurpg olafurpg merged commit 2ae2b6e into scalameta:master Nov 27, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@olafurpg olafurpg deleted the olafurpg:too-large branch Nov 27, 2018

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

Merging since the CI is stable and a few people have already reported back that they got the server working (although with minor bumps along the way). Big thanks to @gabro for reviewing this huge PR! Further improvements can be made in followup PRs, I will try to keep the changes smaller from now on 😼

@olafurpg

This comment has been minimized.

Copy link
Member

olafurpg commented Nov 27, 2018

@laughedelic @lambdas PR to coursier removing the escape ansi code when there are no jars to be downloaded by the bootstrap script coursier/coursier#966

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment