Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LanguageServer can connect and provide basic functionality to NeoVim client #9

Open
4 tasks
scztt opened this issue Apr 12, 2022 · 18 comments
Open
4 tasks

Comments

@scztt
Copy link
Owner

scztt commented Apr 12, 2022

Based on some preliminary investigation, NeoVim can only connect to LSP servers via a stdin/stdout based communication mechanism. This currently will not work with sclang, as there are many places where sclang forcibly writes to stdout, which would disrupt LSP communication. Instead, for the vscode plugin, we use simple UDP pipes since they are already well-supported by sclang and easy to implement in VSCode. Until the sclang stdout issues can be solved, the UDP mechanism will be used.

IF it is indeed not possible to convince NeoVim to connect to an LSP server via UDP, then the solution to this would be as follows. (Note that this is not throwaway work, as an architecture where the LSP "wrapper" and an sclang instance are separated is probably desirable in the long run anyway.)

There is a preliminary implementation that may be a good starting point here:
https://github.com/davidgranstrom/sclang-lsp-stdio

Note that there MAY be extra work in NeoVim specifically to enable SuperCollider support. There are forum messages outlining some of this work, and some things may already be implemented. See:

@ranjithshegde
Copy link

For information, Neovim 0.9 (or perhaps even earlier version) provides a method to connect to LSP rpc via tcp in function vim.lsp.rpc.connect

So something in these lines is already possible

local function start_lsp()
    vim.loop.spawn('/path/to/sclang', {args = {'-i', 'vscode'},}, function ()
       -- Some shutdown mechanism here 
    end)
end

vim.lsp.start({
  name="supercollider",
  cmd=function(...)
    start_lsp()
    return vim.lsp.rpc.connect(host, port)
  end
})

@dannyZyg
Copy link

Is the ideal here to write a program that results in a binary executable (or is python/node/lua fine)? There is a note in David's original proof of concept that suggests it would be good to not have a dependency on Node.

There are many LSPs which are installed via npm etc. so maybe node or whatever is fine.

I'd be happy to give this one a go if there is nobody working on it yet.

@dannyZyg
Copy link

dannyZyg commented Oct 2, 2023

I've had some time and have been working on a Python script to handle this. I've got most of the functionality in place but I've been testing outside of SC mostly.

When I try to run SC from my python script I get this (permission?) error.

The command I run on Mac is /Applications/SuperCollider.app/Contents/MacOS/sclang -l ~/Library/Application\ Support/SuperCollider/sclang_conf.yaml

I had to point it to my config file so it would pick up the Quarks.

Below is the error I get but strangely if I just run from the shell it works fine and I see the ***LSP READY*** message.

I'm using SCLANG_LSP_ENABLE=1.

Does anyone know what might be going on? Thanks!

compiling class library...
        Found 859 primitives.
        Compiling directory '/Applications/SuperCollider.app/Contents/Resources/SCClassLibrary'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Singleton'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Collapse'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/WindowViewRecall'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Log'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/UnitTest2'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Deferred'
        Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/LanguageServer'
        numentries = 877100 / 13676400 = 0.064
        5800 method selectors, 2358 classes
        method table size 14477520 bytes, big table size 109411200
        Number of Symbols 13017
        Byte Code Size 397053
        compiled 368 files in 0.37 seconds
compile done
ERROR: Primitive '_FileMkDir' failed.
caught exception 'boost::filesystem::create_directory: Read-only file system: "/synthdefs"' in primitive in method Meta_File:mkdir
RECEIVER:
class File (0x128c7e700) {
  instance variables [19]
    name : Symbol 'File'
    nextclass : instance of Meta_FileDialog (0x1287aa880, size=19, set=5)
    superclass : Symbol 'UnixFILE'
    subclasses : nil
    methods : instance of Array (0x128c7e880, size=12, set=4)
    instVarNames : instance of SymbolArray (0x128c7ea00, size=1, set=2)
    classVarNames : instance of SymbolArray (0x128c7eb80, size=1, set=2)
    iprototype : instance of Array (0x128c7eac0, size=1, set=2)
    cprototype : instance of Array (0x128c7ec40, size=1, set=2)
    constNames : nil
    constValues : nil
    instanceFormat : Integer 0
    instanceFlags : Integer 0
    classIndex : Integer 1119
    classFlags : Integer 0
    maxSubclassIndex : Integer 1119
    filenameSymbol : Symbol '/Applications/SuperCollider.app/Contents/Resources/SCClassLibrary/Common/Files/File.sc'
    charPos : Integer 0
    classVarIndex : Integer 220
}
CALL STACK:
        MethodError:reportError
                arg this = <instance of PrimitiveFailedError>
        Nil:handleError
                arg this = nil
                arg error = <instance of PrimitiveFailedError>
        Thread:handleError
                arg this = <instance of Thread>
                arg error = <instance of PrimitiveFailedError>
        Object:throw
                arg this = <instance of PrimitiveFailedError>
        Object:primitiveFailed
                arg this = <instance of Meta_File>
        String:mkdir
                arg this = "/synthdefs/"
        Meta_SynthDef:initClass
                arg this = <instance of Meta_SynthDef>
        Meta_Class:initClassTree
                arg this = <instance of Meta_Class>
                arg aClass = <instance of Meta_SynthDef>
                var implementsInitClass = nil
        ArrayedCollection:do
                arg this = [*217]
                arg function = <instance of Function>
                var i = 5
        Meta_Class:initClassTree
                arg this = <instance of Meta_Class>
                arg aClass = <instance of Meta_Object>
                var implementsInitClass = nil
        Process:startup
                arg this = <instance of Main>
                var time = 0.37496
        Main:startup
                arg this = <instance of Main>
                var didWarnOverwrite = false
^^ The preceding error dump is for ERROR: Primitive '_FileMkDir' failed.
caught exception 'boost::filesystem::create_directory: Read-only file system: "/synthdefs"' in primitive in method Meta_File:mkdir
RECEIVER: File

@jamshark70
Copy link
Contributor

SynthDef initClass does:

	*initClass {
		synthDefDir = Platform.userAppSupportDir ++ "/synthdefs/";
		// Ensure exists:
		synthDefDir.mkdir;
	}

If synthDefDir == "/synthdefs/", then it means Platform.userAppSupportDir must be returning an empty string.

userAppSupportDir is populated in C++, in SC_Filesystem_macos.cpp. So the code in this file must not be compatible with the way neovim is launching sclang.

@dannyZyg
Copy link

dannyZyg commented Oct 2, 2023

Thanks for info, I was able to track down the issue by looking at that SC_Filesystem_macos.cpp file. I was setting some env vars for my subprocess but was overwriting my default env so was missing things like HOME.

Thanks!

@dannyZyg
Copy link

dannyZyg commented Oct 3, 2023

I've almost got this working - I think i need to work on my neovim lsp setup because currently when I open a SC file in neovim it will launch my lsp runner but it won't show a connected LSP.

It is definitely launching though and producing promising logs

───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ DEBUG:asyncio:Using selector: KqueueSelector
   2   │ INFO:__main__:RUNNER: SC env vars: {'SCLANG_LSP_ENABLE': '1', 'SCLANG_LSP_CLIENTPORT': '57210', 'SCLANG_LSP_SERVERPORT': '57211', 'SCLANG_LSP_LOGLEVEL': 'info'}
   3   │ INFO:__main__:RUNNER: Launching SC with cmd: ['/Applications/SuperCollider.app/Contents/MacOS/sclang', '-i', 'external', '-l', '/Users/danny/Library/Application Support/SuperCollider/sclang_conf.yaml']
   4   │ INFO:__main__:SC:STDOUT: compiling class library...
   5   │ INFO:__main__:SC:STDOUT:    Found 859 primitives.
   6   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Applications/SuperCollider.app/Contents/Resources/SCClassLibrary'
   7   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/.local/share/SuperCollider/Extensions'
   8   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Singleton'
   9   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Collapse'
  10   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/WindowViewRecall'
  11   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Log'
  12   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/UnitTest2'
  13   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/Deferred'
  14   │ INFO:__main__:SC:STDOUT:    Compiling directory '/Users/danny/Library/Application Support/SuperCollider/downloaded-quarks/LanguageServer'
  15   │ INFO:__main__:SC:STDOUT:    numentries = 877100 / 13676400 = 0.064
  16   │ INFO:__main__:SC:STDOUT:    5800 method selectors, 2358 classes
  17   │ INFO:__main__:SC:STDOUT:    method table size 14477520 bytes, big table size 109411200
  18   │ INFO:__main__:SC:STDOUT:    Number of Symbols 13017
  19   │ INFO:__main__:SC:STDOUT:    Byte Code Size 397053
  20   │ INFO:__main__:SC:STDOUT:    compiled 368 files in 0.32 seconds
  21   │ INFO:__main__:SC:STDOUT: compile done
  22   │ INFO:__main__:SC:STDOUT: localhost : setting clientID to 0.
  23   │ INFO:__main__:SC:STDOUT: internal : setting clientID to 0.
  24   │ INFO:__main__:SC:STDOUT: Class tree inited in 0.01 seconds
  25   │ INFO:__main__:SC:STDOUT: [LANGUAGESERVER.QUARK] Starting language server, inPort: 57210 outPort:57211
  26   │ INFO:__main__:SC:STDOUT: [LANGUAGESERVER.QUARK] Adding provider for method 'initialize'
  27   │ INFO:__main__:SC:STDOUT: ***LSP READY***
  28   │ INFO:__main__:RUNNER: ready message received
  29   │ INFO:__main__:SC:STDOUT: *** Welcome to SuperCollider 3.14.0-dev. *** For help type cmd-d.
  30   │ INFO:__main__:UDP SERVER: connection made
  31   │ INFO:__main__:UDP SERVER: connection made
  32   │ INFO:__main__:RUNNER: UDP Sender running on 127.0.0.1:57210
  33   │ INFO:__main__:SC:STDOUT: [LANGUAGESERVER.QUARK] Message received: 0.435571125, a NetAddr(127.0.0.1, 57845), Content-Length: 3645

strangely when i exit the neovim buffer I get this (which almost looks like an init call)

  34   │ INFO:__main__:SC:STDOUT: [LANGUAGESERVER.QUARK] Message received: 6.327646292, a NetAddr(127.0.0.1, 57845), {"params":{"rootUri":"file:\/\/\/Users\/danny\/Music","workspaceFolders":[{"name":"\/Users\/danny\/Music","uri":"file:\/\/\/Users\/
       │ danny\/Music"}],"initializationOptions":{},"processId":39910,"clientInfo":{"version":"0.9.1","name":"Neovim"},"trace":"off","capabilities":{"textDocument":{"callHierarchy":{"dynamicRegistration":false},"declaration":{"linkSupport":true},"r
       │ eferences":{"dynamicRegistration":false},"documentHighlight":{"dynamicRegistration":false},"documentSymbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSuppor
       │ t":true,"dynamicRegistration":false},"signatureHelp":{"signatureInformation":{"parameterInformation":{"labelOffsetSupport":true},"documentationFormat":["markdown","plaintext"],"activeParameterSupport":true},"dynamicRegistration":false},"im
       │ plementation":{"linkSupport":true},"synchronization":{"willSave":true,"didSave":true,"willSaveWaitUntil":true,"dynamicRegistration":false},"publishDiagnostics":{"tagSupport":{"valueSet":[1,2]},"relatedInformation":true},"codeAction":{"code
       │ ActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"resolveSupport":{"properties":["edit"]},"isPreferredSupport":true,"
       │ dataSupport":true,"dynamicRegistration":false},"rename":{"prepareSupport":true,"dynamicRegistration":false},"typeDefinition":{"linkSupport":true},"definition":{"linkSupport":true},"completion":{"contextSupport":true,"completionItem":{"snip
       │ petSupport":true,"commitCharactersSupport":true,"preselectSupport":true,"deprecatedSupport":true,"documentationFormat":["markdown","plaintext"],"insertReplaceSupport":true,"insertTextModeSupport":{"valueSet":[1,2]},"labelDetailsSupport":tr
       │ ue,"resolveSupport":{"properties":["documentation","detail","additionalTextEdits","sortText","filterText","insertText","textEdit","insertTextFormat","insertTextMode"]},"tagSupport":{"valueSet":[1]}},"insertTextMode":1,"completionList":{"it
       │ emDefaults":["commitCharacters","editRange","insertTextFormat","insertTextMode","data"]},"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]},"dynamicRegistration":false},"semanticTokens":{"
       │ tokenTypes":["namespace","type","class","enum","interface","struct","typeParameter","parameter","variable","property","enumMember","event","function","method","macro","keyword","modifier","comment","string","number","regexp","operator","de
       │ corator"],"formats":["relative"],"overlappingTokenSupport":true,"multilineTokenSupport":false,"serverCancelSupport":false,"augmentsSyntaxTokens":true,"tokenModifiers":["declaration","definition","readonly","static","deprecated","abstract",
       │ "async","modification","documentation","defaultLibrary"],"requests":{"range":false,"full":{"delta":true}},"dynamicRegistration":false},"hover":{"contentFormat":["markdown","plaintext"],"dynamicRegistration":false}},"workspace":{"workspaceF
       │ olders":true,"applyEdit":true,"workspaceEdit":{"resourceOperations":["rename","create","delete"]},"symbol":{"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalWorkspaceSymbolSuppor
       │ t":true,"dynamicRegistration":false},"semanticTokens":{"refreshSupport":true},"didChangeWatchedFiles":{"relativePatternSupport":true,"dynamicRegistration":false},"configuration":true},"window":{"workDoneProgress":true,"showMessage":{"messa
       │ geActionItem":{"additionalPropertiesSupport":false}},"showDocument":{"support":true}}},"rootPath":"\/Users\/danny\/Music"},"id":1,"method":"initialize","jsonrpc":"2.0"}
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

@themissingcow
Copy link

@dannyZyg did you get any further with this - I should have some time to help soon. I would love to get it all working in neovim.

@dannyZyg
Copy link

dannyZyg commented Mar 6, 2024

@dannyZyg did you get any further with this - I should have some time to help soon. I would love to get it all working in neovim.

Hey @themissingcow - I haven't looked at this since last year, but I'd love to get it going if possible. I'll try and have a look in the next few days and let you know how I go. I can't quite remember where I got to with it.

@themissingcow
Copy link

@dannyZyg did you get any further with this - I should have some time to help soon. I would love to get it all working in neovim.

Hey @themissingcow - I haven't looked at this since last year, but I'd love to get it going if possible. I'll try and have a look in the next few days and let you know how I go. I can't quite remember where I got to with it.

🙏

Ace, thanks. I grabbed that branch on your fork - but had issues manually calling vim.lsp.start with the python script - it would spin up the command, so the output.log file appeared, and ready the LSP, but nvim would never attach it to the buffer... The inner workings of LSPs in nvim is new to me, so going to start from the top to try and understand it a little more.

@themissingcow
Copy link

we use simple UDP pipes since they are already well-supported by sclang

@scztt please forgive the naive q - has there been any chat upstream about adding tcp support to sclang?

@themissingcow
Copy link

@dannyZyg I managed to get this working in nvim, it needed a few tweaks to LSP.sc and the runner script. I'll try to package these up soon once I've tidied them up a little. Many thanks to you and @scztt for all the hard work on getting it this far - this is going to be so helpful!

@dannyZyg
Copy link

dannyZyg commented Mar 7, 2024

That's great @themissingcow ! Really glad to hear you carried on with it and fixed it. Curious to see what changes were needed. Excited to use this too!

@themissingcow
Copy link

That's great @themissingcow ! Really glad to hear you carried on with it and fixed it. Curious to see what changes were needed. Excited to use this too!

WIP: https://github.com/themissingcow/LanguageServer.quark/commits/nvim-fixes

Still having some issues where the last packet of multi-packet messages is getting lost. Should hopefully have some time next week to look into it again, and will post the updated Python wrapper.

@themissingcow
Copy link

@dannyZyg got bit further here https://github.com/themissingcow/LanguageServer.quark/commits/lsp-runner. Were you planning on working on the wrapper script some more? Otherwise I was thinking about adding support for configuring paths/config and the likes via LSP initialization options, so you can set these up in your neovim registration (and potentially per-project). Didn't want to mess with your plans though!

@dannyZyg
Copy link

Hey @themissingcow, thanks for continuing with this! I was planning on coming back to it at some point but had not got around to it, so I'm really pleased you picked it up. Your config suggestions sound good, I'd be happy for you to add that. It would be great to get all of this merged in though, so we can make small improvements as we go and have something workable soon! Let me know how you'd like to proceed with that?

What state is it in now? I'll try and test out your changes tonight!

@themissingcow
Copy link

Hey @dannyZyg

Your config suggestions sound good, I'd be happy for you to add that. It would be great to get all of this merged in though, so we can make small improvements as we go and have something workable soon! Let me know how you'd like to proceed with that?

Yeah def. I guess I was hoping to get the setting from the LSP initialise working as a starting point, but I guess it's kind of working as is.

What state is it in now? I'll try and test out your changes tonight!

I was just testing and cross-referenced the LSP docs and I realised that some of the changes I made in LSP.sc aren't actually needed. Ill update the branch to remove those (sorry I'll force-push vs having some revert commits, as we'd def not want to merge it upstream with those in).

@dannyZyg
Copy link

Yeah def. I guess I was hoping to get the setting from the LSP initialise working as a starting point, but I guess it's kind of working as is.

@themissingcow That's no problem, feel free to add that in! I don't mean to rush you, I'm just excited to use this haha 😄

@themissingcow
Copy link

themissingcow commented Mar 12, 2024

@themissingcow That's no problem, feel free to add that in! I don't mean to rush you, I'm just excited to use this haha 😄

@dannyZyg I updated the branch, I have it kind of working (though I think there are still some gremlins with large messages).

I'm adding this to my lspconfig.configs:

local configs = require('lspconfig.configs')

 configs.supercollider = {
   default_config = {
	 cmd = {"python", "/path/to/lsp_runner/main.py", "--sc-lang-path", '/Applications/SuperCollider.app/Contents/MacOS/sclang', "--sc-config-path", "/path/to/sclang_conf.yaml"},
	 filetypes = {'supercollider'},
	 root_dir = function(fname)
		return "/"
	 end,
	 settings = {},
   },
 }

And setting this up with require("lspconfig.configs").supercollider.setup({}), for some reason I had to override the builtin filetype detection as it was flagging my .sc files as Scala despite the logic in filetypes.lua...

vim.filetype.add({
  extension = {
    sc = "supercollider",
    scd = "supercollider",
  },
})

Which gives me this loveliness:

image

dannyZyg added a commit to dannyZyg/LanguageServer.quark that referenced this issue Mar 20, 2024
Provides a python script to support lsp clients which can't use UDP but
can use stdio (e.g. neovim) - as described in this issue scztt#9

The script launches sclang in the background and while communicating
with the language server quark over UDP, relays this IO over stdio to
the client.

Special thanks to Tom Cowland for getting this working properly!

Co-authored-by: Tom Cowland <info@tomcowland.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

5 participants