Option to put Julia connector pipe in home directory

[jacobadenbaum]

Currently generatePipeName puts the pipe for the Julia connector in the /tmp/ directory on Linux.

julia-vscode/src/utils.ts

Lines 24 to 31 in b8af1ed

	export function generatePipeName(pid: string, name: string) {
	if (process.platform === 'win32') {
	return '\\\\.\\pipe\\' + name + '-' + pid
	}
	else {
	return path.join(os.tmpdir(), name + '-' + pid)
	}
	}

This can cause problems attaching to an external REPL if the julia process is running on a separate machine. For instance if you are working on a cluster, and you want to launch an interactive Julia process through the cluster's scheduler (i.,e via srun on Slurm), and then connect to that interactive REPL, then the current process for connecting to an external REPL will throw an error because that location is not accessible on the remote machine.

Is it possible to add an option that allows the Julia connector pipes to live in the ~/.vscode/ directory instead? Or is there another solution for how to connect to an external repl that would work in this situation that I could use instead?

[davidanthoff]

The entire design right now makes a lot of assumptions that the REPL will run on the same machine as the extension. We could explore an option to connect to remote REPLs, but I think that is probably more involved than just this change here.

[pfitzseb]

Wouldn't a shared FS be enough? All communication happens through that pipe anyways.

[davidanthoff]

Isn't the question then whether the extension code itself is also on a shared FS?

[jacobadenbaum]

I think that it generally is, isn't it? When you log into remote machines, the vscode remote development extension automatically installs all the extension code into your home folder (in my setup, it goes into ~/.vscode-server/extensions/). My home directory is accessible to all of the nodes in the cluster, it's just the /tmp directory that isn't shared between them I think.

[davidanthoff]

I have an even more basic question: at least on Windows named pipes can never cross a machine boundary, isn’t that the same on Linux? We would probably have to use sockets to communicate across machine boundaries?

[pfitzseb]

I thought you could access named pipes from different machines for some reason, but of course that doesn't work (which totally makes sense -- going through the FS for pipes would be a performance nightmare).
So yes, we'd need a socket based setup.

[jacobadenbaum]

Ahh well that's a bummer. It would be really cool to have something like that, but I totally understand that it would be more work to implement than just "put the pipe in the home directory"

[JamieMair]

Is there any way to bump up this issue? Having this feature is extremely important when using VS Code on a cluster, since one should not run code on the login nodes (so the normal REPL should not be used for developing/testing). Instead, you have to allocate a REPL session on a different node. This node will have access to the shared home directory of the user, so it makes sense to put any files there.

[JamieMair]

I have just attempted to change the file listed above to:

export function generatePipeName(pid: string, name: string) {
    if (process.platform === 'win32') {
        return '\\\\.\\pipe\\' + name + '-' + pid
    }
    else {
        const tmpFolder = path.join(os.homedir(), ".tmp");
        // Ensure the temporary folder exists
        if (!fs.existsSync(tmpFolder)) {
            fs.mkdirSync(tmpFolder, { recursive: true});
        }
        return path.join(tmpFolder, name + '-' + pid)
    }
}

And this works fine on the local machine as expected, but there is an issue when trying to access it on another machine.

julia> pushfirst!(LOAD_PATH, raw"/nfsshare/home/USER/julia-vscode/scripts/packages");using VSCodeServer;popfirst!(LOAD_PATH);VSCodeServer.serve(raw"/nfsshare/home/USER/.tmp/vsc-jl-repl-08e85473-de66-4912-b98a-fce70364208f"; is_dev = "DEBUG_MODE=true" in Base.ARGS, crashreporting_pipename = raw"/nfsshare/home/USER/.tmp/vsc-jl-cr-0d4394fb-33fa-4f97-abd8-38ffb1bd219f");nothing # re-establishing connection with VSCode
ERROR: IOError: connect: connection refused (ECONNREFUSED)
Stacktrace:
 [1] wait_connected(x::Base.PipeEndpoint)
   @ Sockets /nfsshare/software/julia/v1.7.3/usr/share/julia/stdlib/v1.7/Sockets/src/Sockets.jl:532
 [2] connect
   @ /nfsshare/software/julia/v1.7.3/usr/share/julia/stdlib/v1.7/Sockets/src/Sockets.jl:567 [inlined]
 [3] connect
   @ /nfsshare/software/julia/v1.7.3/usr/share/julia/stdlib/v1.7/Sockets/src/PipeServer.jl:97 [inlined]
 [4] serve(args::String; is_dev::Bool, crashreporting_pipename::String)
   @ VSCodeServer ~/julia-vscode/scripts/packages/VSCodeServer/src/VSCodeServer.jl:110
 [5] top-level scope
   @ REPL[1]:1

I don't mind looking into this, but I am not sure if I am capable of adding this functionality.

[davidanthoff]

Didn't we conclude that this in general won't work across machines because named pipes don't support that?

[JamieMair]

Is there any way to forward the pipe or socket over something like ssh (which is usually available on clusters)?

EDIT: There is a tool called socat which can be used to forward named pipes on linux. I am testing it now, and running the command (on the remote machine):

socat PIPE:/nfsshare/home/USER/.tmp/vsc-jl-repl-pipe SYSTEM:"ssh USER@LOGINNODE socat - /nfsshare/home/USER/.tmp/vsc-jl-repl-5d5c219e-b9b8-4771-ae75-92bd2381b232"

Gives the notification that the REPL is successfully connected. I also did this for the crash reporter named pipe. The only issue is that when we paste the new command into the remote REPL:

pushfirst!(LOAD_PATH, raw"/nfsshare/home/USER/julia-vscode/scripts/packages");using VSCodeServer;popfirst!(LOAD_PATH);VSCodeServer.serve(raw"/nfsshare/home/USER/.tmp/vsc-jl-repl-pipe"; is_dev = "DEBUG_MODE=true" in Base.ARGS, crashreporting_pipename = raw"/nfsshare/home/USER/.tmp/vsc-jl-cr-pipe");nothing # re-establishing connection with VSCode

The process hangs and does not connect. I think this is because the socat command already opened the connection.

[davidanthoff]

I think if we wanted to support remote REPL processes we should a) do that properly via sockets and b) ideally do this once we have some kind of support for multiple active REPL sessions in the UI.