Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start Ydoc with the language server #9862

Merged
merged 68 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
a0c39a8
feat: Ydoc.start
4e6 May 3, 2024
ab855fb
feat: YdocTest
4e6 May 7, 2024
d31c3c3
Exclude helidon modules from ydoc-server fat jar
Akirathan Apr 30, 2024
4896659
Run ydoc-server as a module
Akirathan Apr 30, 2024
4966272
Fix some typos
Akirathan May 6, 2024
a1b389d
Ability to run ydoc-server with `ydoc-server/run`.
Akirathan May 6, 2024
119c888
Ydoc throws AssertionError if it cannot find the ydocServer.js resource
Akirathan May 6, 2024
924006f
fmt
Akirathan May 6, 2024
9e1f954
Add logback modules to ydoc-server module-path
Akirathan May 6, 2024
aafe692
Copy all helidon modules to the component directory
Akirathan May 6, 2024
3b83248
Add chrome-inspector modules on ydoc-server module-path
Akirathan May 6, 2024
16289a7
feat: build.sbt filter helidon jars
4e6 May 9, 2024
0074b67
feat: terrible hack
4e6 May 9, 2024
eea99a0
feat: configure vite
4e6 May 10, 2024
8067e10
fix: helidon version
4e6 May 13, 2024
24d5502
fix: fmt
4e6 May 13, 2024
27d282f
misc: cleanup
4e6 May 13, 2024
df064bd
feat: configurable ydoc url
4e6 May 13, 2024
bd9e27d
feat: remove ydoc assembly
4e6 May 13, 2024
cdd9151
misc: remove unused setting
4e6 May 13, 2024
03be6e2
misc: cleanup
4e6 May 13, 2024
3f73fbe
misc: MainModule cleanup
4e6 May 13, 2024
3f90426
feat: ydoc bundle generator
4e6 May 14, 2024
2b9021f
misc: lint
4e6 May 14, 2024
075132e
revert: remove unused setting
4e6 May 14, 2024
cf0157b
fix: tests execution
4e6 May 14, 2024
34c570d
fix: runtime-benchmarks module path
4e6 May 14, 2024
fec85b4
fix: windows npm command
4e6 May 14, 2024
4434c47
fix: runtime-integration-tests module path
4e6 May 14, 2024
f0e5142
fix: licenses: filter absolute file names in jars
4e6 May 15, 2024
3ec225f
misc: legal review
4e6 May 15, 2024
55e1c71
DRAFT: node workflow
4e6 May 15, 2024
3de57bd
DRAFT: node workflow 1
4e6 May 15, 2024
206dd41
DRAFT: generate workflow
4e6 May 15, 2024
0200619
DRAFT: generate workflow 2
4e6 May 15, 2024
32bad7e
DRAFT: workflow install-command
4e6 May 15, 2024
afb433a
DRAFT: workflow npm install
4e6 May 15, 2024
1b95e04
misc: trigger rebuild
4e6 May 15, 2024
8a16bf9
DRAFT: workflow setup python
4e6 May 16, 2024
f179fcb
update: legal review
4e6 May 17, 2024
9fbc231
misc: review comments
4e6 May 17, 2024
8fa441d
feat: name ydoc executor thread
4e6 May 17, 2024
fe8e027
update: ydoc bundle directory
4e6 May 17, 2024
a79df9e
Merge branch 'develop' into wip/db/ydoc-language-server
4e6 May 18, 2024
2b86a67
fix: after merge
4e6 May 18, 2024
2201242
misc: javafmt
4e6 May 19, 2024
89d147b
ci: fix Build Backend job
4e6 May 19, 2024
bc6717c
ci: fix Standard Library Tests job
4e6 May 19, 2024
1cf794a
feat: ydoc builder
4e6 May 20, 2024
5358331
misc: javafmt
4e6 May 20, 2024
fe8bd42
misc: legal review pureconfig
4e6 May 20, 2024
ed328b8
fix: application config loading
4e6 May 20, 2024
7b94186
feat: remove special undefined value
4e6 May 20, 2024
891fb49
misc: lint
4e6 May 20, 2024
4cad2dd
fix: gui tests
4e6 May 20, 2024
b92bb31
feat: npm install
4e6 May 20, 2024
38edb12
misc: lint
4e6 May 20, 2024
914a4dc
DRAFT: workflow python
4e6 May 20, 2024
fbc910c
ci: revert workflow
4e6 May 20, 2024
b6b91a2
DEBUG: windows
4e6 May 20, 2024
23ee992
DEBUG: windows 1
4e6 May 20, 2024
758714b
DEBUG: windows 2
4e6 May 20, 2024
c9f6562
DEBUG: cleanup
4e6 May 20, 2024
bfd9e10
fix: media type
4e6 May 21, 2024
d6bc5d7
Merge branch 'develop' into wip/db/ydoc-language-server
4e6 May 23, 2024
4c6997a
feat: verify licenses
4e6 May 23, 2024
8bd081c
Merge branch 'develop' into wip/db/ydoc-language-server
4e6 May 28, 2024
2820762
revert: invalid change introduced in #9951
4e6 May 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/gui2/shared/ast/ffiPolyglot.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
* @file This file is used as ffi {@link module:ffi} interface for building the polyglot ydoc server.
* All the exported methods are provided by the ydoc server implementation.
* The interface should be kept in sync with Rust ffi inteface {@link module:ffi}.
* The interface should be kept in sync with Rust ffi interface {@link module:ffi}.
*
* @module ffiPolyglot
*/
Expand Down
2 changes: 1 addition & 1 deletion app/gui2/shared/languageServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ export type TransportEvents = {
* This client implements the [Language Server Protocol](https://github.com/enso-org/enso/blob/develop/docs/language-server/protocol-language-server.md)
*
* It also handles the initialization (and re-initialization on every reconnect); each method
* repressenting a remote call (except the `initProtocolConnection` obviously) waits for
* representing a remote call (except the `initProtocolConnection` obviously) waits for
* initialization before sending the request.
*/
export class LanguageServer extends ObservableV2<Notifications & TransportEvents> {
Expand Down
9 changes: 6 additions & 3 deletions app/gui2/src/stores/project/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,15 @@ function resolveLsUrl(config: GuiConfig): LsUrls {
if (engine.rpcUrl != null && engine.dataUrl != null) {
const dataUrl = engine.dataUrl
const rpcUrl = engine.rpcUrl
let ydocUrl
if (engine.ydocUrl == null || engine.ydocUrl === '') {

let ydocUrl: URL
if (engine.ydocUrl == '') {
ydocUrl = new URL(location.origin)
ydocUrl.protocol = location.protocol.replace(/^http/, 'ws')
} else if (URL.canParse(engine.ydocUrl)) {
ydocUrl = new URL(engine.ydocUrl)
} else {
ydocUrl = new URL(engine.rpcUrl)
ydocUrl = new URL(rpcUrl)
ydocUrl.port = '1234'
}
ydocUrl.pathname = '/project'
Expand Down
47 changes: 4 additions & 43 deletions app/gui2/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

import vue from '@vitejs/plugin-vue'
import { getDefines, readEnvironmentFromFile } from 'enso-common/src/appConfig'
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
import { existsSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import postcssNesting from 'postcss-nesting'
import tailwindcss from 'tailwindcss'
Expand All @@ -15,9 +13,7 @@ import { createGatewayServer } from './ydoc-server'
const projectManagerUrl = 'ws://127.0.0.1:30535'

const IS_CLOUD_BUILD = process.env.CLOUD_BUILD === 'true'
const IS_POLYGLOT_YDOC_SERVER_DEBUG = process.env.POLYGLOT_YDOC_SERVER_DEBUG === 'true'
const IS_POLYGLOT_YDOC_SERVER =
process.env.POLYGLOT_YDOC_SERVER === 'true' || IS_POLYGLOT_YDOC_SERVER_DEBUG
const POLYGLOT_YDOC_SERVER = process.env.POLYGLOT_YDOC_SERVER

await readEnvironmentFromFile()

Expand Down Expand Up @@ -62,7 +58,7 @@ export default defineConfig({
...getDefines(),
IS_CLOUD_BUILD: JSON.stringify(IS_CLOUD_BUILD),
PROJECT_MANAGER_URL: JSON.stringify(projectManagerUrl),
YDOC_SERVER_URL: IS_POLYGLOT_YDOC_SERVER ? JSON.stringify('defined') : undefined,
YDOC_SERVER_URL: JSON.stringify(POLYGLOT_YDOC_SERVER),
Frizi marked this conversation as resolved.
Show resolved Hide resolved
RUNNING_VITEST: false,
'import.meta.vitest': false,
// Single hardcoded usage of `global` in aws-amplify.
Expand Down Expand Up @@ -98,48 +94,13 @@ export default defineConfig({
},
})

let ydocServer: ChildProcessWithoutNullStreams | null
function gatewayServer(): Plugin {
return {
name: 'gateway-server',
configureServer(server) {
if (server.httpServer == null) return
if (POLYGLOT_YDOC_SERVER != undefined || server.httpServer == null) return

if (IS_POLYGLOT_YDOC_SERVER) {
const ydocServerJar = fileURLToPath(
new URL(
'../../lib/java/ydoc-server/target/ydoc-server-assembly-0.1.0-SNAPSHOT.jar',
import.meta.url,
),
)
const runYdocServer = () => {
const args = []
if (IS_POLYGLOT_YDOC_SERVER_DEBUG) {
args.push('-DinspectPort=34567')
}
args.push('-jar', ydocServerJar)
ydocServer = spawn('java', args)
if (IS_POLYGLOT_YDOC_SERVER_DEBUG) {
ydocServer.stdout.on('data', (data) => console.log(`ydoc: ${data}`))
}
ydocServer.stderr.on('data', (data) => console.log(`ydoc: ${data}`))
}
if (!existsSync(ydocServerJar)) {
const cwd = fileURLToPath(new URL('../..', import.meta.url))
const sbt = spawn('sbt', ['ydoc-server/assembly'], { cwd })
sbt.stdout.on('data', (data) => console.log(`sbt: ${data}`))
sbt.on('exit', runYdocServer)
} else {
runYdocServer()
}
} else {
createGatewayServer(server.httpServer, undefined)
}
},
buildEnd() {
if (ydocServer == null) return

ydocServer.kill('SIGTERM')
createGatewayServer(server.httpServer, undefined)
},
}
}
Expand Down
33 changes: 24 additions & 9 deletions app/gui2/vite.ydoc-server-polyglot.config.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import * as fs from 'node:fs'
import { fileURLToPath } from 'node:url'
import { defineConfig, type Plugin } from 'vite'
import defaultConfig from './vite.config'
Expand All @@ -13,22 +14,16 @@ export default defineConfig({
cacheDir,
publicDir,
envDir,
resolve: {
...resolve,
alias: {
...resolve?.alias,
// Use `ffiPolyglot` module as `ffi` interface during the build.
'shared/ast/ffi': fileURLToPath(new URL('./shared/ast/ffiPolyglot.ts', import.meta.url)),
},
},
resolve,
plugins: [usePolyglotFfi()],
define: {
...defaultConfig.define,
self: 'globalThis',
},
build: {
minify: false, // For debugging
emptyOutDir: true,
outDir: '../../lib/java/ydoc-server/target/classes/dist',
outDir: '../../lib/java/ydoc-server/target/ydoc-server-bundle',
rollupOptions: {
input: {
ydocServer: fileURLToPath(new URL('ydoc-server/indexPolyglot.ts', import.meta.url)),
Expand All @@ -39,3 +34,23 @@ export default defineConfig({
},
},
})

/**
* Use `ffiPolyglot` module as `ffi` interface during the build.
*/
function usePolyglotFfi(): Plugin {
const ffiPolyglot = fileURLToPath(new URL('./shared/ast/ffiPolyglot.ts', import.meta.url))
const ffiBackup = fileURLToPath(new URL('./shared/ast/ffiBackup.ts', import.meta.url))
const ffi = fileURLToPath(new URL('./shared/ast/ffi.ts', import.meta.url))

return {
name: 'use-polyglot-ffi',
options: () => {
fs.renameSync(ffi, ffiBackup)
fs.copyFileSync(ffiPolyglot, ffi)
},
buildEnd: () => {
fs.renameSync(ffiBackup, ffi)
},
}
}
8 changes: 7 additions & 1 deletion app/gui2/ydoc-server/indexPolyglot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@ declare global {
onconnect: ((socket: any, url: any) => any) | null
start(): void
}

const YDOC_HOST: string | undefined
const YDOC_PORT: number | undefined
}

const wss = new WebSocketServer({ host: 'localhost', port: 1234 })
const host = YDOC_HOST ?? 'localhost'
const port = YDOC_PORT ?? 1234

const wss = new WebSocketServer({ host, port })

wss.onconnect = (socket, url) => {
const doc = docName(url.pathname)
Expand Down