Fix self-hosting on Windows #6316

westbury · 2019-10-02T16:48:57Z

What it does

While validating #5849 on Windows I had to fix a couple of unrelated bugs. These fixes are required to get self hosting working on Windows.

There are two fixes here.

A simple change to convert URIs to paths using FileUri instead of a home-grown solution that does not work at all on Windows.
Set shell: true on the call to spawn. This is necessary for Windows though not necessary on Linux. I have tested it on Linux and it works well, so I suppose it comes down to performance and which is more robust when it comes to whether we also set 'shell' on Linux.

How to test

On Windows, run Hosted mode: Debug instance (use, for example, https://github.com/tom-shan/theia-plugin-hello-world). With the fixes you should see the self-hosting successfully start.

Review checklist

[x ] as an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

as a reviewer, I agree to behave in accordance with the review guidelines

benoitf · 2019-10-02T18:30:00Z

packages/plugin-dev/src/node/hosted-plugins-manager.ts

            type: LogType.Info
        });
        return Promise.resolve();
    }

-    protected unregisterWatchScript(path: string): void {
-        this.watchCompilationRegistry.delete(path);
+    protected unregisterWatchScript(pluginRootPath: string): void {


Hello, why do we need to rename path to pluginRootPath, it's adding extra changes ?

I think it improves the code, as it better describes the purpose of the variable and puts it more in line with the rest of the module.

@westbury Was there a compilation warning/error because of a shadowed module variable path?

@akosyakov The rename was not technically required, as I could have used a different name for the path import. I thought it was a good idea to make the code a little bit clearer by using the same name for paths that are specifically a path to the plugin root.

mmorhun

Tested on Linux only (Fedora 30) - works for me.

westbury · 2019-10-04T10:54:03Z

@kittaakos Do you have time to test this on Windows? This PR contains the fixes for the problems both you and I found when trying to validate #5849 on Windows.

@akosyakov I seem to have lost my permissions to assign reviewers, add tags etc, probably as a result of the move to eclipse-theia. I have no unaccepted invites that I can find.

akosyakov · 2019-10-04T11:13:09Z

@westbury please check there should be an invitation email since we moved a repo to a new org cc @marcdumais-work

marcdumais-work · 2019-10-04T12:27:28Z

@westbury I also could not find it at first. Here's the email title to search-for:
"[GitHub] @eclipsewebmaster has invited you to join the @eclipse-theia organization"

westbury · 2019-10-04T14:26:06Z

@marcdumais-work I have not been able to find the invitation anywhere. I would expect to find it at this URL irrespective of any missing e-mail:

Sorry to trouble you with this.

marcdumais-work · 2019-10-04T17:26:32Z

Hi @westbury ,

Just a sanity check: have you checked with your email address registered at the Foundation? It could be that the email ended-up in the spam folder and was automatically purged after a little while.

If still unresolved, I would suggest opening a bugzilla about his: start here and select component: GitHub. In the description mention that you did not receive the invite to join the eclipse-theia GH org.

kittaakos · 2019-10-07T06:55:03Z

@kittaakos Do you have time to test this on Windows?

Yes 👍

kittaakos · 2019-10-07T07:21:09Z

I cannot build it on my Windows image anymore. It used to work two weeks ago:

[4/10] ⠠ electron
error C:\Users\kittaakos\Desktop\theia\node_modules\fibers: Command failed.
Exit code: 1
Command: node build.js || nodejs build.js
Arguments:
Directory: C:\Users\kittaakos\Desktop\theia\node_modules\fibers
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@10.16.0 | win32 | x64
gyp info spawn C:\Users\kittaakos\.windows-build-tools\python27\python.EXE
gyp info spawn args [ 'C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-G',
gyp info spawn args   'msvs_version=2015',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\fibers\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\kittaakos\\.node-gyp\\10.16.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\kittaakos\\.node-gyp\\10.16.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\Users\\kittaakos\\.node-gyp\\10.16.0\\<(target_arch)\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\fibers',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\fibers\\build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args   '/clp:Verbosity=minimal',
gyp info spawn args   '/nologo',
gyp info spawn args   '/p:Configuration=Release;Platform=x64' ]
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  fibers.cc
  coroutine.cc
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(122): error C2660: 'v8::String::NewFromOneByte': func
tion does not take 2 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(2802): note: see declaration of 'v8::String::NewFromOneByte' (c
ompiling source file ..\src\fibers.cc)
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(126): error C2660: 'v8::String::NewFromOneByte': func
tion does not take 2 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(2802): note: see declaration of 'v8::String::NewFromOneByte' (c
ompiling source file ..\src\fibers.cc)
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(406): warning C4996: 'v8::String::Utf8Value::Utf8Valu
e': was declared deprecated [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(2891): note: see declaration of 'v8::String::Utf8Value::Utf8Val
ue'
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(433): error C2661: 'v8::Function::NewInstance': no ov
erloaded function takes 2 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(433): error C2672: 'uni::Return': no matching overloa
ded function found [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(621): error C2512: 'v8::TryCatch': no appropriate def
ault constructor available [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(8529): note: see declaration of 'v8::TryCatch' (compiling sourc
e file ..\src\fibers.cc)
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(628): warning C4996: 'v8::Script::Compile': was decla
red deprecated [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(1255): note: see declaration of 'v8::Script::Compile'
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(643): warning C4996: 'v8::TryCatch::StackTrace': was
declared deprecated [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(8593): note: see declaration of 'v8::TryCatch::StackTrace'
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(741): error C2661: 'v8::Value::ToNumber': no overload
ed function takes 0 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(793): error C2660: 'v8::Object::SetAccessor': functio
n does not take 2 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(3348): note: see declaration of 'v8::Object::SetAccessor' (comp
iling source file ..\src\fibers.cc)
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(794): error C2664: 'v8::Maybe<bool> v8::Object::SetAc
cessor(v8::Local<v8::Context>,v8::Local<v8::Name>,v8::AccessorNameGetterCallback,v8::AccessorNameSetterCallback,v8::Mayb
eLocal<v8::Value>,v8::AccessControl,v8::PropertyAttribute,v8::SideEffectType)': cannot convert argument 2 from 'uni::Fun
ctionType (__cdecl *)(v8::Local<v8::String>,const uni::GetterCallbackInfo &)' to 'v8::Local<v8::Name>' [C:\Users\kittaak
os\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(794): note: No constructor could take the source ty
pe, or constructor overload resolution was ambiguous
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\fibers.cc(795): error C2660: 'v8::Object::SetAccessor': functio
n does not take 2 arguments [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
  c:\users\kittaakos\.node-gyp\10.16.0\include\node\v8.h(3348): note: see declaration of 'v8::Object::SetAccessor' (comp
iling source file ..\src\fibers.cc)
  win_delay_load_hook.cc
c:\users\kittaakos\desktop\theia\node_modules\fibers\src\coroutine.cc(138): warning C4552: '!': result of expression not
 used [C:\Users\kittaakos\Desktop\theia\node_modules\fibers\build\fibers.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe` fail
ed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\kittaakos\Desktop\theia\node_modules\node-gyp\lib\build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Windows_NT 10.0.17763
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\kittaakos\\Desktop\\theia\\node_modules\\node-gyp\\bi
n\\node-gyp.js" "rebuild" "--release"
gyp ERR! cwd C:\Users\kittaakos\Desktop\theia\node_modules\fibers
gyp ERR! node -v v10.16.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
node-gyp exited with code: 1
Please make sure you are using a supported platform and node version. If you
would like to compile fibers on this machine please make sure you have setup your
build environment--
Windows + OS X instructions here: https://github.com/nodejs/node-gyp
Ubuntu users please run: `sudo apt-get install g++ build-essential`

Were there any native dependency udpates recently?

kittaakos · 2019-10-07T07:38:18Z

I cannot build it on my Windows image anymore.

I can build the from the master of https://github.com/eclipse-theia/theia. @westbury, can you please rebase this PR?

Signed-off-by: Nigel Westbury <nigelipse@miegel.org>

westbury · 2019-10-07T09:22:29Z

can you please rebase this PR?

Done, though I see nothing in the intervening commits that would have caused or fixed the build failure.

Were there any native dependency udpates recently?

Not since the previous base of this commit. For me I have never found the build environment as reliable on Windows as it is on Linux. It may be specifically my machine rather than Windows in general but I always just re-clone when I see something like this on Windows.

kittaakos · 2019-10-07T09:32:05Z

Done

Thanks!

For me I have never found the build environment as reliable on Windows as it is on Linux.

The environment is (quite) reliable on my Windows image. For instance, I had no issues building your other PR. I do not know 😊 Anyways, thank you!

kittaakos · 2019-10-07T11:14:10Z

I could build it this time, but it does not work for me. Or maybe I am missing a step or two. I did the followings:

Clone https://github.com/tom-shan/theia-plugin-hello-world.
Build (yarn) theia-plugin-hello-world.
Open Theia on theia-plugin-hello-world.
Hosted Plugin: Start Instance.
Nothing happens. What am I doing wrong?

root ERROR Uncaught Exception:  Error: Hosted plugin instance is not running.
root ERROR Error: Hosted plugin instance is not running.
    at ElectronNodeHostedPluginRunner.AbstractHostedInstanceManager.terminate (C:\Users\kittaakos\Desktop\theia\packages
\plugin-dev\lib\node\hosted-instance-manager.js:197:19)
    at Timeout._onTimeout (C:\Users\kittaakos\Desktop\theia\packages\plugin-dev\lib\node\hosted-instance-manager.js:377:
27)
    at ontimeout (timers.js:427:11)
    at tryOnTimeout (timers.js:289:5)
    at listOnTimeout (timers.js:252:5)
    at Timer.processTimers (timers.js:212:10)

kittaakos · 2019-10-07T11:23:26Z

it does not work for me.

I was using 0090cd2.

westbury · 2019-10-07T16:34:31Z

I would suggest opening a bugzilla about his: start here and select component: GitHub

@marcdumais-work Thanks for the advice. That's sorted it out. Apparently it was 'a recently discovered edge case in the sync tool'.

westbury · 2019-10-09T14:25:51Z

Nothing happens. What am I doing wrong?

You're using Electron :-) I had only tested the browser version. I see the following when I try it in Electron (and nothing happens in the UI):

root ERROR Error when loading drives. Error: Request 'getDrives' failed
at Proxy. (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/bundle.js:109432:33)
at FileDialogModel. (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1444:62)
at step (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1367:23)
at Object.next (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1348:53)
at file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1342:71
at new Promise ()
at push.../../packages/filesystem/lib/browser/file-tree/file-tree-model.js.__awaiter (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1338:12)
at FileDialogModel.push.../../packages/filesystem/lib/browser/file-tree/file-tree-model.js.FileTreeModel.drives (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:1438:16)
at LocationListRenderer.push.../../packages/filesystem/lib/browser/location/location-renderer.js.LocationListRenderer.doLoadDrives (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:2427:26)
at LocationListRenderer.push.../../packages/filesystem/lib/browser/location/location-renderer.js.LocationListRenderer.collectLocations (file:///C:/Users/nigwes01/git/theia/examples/electron/lib/3.bundle.js:2418:14)root ERROR Request getDrives failed with error: A dynamic link library (DLL) initialization routine failed.
\?\c:\Users\nigwes01\git\theia\node_modules\drivelist\build\Release\drivelist.node Error: A dynamic link library (DLL) initialization routine failed.
\?\c:\Users\nigwes01\git\theia\node_modules\drivelist\build\Release\drivelist.node
at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:166:20)
at Object.Module._extensions..node (internal/modules/cjs/loader.js:740:18)
at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:166:20)
at Module.load (internal/modules/cjs/loader.js:620:32)
at tryModuleLoad (internal/modules/cjs/loader.js:559:12)
at Function.Module._load (internal/modules/cjs/loader.js:551:3)
at Module.require (internal/modules/cjs/loader.js:658:17)
at require (internal/modules/cjs/helpers.js:20:18)
at bindings (c:\Users\nigwes01\git\theia\node_modules\bindings\bindings.js:81:44)
at Object.exports.list (c:\Users\nigwes01\git\theia\node_modules\drivelist\lib\drivelist.js:52:7)

westbury · 2019-10-11T08:54:54Z

I see the following when I try it in Electron
?\c:\Users\nigwes01\git\theia\node_modules\drivelist\build\Release\drivelist.node Error: A dynamic link library (DLL) initialization routine failed.

This problem is fixed by rebuilding drivelist natives (this may have been an issue on my machine only). However there is a larger problem in starting the Electron instance on Windows. This may take some time (for me at least) to determine the cause. So we can either 1) reduce the scope of this PR to the browser only and merge if it works on the browser, or 2) leave it hanging around until someone figures out the electron issue.

westbury · 2020-01-08T14:31:34Z

The fixes here are required to use self-hosting on Windows. So that browser users are not blocked, I have opened #6846 for the separate electron issue and am merging these fixes.

vince-fugnitto added plug-in system issues related to the plug-in system OS/Windows issues related to the Windows OS labels Oct 2, 2019

vince-fugnitto requested review from mmorhun and benoitf October 2, 2019 17:27

benoitf reviewed Oct 2, 2019

View reviewed changes

mmorhun approved these changes Oct 3, 2019

View reviewed changes

Fix self-hosting on Windows

0090cd2

Signed-off-by: Nigel Westbury <nigelipse@miegel.org>

westbury force-pushed the windows-fixes branch from f0f72f0 to 0090cd2 Compare October 7, 2019 09:14

westbury mentioned this pull request Jan 8, 2020

Self-hosting does not work on Windows in Electron mode #6846

Closed

westbury merged commit 62d13b4 into eclipse-theia:master Jan 8, 2020

westbury deleted the windows-fixes branch January 9, 2020 10:12

westbury mentioned this pull request Jan 9, 2020

Hosted Plugins on Windows not working #5488

Open

westbury mentioned this pull request Feb 4, 2020

Hosted Plugin: Start Instance command does not find plugin on Windows #2868

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix self-hosting on Windows #6316

Fix self-hosting on Windows #6316

westbury commented Oct 2, 2019

benoitf Oct 2, 2019

svenefftinge Oct 3, 2019

akosyakov Oct 7, 2019

westbury Oct 10, 2019 •

edited

Loading

mmorhun left a comment

westbury commented Oct 4, 2019

akosyakov commented Oct 4, 2019

marcdumais-work commented Oct 4, 2019

westbury commented Oct 4, 2019

marcdumais-work commented Oct 4, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

westbury commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

westbury commented Oct 7, 2019

westbury commented Oct 9, 2019

westbury commented Oct 11, 2019

westbury commented Jan 8, 2020

Fix self-hosting on Windows #6316

Fix self-hosting on Windows #6316

Conversation

westbury commented Oct 2, 2019

What it does

How to test

Review checklist

Reminder for reviewers

benoitf Oct 2, 2019

Choose a reason for hiding this comment

svenefftinge Oct 3, 2019

Choose a reason for hiding this comment

akosyakov Oct 7, 2019

Choose a reason for hiding this comment

westbury Oct 10, 2019 • edited Loading

Choose a reason for hiding this comment

mmorhun left a comment

Choose a reason for hiding this comment

westbury commented Oct 4, 2019

akosyakov commented Oct 4, 2019

marcdumais-work commented Oct 4, 2019

westbury commented Oct 4, 2019

marcdumais-work commented Oct 4, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

westbury commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

kittaakos commented Oct 7, 2019

westbury commented Oct 7, 2019

westbury commented Oct 9, 2019

westbury commented Oct 11, 2019

westbury commented Jan 8, 2020

westbury Oct 10, 2019 •

edited

Loading