Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
"...Let me turn on the router. 2067 commits need pushing to master. T…
…his latest commit will have 323 changes. 2) Bundling. Making sure the complation output goes into the right place where the plugin is. Making the pipeline to easily publish the plugin. Making a check so I do not send a debug version by accident. 3) Implementation. I need the plugin to start the server automatically. I also need to implement heartbeating on both sides. While I wait for the internet to come back, let me think about the next part. Right now I am done with the cleanup. In the Build tab in the VS IDE I see `Pack` and `Publish` options that I've never used before. 1:45pm. I've closed the one outstanding issue on the Spiral repo from early 2018. With this I can consider the repo itself to be organized. What I need right now is to organize the compilation. Cleaning up the repo was easy, but this one will require some research. I actually do not know how to do most of the things I want here. Let me take a look at the VS Code extension help page. I should take a look at how the plugins are actually published. Right now I only know how to run them from the extension host. https://code.visualstudio.com/api/working-with-extensions/publishing-extension > Visual Studio Code leverages Azure DevOps for its Marketplace services. This means that authentication, hosting, and management of extensions are provided through Azure DevOps. This Azure thing again...back when I was studying webdev in the earlier part of the year I played with that. > If you add a repository field to your package.json and it is a public GitHub repository, vsce will automatically detect it and adjust relative links accordingly, using the master branch by default. Let me do that. ```fs "name": "spiral-lang", "version": "0.2.0", "description": "VS Code editor support plugin + the Spiral language compiler.", "repository": {"url": "https://github.com/mrakgr/The-Spiral-Language"}, ``` I'll do it like this... 2:10pm. Mhhhh... Ok, let me get that personal access token. Then I'll try publishing Spiral in a broken form. Let me make it my goal to just see the thing show up in the marketplace. I'll unpublish it and figure out how to do versioning after that. ``` C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>vsce package This extension consists of 1309 files, out of which 1057 are JavaScript files. For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore DONE Packaged: C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin\spiral-lang-vscode-0.2.0.vsix (1309 files, 5.71MB) ``` Hmmm, does it not ignore the node_modules stuff by default? But no, I should in fact be bundling the js files. No way around that. The plugin user should not be compiling those. ``` C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>vsce create-publisher mrakgr ERROR The 'create-publisher' command is no longer available. You can create a publisher directly in the Marketplace: https://aka.ms/vscode-create-publisher ``` The docs are out of date. But when I tried the vsce publish, it asked me for the access token right away. Let me go with that. ``` C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>vsce publish This extension consists of 1309 files, out of which 1057 are JavaScript files. For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore Personal Access Token for publisher 'mrakgr': **************************************************** ERROR Access Denied: Microsoft.IdentityModel.Claims.ClaimsIdentity;00030000D1AC6AE8@Live.com needs the following permission(s) on the resource /mrakgr to perform this action: View user permissions on a resource ``` I guess the access token is not good. https://stackoverflow.com/questions/58118616/how-to-fix-access-denied-to-perform-this-action-view-user-permissions-on-a-re 2:35pm. I have no idea what the problem is. I keep getting the 401 error now. This means I am unauthorized. 2:40pm. https://marketplace.visualstudio.com/manage/publishers/mrakgr I tried uploading through the web interface. This seems to work. I have no idea what I am doing wrong that the access token is not being accepted. 2:45pm. Ok, I can see it show up in the VS Code marketplace now. Unlike what I expected it is not showing me the front page of the repo. I also managed to login with a full access token. Nevermind that for now. Ok, I am satisfied with this for now. Let me remove it from the marketplace. 2:50pm. ``` "version": "0.2.0", ``` This is not good. I am not making efficient use of versioning fields. I'll make Spiral v0.2, straight up Spiral 2. ``` "version": "2.0.0", ``` Then I will just increment the last field for patches, and the second field for minor version changes. https://code.visualstudio.com/api/working-with-extensions/publishing-extension#autoincrementing-the-extension-version It shows how to auto increment the extension version here. This is fine. 2:55pm. My mind is scattered. What should come next? I know how to publish plugins. What I really need to do next is figure out how to organize the compilation. How do I direct the output of the F# compiler? https://fake.build/legacy-gettingstarted.html Do I need to learn how to use this? Can't I do this from VS? https://docs.microsoft.com/en-us/visualstudio/ide/compiling-and-building-in-visual-studio?view=vs-2019 ``` Build started... 1>------ Build started: Project: The Spiral Language v0.2, Configuration: Release Any CPU ------ 1>The Spiral Language v0.2 -> C:\Users\Marko\Source\Repos\The Spiral Language\The Spiral Language v0.2\bin\Release\netcoreapp3.1\Spiral.dll 2>------ Publish started: Project: The Spiral Language v0.2, Configuration: Release Any CPU ------ 2>The Spiral Language v0.2 -> C:\Users\Marko\Source\Repos\The Spiral Language\The Spiral Language v0.2\bin\Release\netcoreapp3.1\Spiral.dll 2>The Spiral Language v0.2 -> C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin\compiler\ ========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ========== ========== Publish: 1 succeeded, 0 failed, 0 skipped ========== ``` Oh, I can just publish it into a folder from the IDE. Is it there? Yes. Earlier, I also saw that in the Project Build tab, there is the output folder, but I should not be using that for this. 3:15pm. Ok, I think I get it. From the IDE all I really need is to publish the compiler into the plugin folder. I do not need anything else right now. Now...let me see if I can deal with bundling the compiler plugin. I want to ommit the node module stuff from the end result and inline it all into a single file. Let me go back to vsce. https://code.visualstudio.com/api/working-with-extensions/bundling-extension Adding stuff to `.vscodeignore` works. Now I just need to bundle that `.js` file. 3:20pm. Let me follow the webpack instructions there. Let me do this thing. ``` > webpack --mode development (node:13764) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time [webpack-cli] Compilation finished asset index.js 4.34 KiB [emitted] (name: main) 1 related asset ./src/index.ts 39 bytes [not cacheable] [built] [code generated] [1 error] ERROR in ./src/index.ts Module build failed (from ./node_modules/ts-loader/index.js): Error: Cannot find module 'typescript' Require stack: - C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin\node_modules\ts-loader\dist\compilerSetup.js ``` What does this mean? https://stackoverflow.com/questions/44611526/how-to-fix-cannot-find-module-typescript-in-angular-4 I guess I'll install ts globally then. 3:35pm. No, I get the same error. https://medium.com/jspoint/integrating-typescript-with-webpack-4534e840a02b > So the question is, why would you ever need Webpack if TypeScript compiler (TSC) can transpile TypeScript to JavaScript. Well, in most cases, you don’t. But the TSC isn’t designed to do everything for you. It can compile TypeScript to JavaScript, produce declaration files, produce source maps, and even produce a bundle file using outFile compile-option. Hmmm, it can? > But the outFile option won’t work in all the scenarios. If the output module system is set to ES6, CommonJS or anything other than None, System or AMD, TSC won’t be able to produce a bundle file. I have already explained this Compilation lesson (under the outFile section). Let me try this last thing. Seriuosly, screw Webpack. I absolutely hate it when I have to crawl through config files and read the tea leaves for what some weird compilation error means. It can't find the typescript module? Just what the hell is it expecting me to do here - I am not psychic. 3:45pm. No it does not work. If I try to use `commonjs` as the module system, it does not support it. And anything other than it gives me an error on import statements. ``` $ npm init -y $ npm install -D webpack webpack-cli typescript ts-loader ``` This is how it is installing it here. Let me try installing TS as a dev dependency. ``` C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>npm i -D typescript npm WARN spiral-lang-vscode@2.0.0 No license field. + typescript@4.1.3 added 1 package from 1 contributor and audited 173 packages in 2.092s 17 packages are looking for funding run `npm fund` for details found 0 vulnerabilities C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>npm run webpack > spiral-lang-vscode@2.0.0 webpack C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin > webpack --mode development (node:4156) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time [webpack-cli] Compilation finished asset index.js 84.1 KiB [emitted] (name: main) 1 related asset cacheable modules 80.2 KiB modules by path ./node_modules/zeromq/lib/*.js 27.2 KiB ./node_modules/zeromq/lib/index.js 25 KiB [built] [code generated] ./node_modules/zeromq/lib/native.js 507 bytes [built] [code generated] ./node_modules/zeromq/lib/draft.js 1.76 KiB [built] [code generated] ./src/index.ts 48.5 KiB [built] [code generated] ./node_modules/node-gyp-build/index.js 4.5 KiB [built] [code generated] external "path" 42 bytes [built] [code generated] external "vscode" 42 bytes [built] [code generated] external "events" 42 bytes [built] [code generated] external "fs" 42 bytes [built] [code generated] external "os" 42 bytes [built] [code generated] webpack 5.11.0 compiled successfully in 2135 ms ``` Oh, now it builds. Success. 3:55pm. ``` C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin>vsce package Executing prepublish script 'npm run vscode:prepublish'... > spiral-lang-vscode@2.0.0 vscode:prepublish C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin > webpack --mode production (node:20216) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time (node:20216) ExperimentalWarning: Conditional exports is an experimental feature. This feature could change at any time [webpack-cli] Compilation finished asset index.js 16.6 KiB [compared for emit] [minimized] (name: main) 1 related asset cacheable modules 80.2 KiB modules by path ./node_modules/zeromq/lib/*.js 27.2 KiB ./node_modules/zeromq/lib/index.js 25 KiB [built] [code generated] ./node_modules/zeromq/lib/native.js 507 bytes [built] [code generated] ./node_modules/zeromq/lib/draft.js 1.76 KiB [built] [code generated] ./src/index.ts 48.5 KiB [built] [code generated] ./node_modules/node-gyp-build/index.js 4.5 KiB [built] [code generated] external "path" 42 bytes [built] [code generated] external "vscode" 42 bytes [built] [code generated] external "events" 42 bytes [built] [code generated] external "fs" 42 bytes [built] [code generated] external "os" 42 bytes [built] [code generated] webpack 5.11.0 compiled successfully in 2380 ms DONE Packaged: C:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin\spiral-lang-vscode-2.0.0.vsix (221 files, 4.26MB) ``` Ah, it will automatically execute the prepublish script. I see. That will save me some error. 4pm. Things are coming together. I've orginazed the compilation enough. 3) Implementation. I need the plugin to start the server automatically. I also need to implement heartbeating on both sides. Now it is time to do this thing. I'll leave heartbeating aside for now. Let me focus on just starting the server. I have it right where I want it. I just need to figure out how to start the process. My mind is calling up the basic vs code sample. 4:10pm. Did a PR suggesting to install TS as a dev dependency. Let me find that process start example. https://nodejs.org/api/child_process.html Forget the VS Code samples. Those would take too long. I'll rely on the node docs instead to guide me. https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options Isn't there something easier than this? Also this does IO routing. I am not sure I want that. > On Windows, setting options.detached to true makes it possible for the child process to continue running after the parent exits. The child will have its own console window. Once enabled for a child process, it cannot be disabled. Ah, I see. I'll have to give this a try. It turns out that I did not strictly need ZeroMQ. There is some possibility for communication over pipes, but nevermind that. Forget everything else - let me try the process spawn. ...Actually rather than this, there might be something in the VS Code API that will let me spawn processes. Let me check that out. I see now that I would need to import node if I wanted to use the `child_process` functions. ```ts export class ProcessExecution { /** * Creates a process execution. * * @param process The process to start. * @param options Optional options for the started process. */ constructor(process: string, options?: ProcessExecutionOptions); /** * Creates a process execution. * * @param process The process to start. * @param args Arguments to be passed to the process. * @param options Optional options for the started process. */ constructor(process: string, args: string[], options?: ProcessExecutionOptions); /** * The process to be executed. */ process: string; /** * The arguments passed to the process. Defaults to an empty array. */ args: string[]; /** * The process options used when the process is executed. * Defaults to undefined. */ options?: ProcessExecutionOptions; } ``` There is definitely something there as expected. ```ts export class Task { /** * Creates a new task. * * @param definition The task definition as defined in the taskDefinitions extension point. * @param scope Specifies the task's scope. It is either a global or a workspace task or a task for a specific workspace folder. Global tasks are currently not supported. * @param name The task's name. Is presented in the user interface. * @param source The task's source (e.g. 'gulp', 'npm', ...). Is presented in the user interface. * @param execution The process or shell execution. * @param problemMatchers the names of problem matchers to use, like '$tsc' * or '$eslint'. Problem matchers can be contributed by an extension using * the `problemMatchers` extension point. */ constructor(taskDefinition: TaskDefinition, scope: WorkspaceFolder | TaskScope.Global | TaskScope.Workspace, name: string, source: string, execution?: ProcessExecution | ShellExecution | CustomExecution, problemMatchers?: string | string[]); ``` This is where it is used. ...This is too complicated. Let me import node. ``` "@types/node": "^13.13.5", ``` Wait, I already have this as a dev dependency. ``` import {} from "node" ``` This is giving me an error shit. ``` File 'c:/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/node_modules/@types/node/index.d.ts' is not a module. ``` This is so annoying. What do I do about this? Can I remove this dev dependency? https://stackoverflow.com/questions/44416902/how-to-fix-types-node-index-d-ts-is-not-a-module ``` import * as cp from "child_process" ``` 5pm. Had to stop for lunch. At any rate, I did what that SO answer suggested and now I can access the node functions. How do I start the cp as a separate one? https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options ...It has a ton of overloads that slipped by me. > windowsHide <boolean> Hide the subprocess console window that would normally be created on Windows systems. Default: false. Ah, this option is here as well. Interesting. This will be useful. Ok, now let me give it a try. By it I mean... ```ts export const activate = async (ctx: ExtensionContext) => { console.log("Spiral plugin is active.") const server_process = cp.spawn("../compiler/Spiral.exe", {detached: true}) ``` Hmmm, nothing is happening. Let me put in a direct path just as a test. ```ts const server_process = cp.spawn("C:/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler/Spiral.exe", {detached: true}) ``` This works. Unfortunately, it does not show me the console window. Agh. I had to close the process using the task manager. What do I do about this? ```ts console.log(__dirname); ``` Just what is the current directory when the plugin is run? ``` Spiral plugin is active. index.js:75 c:\Users\Marko\Source\Repos\The Spiral Language\VS Code Plugin\out ``` Hmmm, it is what I thought it is. ```ts console.log(path.join(__dirname,"../compiler/Spiral.exe")); ``` This gives me the string that I want. Let me try spawning it like this. But before that, I really want the console window to show up. ```ts const server_process = cp.spawn(path.join(__dirname,"../compiler/Spiral.exe"), {detached: true, windowsHide: false}) ``` Let me try this. ```ts try { cp.spawn(path.join(__dirname,"../compiler/Spiral.exe"), {detached: true, windowsHide: false}) } catch {} ``` This is the worst - trying to start a new process while the old one is still running fails. That idea where I thought the exception would act as an auto abort fails. The try catch here does not work. https://stackoverflow.com/questions/56535734/nodejs-child-process-spawned-a-subprocess-the-subprocess-not-show-console-windo This says it requires both shell and detached. ...Shit, not only does this fail, Spiral is no longer showing in the task manager, so I can't close it. Let me restart. 5:35pm. I was wrong. It seems the stuff in the answer just does not work. ```ts cp.spawn("node", [], {shell: true, detached: true}) ``` This works, but it not what I want. ```ts const p = cp.spawn(path.join(__dirname,"../compiler/Spiral.exe"), {detached: true}) p.stdout.on("data", (data) => { console.log(data); }) p.on('close', function (code) { console.log('child process exited with code ' + code); }); ``` Let me do this. 5:55pm. ```ts const p = cp.spawn(path.join(__dirname,"../compiler/Spiral.exe"), {detached: true}) p.stderr.on("data", (data) => { console.log(data.toString()); }) p.stdout.on("data", (data) => { console.log(data.toString()); }) p.on('close', function (code) { console.log('child process exited with code ' + code); }); ``` To make matters even worse, the server will randomly crash in the background when I try to run this again. Node's process starting is such a turd. https://zaiste.net/posts/nodejs-child-process-spawn-exec-fork-async-await/ > spawn doesn't create a shell to execute the command while exec does create a shell. Thus, it's possible to specify the command to execute using the shell syntax. exec also buffers the command's entire output instead of using a stream. Let me try exec. ``` Server bound to: tcp://*:13805 Unhandled exception. System.ArgumentException: An item with the same key has already been added. Key: 0 at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior) at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value) at Spiral.Supervisor.__@449-4.Invoke(NetMQSocketEventArgs s) in C:\Users\Marko\Source\Repos\The Spiral Language\The Spiral Language 2\Supervisor.fs:line 485 at Microsoft.FSharp.Control.CommonExtensions.SubscribeToObservable@1734.System.IObserver<'T>.OnNext(T args) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\async.fs:line 1735 at Microsoft.FSharp.Core.CompilerServices.RuntimeHelpers.h@345.Invoke(Object _arg1, TArgs args) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 345 at Spiral.Supervisor.__@449-3.Invoke(Object delegateArg0, NetMQSocketEventArgs delegateArg1) in C:\Users\Marko\Source\Repos\The Spiral Language\The Spiral Language 2\Supervisor.fs:line 449 at NetMQ.NetMQSocket.InvokeEvents(Object sender, PollEvents events) at NetMQ.NetMQPoller.RunPoller() at NetMQ.NetMQPoller.Run(SynchronizationContext syncContext) at NetMQ.NetMQPoller.Run() at Spiral.Supervisor.main(String[] _arg1) in C:\Users\Marko\Source\Repos\The Spiral Language\The Spiral Language 2\Supervisor.fs:line 498 ``` I said the server should be reusable from multiple instances, but that clearly has some debugging left to be done. ```fs if !last_id = id then body msg (address, x) else buffer.Add(id,(address,x)) ``` Ah, it is the message ordering that is making trouble. Damn. I am going to have to remove this. And I'll have to make sure that trying to index the semantic tokens out of range does not throw an exception. 6:05pm. Nevermind this. Right now my biggest problem is that I cannot start the server in a shell. And for some reason, trying to start the same process twice is broken even though it should not influence anything. I think the server being broken the second time it is started is an illusion. Yeah, when I start it myself by beforehand, and then have the plugin try to start one as well, the error does not occur. "C:/Users/Marko/Source/Repos/ConsoleApp6/ConsoleApp6/bin/Debug/netcoreapp3.1/ConsoleApp6.exe" Let me try running something random in the shell instead of the compiler. 6:15pm. ```ts const qwe = path.join(__dirname,"../compiler/Spiral.exe") const asd = "C:/Users/Marko/Source/Repos/ConsoleApp6/ConsoleApp6/bin/Debug/netcoreapp3.1/ConsoleApp6.exe" const zxc = "C:/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler/Spiral.exe" const p = cp.spawn(zxc,{shell: true, detached: true}) ``` What stupid and useless waste of time this is. I have no idea why the Spiral refuses to launch as a shell. 6:20pm. That random app I tried on the side does work. Just the server fails for some reason. I have no idea. I am going to try removing the main and just trying something basic. Maybe NetMQ is getting in the way or some thing else is. Right now I am frustrated because Node being crappy like this is really getting in the way of my debugging. I expected to be done with it all by now, but instead I've barely even started because I am stuck on trying to launch the server properly. 6:20pm. Calm down me. Let me put in some overtime. Let me see if I can launch the hello world when it is in the Spiral folder. ```ts const asd = "C:/Users/Marko/Source/Repos/ConsoleApp6/ConsoleApp6/bin/Debug/netcoreapp3.1/ConsoleApp6.exe" const zxc = "C:/Users/Marko/Source/Repos/The Spiral Language/VS Code Plugin/compiler/Spiral.exe" const hello = "C:/Users/Marko/Source/Repos/The Spiral Language/The Spiral Language 2/bin/Debug/netcoreapp3.1/Spiral.exe" const p = cp.spawn(hello,{shell: true, detached: true}) ``` Right now the compiler only prints hello and waits for enter to be pressed, and yet the process does not show up. If I compare the paths, I see that hello has spaces... ```ts const hello = '"C:/Users/Marko/Source/Repos/The Spiral Language/The Spiral Language 2/bin/Debug/netcoreapp3.1/Spiral.exe"' const p = cp.spawn(hello,{shell: true, detached: true}) ``` Now it works. ```ts export const activate = async (ctx: ExtensionContext) => { console.log("Spiral plugin is active.") const p = cp.spawn('"' + path.join(__dirname,"../compiler/Spiral.exe") + '"',{shell: true, detached: true}) ``` Now it works. I finally crossed this hurdle. It is really hard to fix an error when you have zero feedback on what is going on. 6:35pm. Let me close here. Despite the unexpected hardship, I am not that far from dealing with things. Tomorrow, I am going to remove the message ordering machinery and focus instead on improving the roboustness of the server. That will be easy. Then I will implement heartbeating. And then this will be done. I'll publish the language on the VS Code marketplace and move on to doing the docs, debugging as I go. When I feel like I have enough, I'll move to sponsor-seeking. Once I snag the first fish, I will get back to work on RL agents. That is the way things should go. V0.09 went organically and v2 is going to be the same way. Doing random things for the sake of improving the roboustness of the language is just stupid. I should at least try to get other people to pay for this effort. Just getting that first monthly stipend will means I can upgrade my rig. Hopefully by the time this happens, the supply glut in the latest PC components will have abated. I used to have an AMD CPU over a decade ago, back when it had a lead on Intel and it seems that will be my aim again."
- Loading branch information
