Any plan to support nodejs?

[xiaohanyu]

Recent days, nodejs becomes more and more popular. And one of the major advantages of swank-js is that swank-js support nodejs natively.

So I wonder is there any future plan to support nodejs as a js backend in skewer-mode?

[skeeto]

Xiao Hanyu notifications@github.com writes:

Recent days, nodejs becomes more and more popular. And one of the major advantages of swank-js is that swank-js support nodejs natively.

So I wonder is there any future plan to support nodejs as a js backend in skewer-mode?

It's been a sort of vague plan that I haven't taken any action on yet
since I currently don't use node.js myself. To do it I would add
polymorphic support for different kinds of clients first, then implement
a node.js client backend to talk to Skewer as a new kind of client.

[itegebo]

I would add polymorphic support for different kinds of clients first

I'm not sure what you meant by this.

I've been noodling around with my fork of skewer-mode, and I see that beyond having different GET/POST methods, there's also the matter of node.js not having a DOM, and so the CSS and HTML "extensions" don't make any sense (nor do I want to shoe-horn it in by using something like jsdom). Also, the current method for loading a whole file via a script element also doesn't make sense. Lastly, my most recent discovery is that node.js's __dirname/__filename are only defined in some "script mode" I don't understand quite yet.

I think the current issues are easy enough to deal with - the clients can ignore request types it doesn't understand and file loading can be done in any number of other ways. I've got ideas for the magic script vars, but nothing definitive yet.

The bigger problem for me is how my use case is clearly going to be different than the current one(s). Frankly, it took me a while to even imagine why one would want to broadcast every evaluation request to multiple clients: I assume one is to work on a single project at a time with the desire to do interactive cross-browser development.

I'd like to do both server and client side development in Javascript - thus, broadcasting every request to all clients is going to be a problem.

Going about making the changes for my use case is straightforward; ensuring those changes are acceptable is less so. There are any number of ways to associate Emacs buffers with clients. Has there been any work/thought that I should take into consideration as I go forward?

[skeeto]

@itegebo These are good points.

I would add polymorphic support for different kinds of clients first
I'm not sure what you meant by this.

Ok, so right now a client is represented by a skewer-client defstruct.
This struct has two fields, the process and the user agent string. As
part of adding support for a new kind of client like node.js, I would
switch to EIEIO, with a different class (defclass) for each possible
kinds of client. There would be a browser client class, which behaves
like it does now. There would be a nodejs client class. They'll provide
the same methods so that other functions using the clients don't need to
pay attention to what kind of client it is. Polymorphism.

The most basic method would accept a string containing JavaScript and it
would return, synchronously or asynchronously, the result as a rich data
type (e.g. skewer-eval). How that string gets to the client depends on
the type of client.

I see that beyond having different GET/POST methods, there's also the
matter of node.js not having a DOM, and so the CSS and HTML
"extensions" don't make any sense (nor do I want to shoe-horn it in by
using something like jsdom).

Yup, I would imagine an entirely separate skewer.js script would be
created for supporting nodejs. There's no DOM to worry about and, more
importantly, it doesn't need to worry about supporting half a dozen
vendors with varying levels of compliance.

Frankly, it took me a while to even imagine why one would want to
broadcast every evaluation request to multiple clients:

Two reasons:

• It's makes testing Skewer a lot easier. When I've made major changes
  I'll open up several browsers on two operating systems (Windows and
  Linux) and connect them all to the same Emacs instance. Then I drive
  them all and ensure they'll all giving me the same behavior.
• It's really handy for multicore processing! I've run some Monte Carlo
  sims this way. Fire up one instance of Chrome for each core on each
  participating machine. Chrome limits itself to 6 open connections at
  a time (hardcoded?), so you'll need to tweak it on machines with more
  than 6 cores. Then I can fire off my Monte Carlo experiment and let
  the results from each machine pour into my buffer as they complete.

I'd like to do both server and client side development in Javascript -
thus, broadcasting every request to all clients is going to be a
problem.

Good point. Perhaps there needs to be some interface to associate
certain clients with certain buffers. Right now I can't imagine what
this would look like, though.

Going about making the changes for my use case is straightforward;
ensuring those changes are acceptable is less so. [...] Has there been
any work/thought that I should take into consideration as I go
forward?

The biggest issue with making changes right now is the difficulty in
testing. Any time a significant change happens to browser interaction
code I have to do a lot of manual testing. Some problems with automated
testing:

• ERT doesn't support asynchronous unit tests. Too much of Skewer
  relies on callbacks out of necessity.
• ERT can't observe changes in browsers. Not directly, anyway, so there
  would need to be JavaScript side support to report results back to
  ERT.
• I still need to fire up the 10 or so browsers and connect them all
  before automated testing can begin.

Otherwise I guess I'd say to keep the nodejs-specific functions apart
from the browser-specific functions. I'd say even put it in a different
file (skewer-nodejs.el?).

[itegebo]

Frankly, it took me a while to even imagine why one would want to
broadcast every evaluation request to multiple clients:

Two reasons:

• It's makes testing Skewer a lot easier. [...]
• It's really handy for multicore processing! [...]

Wow, I had not thought about those cases. Very cool.

Going about making the changes for my use case is straightforward;
ensuring those changes are acceptable is less so. [...] Has there been
any work/thought that I should take into consideration as I go
forward?

The biggest issue with making changes right now is the difficulty in
testing. Any time a significant change happens to browser interaction
code I have to do a lot of manual testing. Some problems with automated
testing:

• ERT doesn't support asynchronous unit tests. Too much of Skewer
  relies on callbacks out of necessity.

I was curious, so I went looking. Did you know about ert-async?

It's all of 12 days old, so perhaps not...maybe that's an option now.

• ERT can't observe changes in browsers. Not directly, anyway, so there
  would need to be JavaScript side support to report results back to
  ERT.
• I still need to fire up the 10 or so browsers and connect them all
  before automated testing can begin.

I wonder about integrating Emacs testing with some combination of PhantomJS
and Browserling/Browserstack. I expect that's quite a bit of work though.
Do you have any experience with SaaS browser testing? I don't.

Otherwise I guess I'd say to keep the nodejs-specific functions apart
from the browser-specific functions. I'd say even put it in a different
file (skewer-nodejs.el?).

The immediate problem with this approach is the event-related code at the
end of skewer.js. You can see that I've hacked around this in my nodejs
branch. I'm going to defer making any pull requests until I've implemented
what I need. Once I know what was required, I'll look at how to best
integrate that with the existing use cases and extensions, e.g. ac-js2,
skewer-less. BTW, do you know of any other extension/plugins like
skewer-less?

[shakthimaan]

I am also interested in this enhancement. Given a project with package.json, I would like to be able to use skewer-mode to use Node.js as backend for development and testing.

[InvisibleTech]

Not super deep on spacemacs configuration and development... more interested in the minimalist approach to getting nodejs repl to run my buffers. I really don't even need skewer for a lot of the backend work. So, I tried different approaches to getting this https://marmalade-repo.org/packages/nodejs-repl emacs feature to work in spacemacs and the misery ran from trying the most evil thing of having require in the user-config function. Rejected at the gate. To trying to just curl the elisp file above into a similarly named directory and referencing nodejs-repl from the dotspacemacs-configuration-layers which is also rejected as an unknown package.

So, without further hack and guess, what is the most minimal way to get a package in place to simply allow me to run buffers under the node repl - not caring about pushing to skewer browser window? For now I'll edit in spacemacs and repl in terminal. suboptimal but given my time spent to find out this isn't available and trying to hack around it... I am back to doing my original task as the integration of repl work is getting in the way.

Thanks for any pointers as I am emacs-ignorant and a spacemacs consumer at the beginner level.