Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Silk GUI #79

Open
zodern opened this issue Nov 4, 2014 · 215 comments
Open

Silk GUI #79

zodern opened this issue Nov 4, 2014 · 215 comments
Milestone

Comments

@zodern
Copy link
Contributor

zodern commented Nov 4, 2014

I have started making a GUI in javascript named Silk. You can find the github repository here.

4 main goals:

  • Fast and Lightweight
  • Designed with developers in mind
  • Experiment with new ideas for GUI's
  • Client (window manager) can be accessed remotely

What is Completed

I duplicated Meteor.methods since they make it very for the client and server to communicate.

The basics of window management are done. It is possible to open and minimize windows. It applies the correct z-index to windows.

It is possible to remotely access Silk by going to the url shown in the terminal.

Plans

The first version will have a very simple text editor and file explorer.

There is no security to prevent anyone from remotely accessing the computer, which needs to be fixed. We could look at logon.

Right now the window manager uses iframes to show the apps. Some websites don't allow themselves to display in iframes, including google.com, jw.org and github.com. If the expo browser is used in node os, maybe the window manager could create exoframes (I think that is what they are called) for each app.

To try it, past these commands into your terminal.

npm i -g silk-gui
cd silk
npm start

Screen Shots

Loading Screen

screen shot 2014-11-04 at 11 47 39 am

Desktop

screen shot 2014-11-04 at 11 46 36 am

Taskbar Hover

Notice that the taskbar background becomes solid on hover.

screen shot 2014-11-04 at 11 47 55 am

screen shot 2014-11-04 at 11 48 30 am

Current State of File Explorer

screen shot 2014-11-04 at 11 48 16 am

If you have a suggestion or idea, feel free to create an issue

@derhuerst
Copy link

This is pretty similar to #78.

I wrote:

NodeOS will be great as an efficient and hacking-friendly OS providing the absolute basics. Why does everyone (including @piranna) want to build ontop of web technologies? NodeOS is just supposed to be more than a web-based operating system, it is supposed to become a general-purpose node.js-only OS, right?

I think this is a general problems the node.js world has. While other environments are better at performance, node.js is an environment enabling incredible productivity when hacking all kind of things together. This potential (being cross-platform, high-level and with a vibrant and healthy package ecosystem) is too underrated; node.js isn't just the web anymore.

  • Web technologies grew to what they are today, but in my opinion, they are one of the worst things, you can use on OS-level for graphcis! They are slow, badly designed, insecure and way too complicated to use them at this low level. That's why I am against a web-based authentication system.

@zodern
Copy link
Contributor Author

zodern commented Nov 4, 2014

You have some good points.

I am using web technologies for several reasons:

  • It is easy to allow people to remotely connect to node os
  • It is what I am most experienced with
  • There are plans to add an html rendering engine in node os, and Silk will be able to use that.

Could you give me some specific examples on how they would be insecure?

@derhuerst
Copy link

It is easy to allow people to remotely connect to node os

I agree. I'm not sure, but i think Linux developers solved this problem with the display cllient server architecture, allowing a client from anywhere to connect to the display server and graphically control the machine. Unfortunately, this technique is quite unpopular in comparison to vnc-like products. If NodeOS supported a simeple display client server architecture, we could leave out the whole bloated and – this is the most important point – unsuitable web stack.
I understand that you guys want a simple way to control the machine graphically, but in my opinion, that is not enough to justify the usage of the web stack.

It is what I am most experienced with

Me too. But this has no influence on what I think is the best for a lightweight, server-like operating system.

There are plans to add an HTML rendering engine in node os, and Silk will be able to use that.

Okay, so I should give my opinion elsewhere then.

Could you give me some specific examples on how they would be insecure?

I wanted to mention security in combination with the bad design of the web stack. To clarify:
Since the web stack has been created (as simple linked HTTP files), it is always becoming more powerful. It gets several new features every month (!), influenced by the so-called web community, mainly developers from internet companies like Mozilla or Google. As with any new feature in software (of any kind), there are security issues getting fixed by time. The same happened to the web stack.

The web grew out of nothing else but plain HTML to full-fledged, desktop-like webapps, capable of things websites were never supposed to capable of. There has never been a point where a companies took all those lessons learned and designed a completely new stack, carefully shaped and a solid foundation for webapps.

Because that never happened, we have powerful webapps, which do still have to compromise on specific things. Developers still have to deal with weird things, such as the same origin policy, never-fixed bugs in browsers, the box model, the DOM, etc. Additonally, HTTP being stateless, the slow and bloated combination of JavaScript and the DOM, caching problems and all kinds of fallbacks and plugin/codec issues makes our life harder than necessary.

@derhuerst
Copy link

I wrote further thoughts about this at #68.

@Xarcell
Copy link

Xarcell commented Nov 4, 2014

I for one am in favor of a web stack. I hope to see NodeOS work as server, desktop, laptop, tablet, and smartphone. Wouldn't HTML5/CSS/Javascript be the best GUI for this?

@derhuerst
Copy link

I hope to see NodeOS work as server, desktop, laptop, tablet, and smartphone.

I don't. :D

I think that there should be a clear separation of concerns, implemented by a strict client server architecture.

Wouldn't HTML5/CSS/Javascript be the best GUI for this?

It depends.

If you don't have much time or resources, then the web stack offers an efficient way to support all devices. But to me, no webapp matches the performance, memory efficiency, beauty and user experience of a native app.

I propose to design a general (not web-like) standard and develop several clients. Maybe a web client is a good thing too, but I wouldn't design my protocol to focus on the web stack.

@davidyxu
Copy link

davidyxu commented Nov 5, 2014

I agree with @derhuerst that a common standard for a GUI clients to interface with NodeOS is an important first step. I wouldn't be surprised if web clients ended up being the norm due to the convenience factor, but NodeOS shouldn't be tied to the web stack specifically. It would be great to see multiple GUIs for NodeOS designed with different purposes in mind, used interchangeably or even in conjunction.

@derhuerst
Copy link

@davidyxu

Good to hear that my remarks are understandable.

I would go even further: Create a convenient CLI API for NodeOS and everyone will be happy.

  • A good CLI API is – and this has been proved many times – way more productive than a GUI.
  • Any GUI client can still translate GUI actions into CLI commands, making both CLI and GUI possible.

@piranna
Copy link
Member

piranna commented Nov 6, 2014

I think that there should be a clear separation of concerns, implemented by a strict client server architecture.

Me too, it's the best way to have a minimal core and being easily extensible. This way the client using a web stack or a native one is an implementation detail and can there be several of them for a single server, another discussion is if we'll adopt a web based one for an hypotetical NodeOS GUI, I'm for it but being isolated the server from their clients and all of them as independent projects, any one can use what he wants :-)

@serapath
Copy link

serapath commented Nov 7, 2014

maybe var nodeOS = require('nodeOS-cli'); in order to be able to use the exact same interface that is available through the command line, when translating "GUI actions" into commands as suggested by @derhuerst

@formula1
Copy link
Contributor

formula1 commented Nov 9, 2014

One of the biggest problems with the GUI when it comes to web technologies is the need to render the HTML/CSS as well as ensure that node modules and/or web javascript libraries are available

https://github.com/rogerwang/node-webkit

This project is extremely relevent

When it comes to implementation of a file explorer or methods available, the method is a bit irrelevent. Web programmers are a dime a dozen, I would know. And the best libraries will likely rise to the top.


I do want to point out though, most anything can be done through "child_process"

@piranna
Copy link
Member

piranna commented Nov 9, 2014

node-webkit is awesome, the problem is that applications are webpages with logic written in Node.js, not browser javascript. This is not portable, and more specially, this is Node.js-centric, so you lost the "by free" remote execution. AppKit has a better dessign since it creates a HTTP server with an API and makes clear the separation of interface (client-side) and logic (server-side) code, but seems it's abandoned and it's still a single executable.

The best is to have two projects per program, one a server for the logic written in Node.js, and one or more of them in html and browser side javascript (or other technologies) that communicate over a JSON-RPC protocol via HTTP requests or WebSockets, for example with my library rpc-builder, that's designed with a P2P architecture although JSON-RPC 2.0 is client-server, so events and requests can be send in both directions, and if you want to exec that graphic apps "locally", use a web renderer like Chrome Embebed Framework that's the basis of node-webkit, or said other way, it's node-webkit without the node part :-) It's only a question of porting it to EGL or DirectFB so it can run directly on a Linux terminal without needing X11.

Some could argue about the "excesive" usage of web technologies (http, websockets, json-rpc...) of this architecture I propose, but I think it's an intelligent and adeduate way to use them, and probably the easier, more elegant and most portable way of doing the things in this context, and most important all the client-server applications that currently are available for Node.js will run as native ones on NodeOS without needing to change their code at all.

@piranna
Copy link
Member

piranna commented Nov 9, 2014

An example of how applications should be done for NodeOS:

https://github.com/benkaiser/node-music-player

A webserver that read the music files stored on the hard disk that stream the audio data to a webpage running in a browser. I want this f_cking awesome app running on NodeOS like h_ll }:-)

@sidorares
Copy link

You can still run node-webkit app from remote box by forwarding X11 session to your local server, so your argument is similar to "wayland is worse than X because it does not allow remote execution" (solution: use VNC) But in general I don't think having webkit as requirement for all applications is a good idea

@formula1
Copy link
Contributor

formula1 commented Nov 9, 2014

I believe we're on the same page

Node Webkit is doing something superflueous. Attempting to integrate node heavilly in the browser is not really exactly necessary though it would perform better since the messages wouldn't need to be sent and parsed whether it be post/get/websocket messages

That music player is damn sexy

@piranna
Copy link
Member

piranna commented Nov 9, 2014

NodeOS is minimalistic with no extra layers that bloat the system, so no
X11 here, specially for only running a web renderer. You want node-webkit,
then port webkit to EGL, but by having webkit available, who want to use
node-webkit? :*)

Sorry, no X11 or Wayland here, and if asked, I would say no VNC too at
least as a general solution (if you can be able to render the client in
your web browser, why you would want to do it remotely?).
El 09/11/2014 10:41, "Andrey Sidorov" notifications@github.com escribió:

You can still run node-webkit app from remote box by forwarding X11
session to your local server, so your argument is similar to "wayland is
worse than X because it does not allow remote execution" (solution: use
VNC) But in general I don't think having webkit as requirement for all
applications is a good idea


Reply to this email directly or view it on GitHub
#79 (comment).

@sidorares
Copy link

You are describing ChromeOS :)

@piranna
Copy link
Member

piranna commented Nov 9, 2014

You are describing ChromeOS :)

Yes, somewhat :-) The main diference is that ChomeOS offer access to low
level OS internals using custom Javascript objects that are only available
to ChromeOS making ChromeOS apps not portables to other
platforms/browsers/OSes, that is somewhat similar to what you could get by
using node-webkit. FirefoxOS has a similar aproach, but at least they are
proposing their own FirefoxOS-only APIs as W3C standards, that's an
improvement over ChromeOS way, but in any case they are focused on
browser-based apps, not server-based ones.

What I'm proposing instead is, at least on a first stage, to don't offer
any mechanism to access to those low level things to GUI apps, so this
access is done by Node.js apps thenselves, and later they use their own
mechanism and protocol to interconnect with their GUI written in html
running on a web browser or what they want, so that GUI can run locally or
on another machine. Later if people demand it we could think about define
an homogeneos protocol or transport, but that's a huge task that don't make
sense to think about at this moment. I'm open to retake it later, but not
before the alpha has been released.

@ghost
Copy link

ghost commented Nov 9, 2014

@piranna and how about atom-shell or thrust? I would definitely use a NodeOS-like OS as a desktop OS.

@piranna
Copy link
Member

piranna commented Nov 9, 2014

@piranna and how about atom-shell or thrust?

Didn't know of thrust, seems interesting. As they say, both node-webkit and
atom-shell embed a Node.js runtime and you program your app in Node.js
based Javascript, so they are out of place if you are interested on making
your app accesible remotely. Thust has an architecture more similar to what
I was saying, but you are still managing the Chrome-based windows from
Node.js in a similar way to what you would be doing with Selenium, so it
could still be a little bit bloated for NodeOS. It seems a good aproach to
how to write apps, but it's still focused on desktops environment, while I
believe a NodeOS GUI would be better by using an architecture more similar
to ChromeOS or FirefoxOS (it's said, standard browser webapps) but accesing
to the Node.js priviledged APIs by using a REST API to a localhost or
remote server.

I'll try to write an entry in the wiki since it seems there's a lot of
interest on this topic.

I would definitelly use a NodeOS-like OS as a desktop OS.

Well, definitely it would be a cool thing to hack and tinker and as
development platform... :-)

@piranna
Copy link
Member

piranna commented Nov 10, 2014

I've created a page on the wiki showing my ideas and some of the conclusions talked here about the GUI issue, please comment your points about it:

https://github.com/NodeOS/NodeOS/wiki/GUI

@ghost
Copy link

ghost commented Nov 10, 2014

Is there really a difference between the web renderer and the canvas? I mean the web renderer should be able to render a canvas element, so it's basically two birds with one stone.
The only thing I can think of when the native canvas would be better solution where speed is essential and memory usage should be minimal (you don't have to spin up a new web renderer to draw something).

@piranna
Copy link
Member

piranna commented Nov 10, 2014

Is there really a difference between the web renderer and the canvas? I
mean the web renderer should be able to render a canvas element, so it's
basically two birds with one stone.
The only thing I can think of when the native canvas would be better
solution where speed is essential and memory usage should be minimal (you
don't have to spin up a new web renderer to draw something).

You have answered yourself :-) Also, EGL will probably needed for CEF too,
so its two birds with one stone too :-D

Canvas can be used for simple, full screen applications, but for some more
complex environments like a Desktop Environment with taskbar, window
manager and so, I think is better to build it by using CEF and html, and in
fact there are already several of them build yet.

@piranna
Copy link
Member

piranna commented Nov 10, 2014

I've updated and improved the wiki page about GUI programming.

@ghost
Copy link

ghost commented Nov 10, 2014

Do you know if DirectFB is up to the task? While I like the idea behind DFB, the only thing I could find about EGL is that they will support it in the future: http://directfb.org/wiki/index.php/DirectFB_2.0:_Enabling_Khronos_APIs
That wiki article is 6 years old... But I'm not sure yet how this all could work together, so it's possible that I missed something important here.

@piranna
Copy link
Member

piranna commented Nov 10, 2014

DirectFB is an example, I'm not willingly to use it for canvas object on
terminal. Both Cairo (used by node-canvas) and OpenVG (used by
node-openvg-canvas) has support for EGL, there are less layer involved and
we can get better performance.

2014-11-10 14:20 GMT+01:00 Csaba Szabo notifications@github.com:

Do you know if DirectFB is up to the task? While I like the idea behind
DFB, the only thing I could find about EGL is that they will support it in
the future:
http://directfb.org/wiki/index.php/DirectFB_2.0:_Enabling_Khronos_APIs
That wiki article is 6 years old... But I'm not sure yet how this all
could work together, so it's possible that I missed something important
here.


Reply to this email directly or view it on GitHub
#79 (comment).

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton
de sitios diferentes, simplemente escribe un sistema operativo Unix."
– Linus Tordvals, creador del sistema operativo Linux

@ghost
Copy link

ghost commented Nov 10, 2014

It sounds like node should handle all the inputs too. Is that right? So something like this:
https://gist.github.com/rhiokim/2034229
I'm asking this, because if I'll have some time, then in the coming weeks I'd like to hack together a proof-of-concept stuff. But I'm not sure about all that is necessary for this, and how much actual work would it take.
I know that none of you want to rush this, so this is just for me to play around in my free time.

@piranna
Copy link
Member

piranna commented Nov 10, 2014

It sounds like node should handle all the inputs too. Is that right?

On the terminal and by using a canvas object yes, Node.js will handle all the inputs.

So something like this: https://gist.github.com/rhiokim/2034229

Ey, I was already investigating on mouse support! :-P You are welcome to work on it :-)

I'm asking this, because if I'll have some time, then in the coming weeks I'd like to hack together a proof-of-concept stuff. But I'm not sure about all that is necessary for this, and how much actual work would it take. I know that none of you want to rush this, so this is just for me to play around in my free time.

Well, at this moment we are all working on this on our spare time... :-)

@mitsukaki
Copy link
Contributor

QEmu is configured to access ports 50080->80 and
50433->433, but the reverse proxy is only listening to the 80.

Ah okay. I was just having issues connecting to oneshoot running on QEMU. Does ifconfig in NodeOS return the correct IP or is that a default of sorts?

@piranna
Copy link
Member

piranna commented Nov 26, 2015

Does ifconfig in NodeOS return the correct IP or is that a default of sorts?

Inside QEmu it will always get the 10.0.2.15 by DHCP. Don't know how complete is NodeOS ifconfig, I think it only has support to put up the network interface, but it's not being used at this moment since at this moment the IP is being get by the kernel itself on the boot process.

@zodern
Copy link
Contributor Author

zodern commented Nov 26, 2015

If you make a generic interface, I will implement the same. (Woohoo! You get to write a spec! 😆 )

Let me get it working, and then we can discuss this. Basically there needs to be a way to pass which port to use to the app (maybe as an argument when spawning) or allow the app to share the port with the desktop, and then allow calling the api across processes. Would your "voidspace" help with that?

by providing a basic outline of the mandatory things, that remain the same, apps will be able to operate properly under any desktop manager

Interacting with windows and opening a file is what first comes to my mind as being part of a standard api. Should we also have a standard app.json for apps? I will create the issue next week when I get back from my trip (I won't have internet for the rest of the week).

@piranna That is exciting! I got it to work. It would be nice if it would let you try again after getting the password wrong 😄 . I switched to private browsing to get around that. Also, I am confused if oneshoot is a GUI or if it will be used by GUIs because of this line.

I got silk remote to work while running in NodeOS, though only some of the apps started within the allotted 5 seconds.

@piranna
Copy link
Member

piranna commented Nov 27, 2015

Basically there needs to be a way to pass which port to use to the app (maybe as an argument when spawning) or allow the app to share the port with the desktop, and then allow calling the api across processes

At this moment oneshoot is writting the port that's using on stdout so this way nodeos-reverse-proxy can know where to target the requests. I was thinking the same mechanism would be used for GUI apps that need a custom server backend: use the oneshoot WebSocket to start a new instance of the server, capture the port where it's running, and open the new window pointing to it. I was thinking maybe we could do it in a generic way, for example using a new file descriptor instead of stdout to don't clash with apps current behaviour.

Interacting with windows and opening a file is what first comes to my mind as being part of a standard api.

Interacting with windows is more a task of the Windows Manager, on the client side. Do you propose to have here a common API to do common tasks (maximize, minimize, close, move...)? I think OpenDesktop has something similar, maybe we could take a look there. Regarding files access, the simple and obvious thing is use directly HTTP methods.

Should we also have a standard app.json for apps?

I think it's better to re-use package.json adding the custom fields that we would need (icon? anything more?) instead of add a new manifest file...

It would be nice if it would let you try again after getting the password wrong 😄 .

I'm not sure if this is how HTTP Auth works, maybe there's a way to revoke the credentials...

Also, I am confused if oneshoot is a GUI or if it will be used by GUIs because of this line.

It's intended to be used by GUIs to do standard tasks like serve static content and provide WebSocket connections, if a GUI app needs to access to some Node.js APIs because they are not available on current browsers (for example open an UDP connection or configure the wifi or mount a pendrive) then it would use a WebSocket connection to start it's own backend server to do this non-standard tasks. I know, this would lead to have two OSes at the same time with incompatible APIs (NodeOS in Node.js, and NodeOS GUI in HTML + Javascript), but it's the best option for organization and interoperability.

I got silk remote to work while running in NodeOS, though only some of the apps started within the allotted 5 seconds.

That's strange, the 5 seconds only take effect when there's no more open connections to that user server (both HTTP and WebSocket) and get cancelled and renewed when a new one occurs... I've just reviewed the code and didn't find any problem here... :-/

@piranna
Copy link
Member

piranna commented Nov 27, 2015

I have been thinking regarding the GUI apps, and I think we can make the scheme really simple. We have two kind of apps:

  • pure web apps, that can run on regular browsers
  • hybrid web apps, that require a server to access Node.js APIs

Apps can be "stored" inside npm packages, augmenting the package.json file with some fields like icon and others, and pure web apps could set the main field to its index.html. Hybrid web apps would be regular Node.js apps with its own webserver and so, being that at server.js (the default for npm start).

We launching a GUI app, to identify them it's just a matter of check for the existence of the icon field on package.json and a index.html or a server.js file. Pure webapps will just open the index.html file by its URL, while hybrid ones will need to start the server.js first (HTTP REST request? WebSocket?), that will return the URL of the just started web server.

In that sense, oneshoot would be considered the server.js of a Desktop Environment app (or something very similar) but this has a drawback when connecting from an external machine (at this moment the only way), that's required to NAT a port for each app. HTTP Digest Auth has a realm field that seems to be somewhat specific to the resource, so it would be set to the TCP port of the app so nodeos-reverse-proxy can be able to identify it and do the redirection by only exposing the ports 80 and 433, the same way currently is being done with the user name.

Thoughts? Comments? Do you think there's another way?

@piranna
Copy link
Member

piranna commented Nov 28, 2015

I've reviewed the Auth Digest and it's not possible to fetch the realm on the server. I thought about sending the port as a part of the path to identify the requests that don't go to the main one (the GUI server, oneshoot), but this has the problem that's not possible to know when a server can be clossed or how to restart it, so only solution seems to be to send a custom header with the path of server.js file so nodeos-reverse-proxy can be able to start it on demand. This has the advantage that alloe to make the system fully homogeneous, too :-)

@mitsukaki
Copy link
Contributor

Basically there needs to be a way to pass which port to use to the app (maybe as an argument when spawning) or allow the app to share the port with the desktop, and then allow calling the api across processes. Would your "voidspace" help with that?

Voidspace helps with sharing variables across multiple processes. It also allows RPC calls to be made between them. If that's what your program needs, yes voidspace can help 😄

Voidspace is accessible externally, as long as the port voidspace is listening on, is known by the connecting client, and open for connections.

@mitsukaki
Copy link
Contributor

solution seems to be to send a custom header with the path of server.js file so nodeos-reverse-proxy can be able to start it on demand. This has the advantage that alloe to make the system fully homogeneous, too :-)

This doesn't sound toooo hard 😝

@piranna
Copy link
Member

piranna commented Nov 28, 2015

This doesn't sound toooo hard 😝

Yeah, but there's the problem of needing to add the header on the petitions, and this is not transparent... :-/ Honestly, the only problem here is when an app makes use of AJAX requests or asynchronous load (like by using templates or specially injecting img tags or similar, where you can't be able to add headers), because if not, all the request would go the server could be done on load and the server would die after that, so the problem is about how to restart it later :-/

I think I'll try to do something working to try to find how to make this work by using a "design by doing" approach, I think static content would be served by the main one without problems.

@mitsukaki
Copy link
Contributor

when an app makes use of AJAX requests or asynchronous load (like by using templates or specially injecting img tags or similar, where you can't be able to add headers)

Ahh I see what you mean. What was the issue again with sending the destination as a URL parameter?

@piranna
Copy link
Member

piranna commented Nov 28, 2015

What was the issue again with sending the destination as a URL parameter?

To be able to route it to the port where the app server is listening. I was thinking about using the port as a prefix in the URL, but this could change. Now I think I could be able to use the server path as the prefix the same way I was going to send it on the header, since this doesn't change. I would need to parse the URL, but maybe it's possible...

@mitsukaki
Copy link
Contributor

To be able to route it to the port where the app server is listening. I was thinking about using the port as a prefix in the URL, but this could change. Now I think I could be able to use the server path as the prefix the same way I was going to send it on the header, since this doesn't change. I would need to parse the URL, but maybe it's possible...

Oh so that is what you're going to do? Okay yeah that sounds like an excellent idea. If you need a hand, tag me as always ^_^

@piranna
Copy link
Member

piranna commented Dec 1, 2015

Oh so that is what you're going to do? Okay yeah that sounds like an excellent idea. If you need a hand, tag me as always ^_^

After thinking too much, the only solution I have found to be able to map per-application servers to the external world is to include the server name or path inside the URL, separating it with some special charactes. In this way, I think the best is to identify the first path element to see if it's in brackets, and if so, its content say the server that must be used. For example, ws://12.34.56.78/[terminal]/foo/bar would start the server of the terminal app on the /foo/bar directory. The brackets would host a full path if the server is not from a (user) globally installed module, like http://12.34.56.78/[/Steam/steam-ui]/hl3 would start the web server located at /Steam/steam-ui (if it's not already started) and give him the path /hl3.

@lite20 and @zodern, what do you think about this scheme to start backend servers for the GUI apps?

@zodern
Copy link
Contributor Author

zodern commented Dec 1, 2015

I like that idea.

Above I wrote

  1. A graphical way to access a server. Currently you can install it on the server, run with silk -r and go to the remote url. It needs to be more secure and not have to be started each time using ssh.

If silk could be started with ws://12.34.56.78/[silk]/foo/bar that would be great. Go to remote.silkgui.com, enter the server address and then get redirected to the window manager once it is ready.

When accessing icon.png from the steam-ui server after it has been started, would I use http://12.34.56.78/[/Steam/steam-ui]/hl3/icon.png or http://12.34.56.78/hl3/icon.png or something else ?

For the actual apps, I think it would be better for Silk (or the desktop the user is using) to start them so it can make the api available to the app and have control over the process so it can stop the app when the user is done with it. Though if I use VoidSpace I could use that instead for the api and use your proposed idea to start the apps.

@zodern
Copy link
Contributor Author

zodern commented Dec 1, 2015

One concern I have is Silk will have one way of running when in NodeOS and another when running on any other operating system.

@piranna
Copy link
Member

piranna commented Dec 1, 2015

If silk could be started with ws://12.34.56.78/[silk]/foo/bar that would
be great.

That's just the idea :-) nodeos-reverse-proxy will then later request
itself the credentials using HTTP Auth.

When accessing icon.png from the steam-ui server after it has been
started, would I use http://12.34.56.78/[/Steam/steam-ui]/hl3/icon.png or
http://12.34.56.78/hl3/icon.png or something else ?

Good quedtion, since this was the use case that made me bang the head
against the wall all the weekend. It should be used relative paths, it's
said, hl3/icon.png, this way it gets transparent to the brackets section in
the url both if using its own web server or by using the static one from
oneshoot.

For the actual apps, I think it would be better for Silk (or the desktop
the user is using) to start them so it can make the api available to the
app and have control over the process so it can stop the app when the user
is done with it. Though if I use VoidSpace I could use that instead for the
api.

Take in account that servers need to finish someway, so if they are not
stateless AND designed to die themselves like oneshoot is, then you'll need
to end the backend process someway to don't left it running and you'll also
need to have some mechanism to recover if the backend server dies... I
think when this gets more mature I'll made a library based on oneshoot to
make this process simpler.

@piranna
Copy link
Member

piranna commented Dec 1, 2015

One concern I have is Silk will have one way of running when in NodeOS
and another when running on any other operating system.

Why you think so?

@zodern
Copy link
Contributor Author

zodern commented Dec 1, 2015

Why you think so?

I do not understand oneshoot enough. I am going to spend this weekend working on Silk and let you know then.

@mitsukaki
Copy link
Contributor

@lite20 and @zodern, what do you think about this scheme to start backend servers for the GUI apps?

I like this implementation, It's simple. ^_^

I do have a few questions though. Could services be remotely started with this? That could become a security issue as once a malicious script is on the device, a hacker could easily execute it via this interface. Are there plans to add some sort of authentication?

When accessing icon.png from the steam-ui server after it has been
started, would I use http://12.34.56.78/[/Steam/steam-ui]/hl3/icon.png or
http://12.34.56.78/hl3/icon.png or something else ?

Good quedtion, since this was the use case that made me bang the head
against the wall all the weekend. It should be used relative paths, it's
said, hl3/icon.png, this way it gets transparent to the brackets section in
the url both if using its own web server or by using the static one from
oneshoot.

If putting the application in brackets would launch the application, wouldn't this request launch a new instance of the application? Infinitely calling requests and no end would actually be reached?

How would we go about implementing it? I don't know enough about websockets in nodejs but in the example you bound to a path. I didn't even know that was do-able 😛 After you implement everything, a bit of experimentation should suffice to get a working system off it.

My understanding of oneshoot is also still extremely cloudy. If you could write an example implementation or even a super basic tutorial explaining what's happening behind the scenes that would be incredible 😄

The codefest continues...

@piranna
Copy link
Member

piranna commented Dec 2, 2015

Could services be remotely started with this?

More than services, GUI apps backend server. But yes, that's the idea.

That could become a security issue as once a malicious script is on the
device, a hacker could easily execute it via this interface. Are there
plans to add some sort of authentication?

Yes, all requests are authenticated using HTTP Auth agains the user name
and password used by logon :-) Currently this is done using HTTP Basic Auth
that send passwords in clear text so someone sniffing could get them, but
combined with SSL this is more secure and faster than Digest Auth, so I'm
thinking we could made HTTPS mandatory, the problem is the management of
certificates, since this would be common for all users... :-/ Maybe the
administrator boot mode could do the trick to install them :-)

If putting the application in brackets would launch the application,
wouldn't this request launch a new instance of the application? Infinitely
calling requests and no end would actually be reached?

No, it isn't :-) Servers are started by nodeos-reverse-proxy, and it has a
map from the server path to the server instance, so if there's an entry for
that path, it doesn't start a new server instance :-)

How would we go about implementing it? I don't know enough about
websockets in nodejs but in the example you bound to a path. I didn't even
know that was do-able

Magic, you know ;-) When I found that trick was possible some years ago I
was as surprised as you :-D

My understanding of oneshoot is also still extremely cloudy. If you could
write an example implementation or even a super basic tutorial explaining
what's happening behind the scenes that would be incredible

I'm going to rewrite oneshoot as a library. Think of it as a normal server
created with the Node.js 'http' module that close itself when there are no
more open connections, there's no more than that. The other half is done by
nodeos-reverse-proxy, that start the oneshoot server when a new connection
is incoming and there's no currently a (oneshoot) server to listen to it,
no more. Hope that helps to understand how they work together :-)

@zodern
Copy link
Contributor Author

zodern commented Dec 2, 2015

Think of it as a normal server
created with the Node.js 'http' module that close itself when there are no
more open connections, there's no more than that. The other half is done by
nodeos-reverse-proxy, that start the oneshoot server when a new connection
is incoming and there's no currently a (oneshoot) server to listen to it,
no more. Hope that helps to understand how they work together :-)

That's what I needed! Thanks. My concern is gone.

@piranna
Copy link
Member

piranna commented Dec 2, 2015

That's what I needed! Thanks. My concern is gone.

Cool dude ;-)

@zodern
Copy link
Contributor Author

zodern commented Dec 2, 2015

Regarding files access, the simple and obvious thing is use directly HTTP methods.

Sorry I didn't explain what I meant well. I was referring to clicking a file in file explorer and then having another app open the file. Silk has a property in the app.json for what files the app can open and a client api for when it is supposed to open a file. Currently Silk opens a window for the app that will open the file and adds a query string with the file path. There probably is a better way of doing it.

I think it's better to re-use package.json adding the custom fields that we would need (icon? anything more?) instead of add a new manifest file...

I agree. There is the icon, window url, if it supports multiple windows, if it needs any other ports available remotely, types of files it opens, and the app name that all would need custom fields. I can use a bower.json for the bower dependencies and get the window title from the html.

That's strange, the 5 seconds only take effect when there's no more open connections to that user server (both HTTP and WebSocket) and get cancelled and renewed when a new one occurs... I've just reviewed the code and didn't find any problem here... :-/

Silk gives apps 5 seconds to start, and qemu without kvm was too slow. It wasn't because of oneshoot.

Apps can be "stored" inside npm packages, augmenting the package.json file with some fields like icon and others, and pure web apps could set the main field to its index.html. Hybrid web apps would be regular Node.js apps with its own webserver and so, being that at server.js (the default for npm start).

I was thinking about this last spring! I was also trying to figure out how to use npm for downloading, updating and removing apps but I am not sure I want apps to be inside node_modules.

@zodern zodern mentioned this issue Dec 2, 2015
4 tasks
@piranna
Copy link
Member

piranna commented Dec 2, 2015

I was referring to clicking a file in file explorer and then having another app open the file.

This is a totally different discussion. I think this is better handled by using protocol handlers and content handlers.

I can use a bower.json for the bower dependencies and get the window title from the html.

Ok for the window title thing. Why need to use bower? We can get the dependencies from npm too if the browser libraries are included as it's done in the main ones...

Silk gives apps 5 seconds to start, and qemu without kvm was too slow. It wasn't because of oneshoot.

Ah, ok. Maybe is there any other way to check that they start that's not using a timeout?

I am not sure I want apps to be inside node_modules.

Obviously this is not mandatory, and my brackets scheme allow to have apps in any place, but what problems would we have by having the apps inside node_modules? I think it's its natural place, specially if they are managed by npm... This way they can have later an entry on /bin, too...

@piranna
Copy link
Member

piranna commented Dec 2, 2015

I've just updated onshoot to make it require()able, but didn't test it yet. Take a look on it to see if you understand it better ;-) I've also changed it to allow to send the port over the communication channel if executed using child_process.fork() :-)

@mitsukaki
Copy link
Contributor

Think of it as a normal server
created with the Node.js 'http' module that close itself when there are no
more open connections, there's no more than that. The other half is done by
nodeos-reverse-proxy, that start the oneshoot server when a new connection
is incoming and there's no currently a (oneshoot) server to listen to it,
no more. Hope that helps to understand how they work together :-)

It all makes sense now. XD

@piranna
Copy link
Member

piranna commented Dec 16, 2015

I've found that the latests changes on Cloud9 almost implement exactly what I was thinking for a GUI, but not also that. it's free software, it's published on npm and follow the same scheme of server + URL I was thinking about! Yay! :-D

If not as GUI desktop, at least we need to make this work as IDE :-)

@mitsukaki
Copy link
Contributor

Hahaha leaving this thread open for discussion. If it stays inactive, we can close it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests