New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keynote on NodeJsMadrid group #73

Closed
piranna opened this Issue Oct 14, 2014 · 17 comments

Comments

Projects
None yet
4 participants
@piranna
Copy link
Member

piranna commented Oct 14, 2014

They have just accepted that I do a keynote about NodeOS, so I'll need to prepare something... :-) Not a hurry, probably it will be in some months, there are other ones on the queue and I'm a bit bussy at this moment, but it's a step :-)

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Oct 19, 2014

This is fantastic. Let me know if you need any help, and feel free to email/gchat me.

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Oct 19, 2014

This got some interest here at Spain and on Twitter, so don't doubt I'll ask you if I need help ;-)

@aulvi

This comment has been minimized.

Copy link
Member

aulvi commented Oct 25, 2014

Jesus, congrats on your talk acceptance, it's super cool to hear that
you'll be talking about NodeOS :D

On Sun, Oct 19, 2014 at 11:55 AM, Jesús Leganés Combarro <
notifications@github.com> wrote:

This got some interest here at Spain and on Twitter, so don't doubt I'll
ask you if I need help ;-)


Reply to this email directly or view it on GitHub
#73 (comment).

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Oct 25, 2014

Well, this will not be soon, there're other talks in the queue and also I'm busy with the work and the college, but definitely it's a great news :-D

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Nov 6, 2014

Just a note: I talked with one of my ex-teachers at the Operating Systems group at my university (the ones that were crazy enough about Plan9 up to the point to try to teach it us :-P ) and got impresed about NodeOS and is open to host a keynote about it if I'm interested :-) so maybe I could make the one for the NodeJsMadrid group and for my university both at the same time... :-D

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Nov 18, 2014

NodeOS has been accepted for the spanish national Universitary Free Software Championship :-)

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 13, 2015

Finally I will be doing the keynote the next thursday January 27th :-)

http://www.meetup.com/Node-js-Madrid/events/219754655/

Give me luck! :-D

@formula1

This comment has been minimized.

Copy link
Contributor

formula1 commented Jan 13, 2015

GOOD LUCK! KICK SOME ASS! ^_____________^

err... don't kick anyones butt... But I'm confident they will be impressed :)

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 13, 2015

GOOD LUCK! KICK SOME ASS! ^_____________^

err... don't kick anyones butt... But I'm confident they will be impressed :)

Lol! X-D

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 28, 2015

Here you have the hangout of the keynote. Unluckily, it only recorded the audio :-( The next time I'll confirm it ;-)

The people liked the project, half of them was ashtonished and the other half didn't understand anything X-D Some things they asked more where about

  • security of the system regarding other Linux
  • memory footprint (start a Node.js instance for each process is huge, we need to improve this)
  • drivers compatibility
  • access to other systems (they didn't understand very well the concept of "open standards"...)

And as a personal note, we need

  • some cool demos working, like blessed-contrib
  • improve final user (nsh, console keymaps and fonts...)
  • a markdown-to-console parser so the keynotes can be run on a NodeOS console :-P

The general reception was good, that's really good being the most of them web developers and not system programmers... ;-) Next time it will be better :-)

@piranna piranna closed this Jan 28, 2015

@piranna piranna removed the discussion label Jan 28, 2015

@formula1

This comment has been minimized.

Copy link
Contributor

formula1 commented Jan 29, 2015

I'm listening to it now :) Unfortunately, my spanish is very rough but I understand "python", "trabajando", "minimalista", "pero", "necessito", "tiene", "primera", "el systema", "permisos", "mas cosas", "problemas", "tan poco", "authero", "personalmente", "root", "mas instances", "normales", "aprender", "como se dice", "mas todas", "repositorio", "positivos"... Basically words and phrases but not sentences.

I tried out closed captioning english translation. I unfortunately got what I was expecting

25:13 - the aircraft's railway system are as of unemployment can be divided ass

Security

This is something I was considering myself however we then need to consider what can be made secure?

  • User to User- I'm pretty node processes generally are given the same permissions as the user. I know this linux system is pretty stripped down, but when it comes to closing other peoples processes/filesystem I assume that is in linux core. However, this would then force many of the files/folders to be under 770 permissions. Which may produce some breaking aspects.

  • Foreign User-This is something that makes things a little weird. Because it is an open environment, it is insecure by feature. There is no roadblock to prevent users from creating accounts. And if the roadblock were to exist, there is no "root" to maintain it.

  • Foreign Applications-Since almost all applications installed are created by the user, they have the permission to do whatever they want within that users file system/other processes owned by the same user. It is nearly impossible to prevent this without a few possibilities

    Create a whitelist for applications per user
    Application's are always started as a permissionless dummy user unless they are whitelisted

  • Ports-I believe you mentioned that each user will have their own emulated server (I forget the technical name)

    However, when making those ports public to the world is a shared space that requires admin permissions

Memory FootPrint

I'm not sure the best way to go about this. Heres a couple questions

  • Are we creating the instances as if we are running node /path/to/file
  • Is there a benefit to using child_process if we aren't using it?
  • Is the memory footprint an issue with creating v8, libuv or node instances?
  • If v8, is it possible to use the same v8 runtime as a different isolate? Would that reduce memory? Would it also prevent important interactions with Linux and/or node modules?
  • if libuv (which I doubt), should we consolidate the runtimes to use the same eventloop (which sounds like a bad idea already :P)?
  • if node runtime, is there anything global that we make requirable instead that would reduce the footprint significantly?

Drivers

I think thats where things always fall apart : / Propriety anything generally only wants to interact with popular frameworks. Look at RealSense, its for windows 10 only :C And trying to recreate every driver ever made seems like a losing battle : /

Access to other systems

What were they asking about? SSH? Communicating over the network with other OS'? Recognizing other partitions File Systems?

@formula1

This comment has been minimized.

Copy link
Contributor

formula1 commented Jan 29, 2015

https://gist.github.com/formula1/e525704728293f7d27a6

I figured I'd help you make the markdown to blessed since its mostly regex->Whatever. However, ran into some issues. This is my first time using blessed so I might be rendering things wrong. Additionally, I just through out random heights because it wasn't rendering the first time. Unfortunately it didn't render the second either. That being said this http://spec.commonmark.org/0.17/ is extremely casual. I know its supposed to be or whatever, and that there is no "standard" but theres quite a few things I decided to skip just because it really didn't seem worth my time. Amoung them are

  • HTML elements - All of them. Its kind of rediculous, the concept of writing html in markdown. Why not just write in html at that point?
  • Identation for fenced code - 3 spaces of identation negation seems hardly worth it. Especially when most code uses tabs or double space. At that point its ludicrous to consider that 3 spaces will make or break readability

What I didn't get to

  • Block quote and below

Some cute things I tried to include

  • code highlighting via console-highlight
  • Using streams transform to allow piping asynchronously
@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 31, 2015

I'm listening to it now :) Unfortunately, my spanish is very rough but I understand "python", "trabajando", "minimalista", "pero", "necessito", "tiene", "primera", "el systema", "permisos", "mas cosas", "problemas", "tan poco", "authero", "personalmente", "root", "mas instances", "normales", "aprender", "como se dice", "mas todas", "repositorio", "positivos"... Basically words and phrases but not sentences.

We were using the laptop microphone, maybe that's the reason it has
bad sound. Anyway, you got a lot of things, congrats! :-)

I tried out closed captioning english translation. I unfortunately got what I was expecting

25:13 - the aircraft's railway system are as of unemployment can be divided ass

Lol :-P

Security

This is something I was considering myself however we then need to consider what can be made secure?

User to User- I'm pretty node processes generally are given the same permissions as the user. I know this linux system is pretty stripped down, but when it comes to closing other peoples processes/filesystem I assume that is in linux core. However, this would then force many of the files/folders to be under 770 permissions. Which may produce some breaking aspects.

In fact, we are using umask 0077, so anybody except the files and
directory owner can access to them, and in / directories most of them
has mode 0111 so anybody except the root user can know what's there
and in some cases also 0100. Too restrictive, but the idea here is
that "if it's supossed you should not be fiddling somewhere, you
definitely can access there" :-) For example, normal users don't need
to know about what's in the /lib or /proc folders... It's done this
way by design, but I'm open to discuss this point (in another issue to
don't contaminate this one, please ;-) )

Foreign User-This is something that makes things a little weird. Because it is an open environment, it is insecure by feature. There is no roadblock to prevent users from creating accounts. And if the roadblock were to exist, there is no "root" to maintain it.

At this moment is not implemented, but it's one of my ideas for logon,
add a mode where anybody can create a new account on the system in a
similar way to how Plan9 login system works (anyone can login, but you
need to give some credentials to access your files), that's basically
a new folder on the users filesystem. This is a consecuence of not
having a central users list like /etc/passwd. I was thinking to
require reboot in "administer mode" to enable/disable this global
behaviour flags or do this maintenance task... This would also force
to think out-of-the-box to don't need to reboot the machine all the
days :-) What do you think?

Foreign Applications-Since almost all applications installed are created by the user, they have the permission to do whatever they want within that users file system/other processes owned by the same user. It is nearly impossible to prevent this without a few possibilities

Create a whitelist for applications per user
Application's are always started as a permissionless dummy user unless they are whitelisted

The idea is that users are sandboxed, if something terrible occurs,
just delete that account and create a new one. I don't think it makes
sense to try to prevent one application damaging the data or other
applications inside an user folder since we want to give the feeling
it's the (fake) owner of all the system and can do everything... Is
this what you were talking about?

Ports-I believe you mentioned that each user will have their own emulated server (I forget the technical name)

Yeah, some time ago I though having a global reverse proxy that would
redirect incoming petitions to each user server, but this mostly only
would work for HTTP & WebSockets, since they have the concept of path
on each request petition, so you can filters them someway. For other
more "simple" protocols like FTP this would be impossible. HTTP &
WebSockets are becoming the de-facto standard and offer a lot of
flexibility for this kind of managemente, so we could "force" to give
only them for NodeOS developers as basic tools, so they don't need to
fight with ports management and other low-level things, something like
a "PaaS in da haus" :-P In fact, I'm not sure if Heroku, Nodejitsu and
other PaaS offer another access mechanism that are not HTTP &
WebSocket and seems to be just enought, isn't it?

However, when making those ports public to the world is a shared space that requires admin permissions

This is another alternative I though more recently, use LXC containers
so not only users & process are totally isolated, but also could
access to a full virtual range of ports and could also be controled
other system resources. This is more or less like how Docker works. it
would me more flexible and probably more natural for developers that
the reverse proxy solution (it depends how it's implemented, one
solution is to change the Node.js http module for a handcrafted one
that makes transparent the usage of the reverse proxy), but definitely
it will be more dificult... :-/

Memory FootPrint

I'm not sure the best way to go about this. Heres a couple questions

Are we creating the instances as if we are running node /path/to/file

Yes :-/ We don't know if the user will be trying to exec a binary
command, we are not mandating (yet) that all executables are
Javascript files... One posibility is to check on the /usr/bin/env
binary or at the child_process.spawn method if we are triying to exec
a Javascript file and attach it to a current Node.js process.

Is there a benefit to using child_process if we aren't using it?

Exec binaries without worry about they are Javascript or compiled ones
:-) Also be able to daemonize a process, and some time ago I got
problems at work by using execFile (don't remember what, maybe race
condition). We could investigate this use cases to monkeypatch
child_process for this cases, though.

Is the memory footprint an issue with creating v8, libuv or node instances?

http://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options
"These child Nodes are still whole new instances of V8. Assume at
least 30ms startup and 10mb memory for each new Node. That is, you
cannot create many thousands of them. "

So yes... :-/ I believe this is the reason why x86-64 needs to upgrade
QEmu memory to 256mb, since I'm using spawn instead of fork (my
fault... and one of the reason I'm thinking to go back to use the
coreutils compiled one :-/ ).

If v8, is it possible to use the same v8 runtime as a different isolate? Would that reduce memory? Would it also prevent important interactions with Linux and/or node modules?

Definitely yes, it's possible to do this and will somewhat reduce
memory footprint :-) Problem is that there could appear race
conditions since they will be running on diferent theads in the same
process instead of on diferent process... :-/ But for some use cases,
yet, definitely it's posible.

if libuv (which I doubt), should we consolidate the runtimes to use the same eventloop (which sounds like a bad idea already :P)?

I don't think libuv has nothing to do here... :-P

if node runtime, is there anything global that we make requirable instead that would reduce the footprint significantly?

As I said before, we could check we are trying to exec a Javascript
file and attach it to a current Node.js instance... :-) Problems are
race conditions :-/ I remember to read about now Node.js is capable of
having two process on the same events queue thanks to Atom
requeriments, maybe the race conditions are not anymore a problem in
v0.11.x...

Drivers

I think thats where things always fall apart : / Propriety anything generally only wants to interact with popular frameworks. Look at RealSense, its for windows 10 only :C And trying to recreate every driver ever made seems like a losing battle : /

As I said in the keynote, this is a Linux system, if Linux support it
we support it. Only problem is that we are not using modules
(everything is compiled in the kernel for simplicity). We can be able
to revisit this in the future, though. Also, by using FUSE & CUSE and
compiled modules, everybody can be able to craft their own drivers in
Javascript... :-)

Access to other systems

What were they asking about? SSH? Communicating over the network with other OS'? Recognizing other partitions File Systems?

SSH, network communication and bash, yes. They didn't fully understood
the concept of standard protocols and the fact that is the remote
system use bash you don't need to have it installed locally... :-P

This reminds me that we need an USB pen-drive automounter, hum...

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 31, 2015

https://gist.github.com/formula1/e525704728293f7d27a6

I figured I'd help you make the markdown to blessed since its mostly regex->Whatever. However, ran into some issues. This is my first time using blessed so I might be rendering things wrong. Additionally, I just through out random heights because it wasn't rendering the first time. Unfortunately it didn't render the second either. That being said this http://spec.commonmark.org/0.17/ is extremely casual. I know its supposed to be or whatever, and that there is no "standard" but theres quite a few things I decided to skip just because it really didn't seem worth my time. Amoung them are

HTML elements - All of them. Its kind of rediculous, the concept of writing html in markdown. Why not just write in html at that point?
Identation for fenced code - 3 spaces of identation negation seems hardly worth it. Especially when most code uses tabs or double space. At that point its ludicrous to consider that 3 spaces will make or break readability

What I didn't get to

Block quote and below

Some cute things I tried to include

code highlighting via console-highlight
Using streams transform to allow piping asynchronously

Wait... have you carft a markdown to blessed parser? O_o Wow,
impresive work... I'm so sorry to give you bad news, but...

  1. there's already several markdown parsers for Node.js :-P kramed
    seems to be a good one, it's being used by Gitbook.io and it's pretty
    complete :-)
  2. blessed is mostly focused to design text-based user interfaces:
    forms, textareas... I think a terminal-based keynotes tool only would
    need colors and ANSI characters, and maybe some Unicode characters to
    make it more cool (basically, all the building blocks that blessed is
    build on top of :-) )
  3. I was talking about blessed only for the impresive demo of
    blessed-contrib, that's really cool to show on a keynote :-P Problem
    is that it can run correctly on NodeOS yet, seems it's a problem with
    Unicode support (mostly, we don't support it yet :-/ ).

Anyway it's a huge work, congrats! :-D Maybe we can use it someway :-)

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jan 31, 2015

Regarding to the keynote tool on the console, I've found this ones... and some are written in Node.js ;-)

@formula1

This comment has been minimized.

Copy link
Contributor

formula1 commented Feb 1, 2015

0077 Permisions

Though I'm working within ubuntu, I know very little about linux : / I know the bare minimum to do server work and make sure my own desktop system is in place. I trust whatever system you employ. I will definitely say, what you're suggesting definitely seems ideal.

Create a new user when your system gets messed up

I like this idea alot and I remember you talking about creating user profiles so that they can be shared/reinstalled as well similar to chef and docket. This definitely seems ideal. I suppose I never thought of the idea of making everything dispensible, however, I suppose when it comes to important documents, backing them up is probably the best decision anyway.

HTTP and Websockets are enough, Its possible to use LXC containers

Thats a good question. You're correct that heroku and nodjitsu don't allow FTP access. Though SSH is pretty important in many regards, it can be emulated via websockets as well. I was attempting to do my own research on it previously though I didn't write much as it started to become apparent the learning curve for it was much higher than a week long project. LXC seems well maintained. There doesn't seem to be any node bindings, though that is not be a huge issue. I suppose for

v8, fork/spawn and race condtions

So atom forced node to do some good things, interesting. I wonder if io.js even has this on their radar. Considering they are a fork, they have everything node does though it is pretty interesting nonetheless. To bring everything into the same event queue, wouldn't that make every module just requirable?

Drivers

That makes sense. I stand corrected.

USB pen drives

node-serialport may be relevent

I recreated the wheel

T___T No issue, honestly they probably did it better than I did. Not going to lie, I spent maybe 7 hours on it so I don't consider it worthwhile, though it was kinda fun. If theres already stuff out there, I'm not going to work on mine anymore :P. I have other projects I'd like to get back into, though helping NodeOS succeed is definitely on my priority list.

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Feb 1, 2015

0077 Permisions

Though I'm working within ubuntu, I know very little about linux : / I know the bare minimum to do server work and make sure my own desktop system is in place. I trust whatever system you employ. I will definitely say, what you're suggesting definitely seems ideal.

This is only done on users folders so only them can access to their
files, as I said on other dirs I'm using custom ones. By the record:
if you don't have enabled the execution dir (0111) you can be able to
access to its files at all (I shooted in my feet by setting 0000 on
system folders... :-P ).

Create a new user when your system gets messed up

I like this idea alot and I remember you talking about creating user profiles so that they can be shared/reinstalled as well similar to chef and docket.

Yes, a per-user package.json with all its installed packages :-)

This definitely seems ideal. I suppose I never thought of the idea of making everything dispensible, however, I suppose when it comes to important documents, backing them up is probably the best decision anyway.

Obviously... :-D But create a new user is not like format your hard
disk. Root partition is intended to be minal and read-only, so it
should work allways, if you mess up is in your home directory, and in
Node.js the only directories that would need to be removed to start
over are ~/bin and ~/.lib/node_modules, only problem here would be if
you installed a library/program that's not on NPM or you did it
manually.

But yes, a backup is always a good idea, hard disk failures happens :-P

HTTP and Websockets are enough, Its possible to use LXC containers

Thats a good question. You're correct that heroku and nodjitsu don't allow FTP access. Though SSH is pretty important in many regards, it can be emulated via websockets as well.

I was thinking so, WebSockets can emulate SSH, with the advantage that
the reverse proxy would redirect directly to your own wssh instance in
your own home folder :-)

I was attempting to do my own research on it previously though I didn't write much as it started to become apparent the learning curve for it was much higher than a week long project. LXC seems well maintained.

LXC is a feature of Linux kernel, what you pointed out are the
user-space tools. Anyway, it's well maintained and tested since more
than 5 years ago and it's derived from other system used for about 15
years ago ;-) Oh, and based on one of the corner-stone of Plan9,
designed in 1991... :-P

(Disclaimer: I learned operating systems at university using Plan9,
and my teacher was Francisco J. Ballesteros "nemo", one of the Plan9
popes and author of http://lsub.org/who/nemo/9.intro.pdf, that was our
class book, that's why I talk so much about it :-P)

There doesn't seem to be any node bindings, though that is not be a huge issue. I suppose for

Well, there's something... :-)

https://www.npmjs.com/search?q=lxc
https://www.npmjs.com/package/lxc
https://www.npmjs.com/package/libvirt

But definitely not too much and in a bad state :-( But it's something
to start about :-)

** V8, fork/spawn and race condtions **

So atom forced node to do some good things, interesting.

http://strongloop.com/strongblog/whats-new-node-js-v0-12-multiple-context-execution/
Money can do almost anything... ;-)

I wonder if io.js even has this on their radar. Considering they are a fork, they have everything node does though it is pretty interesting nonetheless. To bring everything into the same event queue, wouldn't that make every module just requirable?

They doesn't need to be requirable, it's something related to the
runtime, no more. The event queue is just a list of functions waiting
to be executed, each of them with a reference to their context stack,
it's just a matter of removing global variables :-) The collateral
consecuence of this, is that without global variables, you can consume
the event queue by several CPUs without race conditions or performance
problems, too!!! :-D

** I recreated the wheel **

T___T No issue, honestly they probably did it better than I did. Not going to lie, I spent maybe 7 hours on it so I don't consider it worthwhile. If theres already stuff out there, I'm not going to work on mine anymore :P. I have other projects I'd like to get back into, though helping NodeOS succeed is definitely on my priority list.

Well, don't worry, it was funny, isn't it? :-) Anyway, I was reading
about the other alternatives and about blessing and blessing-contrib,
and seems it was not a bad idea at all :-) The alternatives are
somewhat primitive, being tvmc the most mature but using a custom
format, and being slide.js a bunch of spaguetti code by it's readme
(!). Also, by using directly ANSI color libraries you would recreate
what blessing and blessing-contrib are already doing (text label,
images...), so I would only remove the markdown parser part for a good
tested one and continue from there ;-) Seems it would be an
interesting project, specially if you craft a code higligh blessed
plugin, make it somewhat compatible with impress.js and add
animations... :-P

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment