Let's build an alpha

[piranna]

I have created a milestione so we can start to focus on develop an initial public release. We can discuss here what things should be added. Please maintain the features to the minimal (it's a first release, after all), and just put here what the idea, but develop them in a new issue.

• update & improve documentation
• minimal system
  • ramdisk root filesystem bundle (kernel + node binary packed in a ramdisk image)
  • minit (minimal init for PID 1)
  • boot module (for starting dhcp/getty via asgard)
  • asgard (to run as PID 2) Using forever-starter instead
    • getty replacement written in Node.js (for binding to /dev/console)
  • multiuser (logon + filesystem permissions)
  • OverlayFS-based root filesystem
    • read-only ramdisk root filesystem
  • network support
    • enable/disable network interfaces
    • static IPs
  • command line tools
    • npm
    • nsh update
    • text editor
    • git client, maybe we can use gitlet
  • The Big Red Button (script to build everything with just one command)
    • generate images for all platforms at once
    • prepare an empty file system
    • install all system packages (download?)
  • different UIDs & GIDs for each user home folder
    • DevicesManager (/dev filesystem)
  • /proc accesible for everybody
  • fbdev
• Instalable distro
  • Self-contained source code
    • custom, minimal compiled kernel
    • libraries
  • Multiple architectures
    • x86
    • x86_64
  • Multiple platforms
    • Docker Use Vagga, maybe with Docker compatibility
    • x86 QEmu
    • x86_64 QEmu
  • Run on hardware
    • kernel & initramfs in rootfs images
    • GRUB image (instalable by hand with dd and/or cp/script)
• no binary executables (only libraries bindings, executables are only Node.js programs)
  • replace /usr/bin/env with Javascript alternative (function? NODE_PATH? patched kernel? script with hard-coded she-bang?)

Minor tasks:

• Read kernel arguments from /proc (root and home)
• Allow logon to create new users
• Move root home to usersfs
• Make usersfs ownership to root
• Mount rootfs as read-only

[groundwater]

This is great!

Here's what I want to get done, by which I mean actually tested and reliable:

• minit (minimal init for PID 1)
• asgard (to run as PID 2)
• nsh update
• npkg update

If you want to run on hardware, you also need:

• getty replacement (for binding to /dev/console)
• dhcp client (including ability to set ip/rotue/netmask)
• boot module (for starting dhcp/getty via asgard)

We also need a script to take all these packages and prepare an empty file system.

/bin/node
/etc \
/usr  | stuff for libc
/lib /
/dev  (devfs)
/proc (procfs)
/sys  (sysfs)
/root/
  lib/node_modules/
    mini-init/
    asgard/
    nodeos-boot/
      node_modules/
        getty/
        dhcp-client/
  bin/
    asgard
    mini-init

In principle, this is all described by what modules get installed to /root. Perhaps we can just write an npm package with a bunch of dependencies in it's package.json, and write a npm-user install <PKG> to generate the desired system.

[piranna]

I've updated the head entry with your proposals. Some points:

dhcp client

I've set this as proposed because I think for a minimal first alpha
release we can start working with static IPs, but if you consider it
really necesary and/or there's one already published, we can add it to
that release :-)

boot module (for starting dhcp/getty via asgard)

Can you explain what do you want to achieve here? Is it script like
autoexec.bat launched by minit that starts asgard and register the
dhcp & getty services?

/bin/node
/etc
/usr | stuff for libc
/lib /
/dev (devfs)
/proc (procfs)
/sys (sysfs)
/root/
lib/node_modules/
mini-init/
asgard/
nodeos-boot/
node_modules/
getty/
dhcp-client/
bin/
asgard
mini-init

Filesystem looks pretty nice :-) I would like to clean / (/bin, /etc
and /lib could be moved to /root/* or have then hidden in the ramdisk,
and the other ones are not mandatory, just interfaces to kernel
structures. Only problematic one is /usr/bin/env, but probably can be
trick someway...), maybe it can be done later.

In principle, this is all described by what modules get installed to /root. Perhaps we can just write an npm package with a bunch of dependencies in it's package.json, and write a npm-user install to generate the desired system.

Like Debian meta-packages? Cool! :-)

[groundwater]

Can you explain what do you want to achieve here? Is it script like
autoexec.bat launched by minit that starts asgard and register the
dhcp & getty services?

The system needs to start services to be useful on boot. We should contain that boot behavior to an npm-style module. It would be easy to create both alternative boot strategies, and upgrade the boot package.

Assuming we use grub as our boot-loader, you would specify the package name in your grub entry. This makes it easy to boot the system in different ways by creating multiple grub entries.

Filesystem looks pretty nice :-) I would like to clean / (/bin, /etc
and /lib could be moved to /root/* or have then hidden in the ramdisk,

Some of this relies on how node is compiled. I'm using the libc from Ubuntu, so we should probably leave everything required in the Ubuntu spot. I'm all for looking into either a static node, or compiling node against some other libc.

Only problematic one is /usr/bin/env, but probably can be
trick someway

This is a minor pain, but if we decide the OS will be all-node, and only node we can hack this easily.

[piranna]

The system needs to start services to be useful on boot. We should contain
that boot behavior to an npm-style module. It would be easy to create both
alternative boot strategies, and upgrade the boot package.

Maybe asgard could store in a file the list of enabled services and
launch them on start-up, similar to what Ubuntu serviced does (when
you do "service start ", it gets started after reboot). Also, it
could collaborate with minit so it doesn't need to be running: when
launching a service, it's managed by minit and a command to a
Javascript file that's the one launched on start, so asgard would be
just a command line administration tool :-)

Assuming we use grub as our boot-loader, you would specify the package name
in your grub entry. This makes it easy to swap boot the system in different
ways by creating multiple grub entries.

+1 to GRUB :-)

Filesystem looks pretty nice :-) I would like to clean / (/bin, /etc
and /lib could be moved to /root/* or have then hidden in the ramdisk,

Some of this relies on how node is compiled. I'm using the libc from Ubuntu,
so we should probably leave everything required in the Ubuntu spot. I'm all
for looking into either a static node, or compiling node against some other
libc.

I should have add this comments to filesystem topic, sorry. It's not a
short term task, but I definitelly like the idea of using a static
node binary to remove all this filesystem garbage (this is why I asked
about make the user believe that "$home is $root").

Only problematic one is /usr/bin/env, but probably can be
trick someway

This is a minor pain, but if we decide the OS will be all-node, and only
node we can hack this easily.

Great ;-) I was also thinking in rewrite it in node and look over the
NODE_PATH, since in a first point it's the only one we would allow to
use...

[piranna]

At this moment I'm using the initramfs as root filesystem and mounting the users filesystem in /home, but don't know if It would be better to use AUFS as Docker does and mount it on /. This is mostly related to how common elements as TTYs should be initiated and managed, since by mounting on / would allow to use /root to install that common things, but also it would shown an asimetry between system users giving more importance to the root user as current system does, and also would means to give more priviledges to him...

[dbkaplun]

I think #67 git support is crucial, since so many npm packages depend on it! Can we add that under 'minimal system'?

[piranna]

git command line client added to the NodeOS Alpha wish-list due to popular demand

;-)

[groundwater]

I will be presenting node-os along with other interesting node-related systems work at JSConf Asia in Nov. I wonder if we can shoot for a demo that the audience can download from docker and run that is more up to date than my original?

I don't think we need anything that's feature-complete, I think people just want something they can start playing with.

In the talk, I would also like to enumerate all the people who helped get the demo working (especially @piranna).

[piranna]

My pull-request can be moved to master without problems.

P.D.: I've had holidays and later my leg broken the last weeks, that's why I had so much time to work on NodeOS... ;-)

[groundwater]

@piranna 😮

I hope you are okay!

I've freed up time to work on this now. Last month was crazy busy. I'm going to spend the next few days coming up to speed with your changes. Thank you 1000x again. I hope you get better soon!

[piranna]

Hi @groudwater! Yes, my leg is ok since several weeks ago. It sometimes hurts my knee but I'm doing normal life, thank you :-)

I have been having some hard weeks at works, but seems it will get slowly, and in fact I fixed some security issues this weekend :-) I will not have too much time in the next months due to classes but I'll try to find some time for NodeOS :-)

[groundwater]

Hey no problem. I'm going to try getting your pull-request working so I can use that as the final demo. Once I've got that going, I'd like to post working artifacts somewhere so people can quickly try it out.

[piranna]

Seems that I missed an issue with QEmu and it's not starting correctly, needs some investigation. Also would be good to have AUFS or OverlayFS support, but we can left it for a future release to see if OverlayFS gets integrated in linux kernel main code.

[piranna]

I've checked it and the issues were related to some dependencies (my fault, sorry), now it should work flawlessly on QEmu. We should have some automated tests, ideally each night due to this issues on third-party packages... :-/

[piranna]

The code at AUFS branch builds flawlessly, at least for QEmu 32 bits. @groundwater, could you be able to check it again for the demo at JSConf?

[piranna]

NodeOS now cross-compile and run on QEmu x86 32 & 64 bits ^^ Code is on master, needs some fix-ups for launching and testing (they are currently focused only for 32 bits) but are minor ones :-)

[mitsukaki]

Only two more things to go!