New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filesystem Layout #19

Closed
groundwater opened this Issue Feb 21, 2014 · 31 comments

Comments

Projects
None yet
7 participants
@groundwater
Copy link
Contributor

groundwater commented Feb 21, 2014

I should explain the filesystem layout of node-os a little bit, because it's a slight depart form what you'll find on Ubuntu, Redhat, Arch, etc.

I think requiring root privileges in order to install a package is lame. I've ditched the concept of global or system packages altogether, the only global executable is node located at /bin/node. The directories /etc, /lib, and /usr contain a few files necessary for node to run. I would love to reduce the number of extra files laying about, but one step at a time.

Any time you run npgk install, the new module will be downloaded into $HOME/lib/node_modules, and executables will be linked into $HOME/bin. Every user experiences a unique system, and every user has access to npkg. You do not need root privileges to install packages.

For example, if there are two users on the system bob and kim, and bob runs npkg install ncurl, the filesystem will look roughly like:

/home/
  bob/
    bin/ncurl --> ../lib/node_modules/ncurl/ncurl.js
    lib/node_modules/ncurl
  kim/
    bin/
    lib/node_modules/

root and init

When the kernel is ready, it will start init as PID 1. In node-os, init is located in /root/bin/init. Init is a module like any other executable. It is available on npm, and can be updated with npkg install. Modules in /root come pre-installed with node-os. They are the necessary modules to boot the system into a usable state.

An example set of default modules might be:

@groundwater groundwater added the blog label Feb 21, 2014

@EndangeredMassa

This comment has been minimized.

Copy link

EndangeredMassa commented Feb 21, 2014

I'd actually like to see something where the directory structure is grouped more by package than by type of file. An example would look like:

/home
  /bob
    /npkg  # or whatever a good name might be
      /ncurl
        /bin
        /lib # normal contents of a node module
        /config
          /environment.json

This would require a more complex PATH environment variable, but that could easily be managed by npkg. This approach keeps everything related to a single package in a single directory.

We could even hide to other directories that normal operations won't care about, like Gobo Linux.

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Feb 21, 2014

Right now, installing to $HOME/lib/node-modules and $HOME/bin follows the npm conventions around npm install -g. You can actually do npm install -g --prefix=$HOME to get the same result as npkg install. I'd be pretty resistant to changing this.

Modules are still completely contained in $HOME/lib/node_modules but symlinks to executables are placed in $HOME/bin, so each users path must include $HOME/bin.

@EndangeredMassa

This comment has been minimized.

Copy link

EndangeredMassa commented Feb 21, 2014

Deviating from npm here isn't very attractive, sure. This was just a "perfect world" scenario.

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Feb 21, 2014

Yah understood. I like that I'm getting the chance to discuss the many design decisions in detail.

@juliangruber

This comment has been minimized.

Copy link
Member

juliangruber commented Feb 23, 2014

Having completely separate paths for module would be more tidied up but until a real issue arises from not having that I don't see a reason to diverge from npm

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Feb 24, 2014

Modules already have separate paths. They are 100% contained in $HOME/lib/node_modules. Anything placed in $HOME/bin is a symlink.

@distransient

This comment has been minimized.

Copy link

distransient commented Feb 27, 2014

Aren't there big security problems with letting any user or group install packages?

@juliangruber

This comment has been minimized.

Copy link
Member

juliangruber commented Feb 27, 2014

Not if rights management is correctly implemented, i.e. a user can only write her files and nothing else. The worst thing that could happen is a user deletes all her files or installs daemons she doesn't want running.

@distransient

This comment has been minimized.

Copy link

distransient commented Feb 27, 2014

How would permissions for packages be handled?
On Feb 27, 2014 12:42 AM, "Julian Gruber" notifications@github.com wrote:

Not if rights management is correctly implemented, i.e. a user can only
write her files and nothing else. The worst thing that could happen is a
user deletes all her files or installs daemons she doesn't want running.

Reply to this email directly or view it on GitHubhttps://github.com//issues/19#issuecomment-36221179
.

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Feb 27, 2014

@mkgh great questions, let me see if I can clear some things up.

Think of each user much like you'd think of another npm module. All its dependencies are in local, so in a users case, everything is contained in their home directory.

If you install a command line tool like curl for example, only you will have see curl in your $PATH. Since it's installing into a folder you own, you have full write access.

As for services, currently init runs as root, and it would be insecure to let non-root users modify init jobs. In a proper multi-user environment, init can be secured either with http basic auth, or by binding to a unix socket that only root can access.

Each user on the system will end up with their own init daemon, from which they can start/stop jobs. This is basically how launchd on osx works.

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 10, 2014

Is not more NPM oriented to use $HOME/node_modules/.bin/ and $HOME/node_modules/ instead, specially if each user home is considered as a node package?

@naturalethic

This comment has been minimized.

Copy link

naturalethic commented Jun 10, 2014

Frankly I’ve always wished node modules would be installed in ~/.npm or pwd/.npm — I hate seeing that directory and I hate underscores with a passion.

On Jun 10, 2014, at 2:22 PM, Jesús Leganés Combarro notifications@github.com wrote:

Is not more NPM oriented to use $HOME/node_modules/.bin/ and $HOME/node_modules/ instead, specially if each user home is considered as a node package?


Reply to this email directly or view it on GitHub.

@distransient

This comment has been minimized.

Copy link

distransient commented Jun 10, 2014

Don't forget that Node's require system was made to work package manager
agnostic (node_modules comes from Node.js, not npm)

.node would make more sense

Michael Keating
https://github.com/mkgh
On Jun 10, 2014 2:30 PM, "Joshua Kifer" notifications@github.com wrote:

Frankly I’ve always wished node modules would be installed in ~/.npm or
pwd/.npm — I hate seeing that directory and I hate underscores with a
passion.

On Jun 10, 2014, at 2:22 PM, Jesús Leganés Combarro <
notifications@github.com> wrote:

Is not more NPM oriented to use $HOME/node_modules/.bin/ and
$HOME/node_modules/ instead, specially if each user home is considered as a
node package?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#19 (comment).

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Jun 13, 2014

The choice of node_modules is ugly, but we're stuck with it for now.

I use the default layout that npm i -g uses, which places modules in $ROOT/lib/node_modules, and links executables into $ROOT/bin. Yes, that means you have a bin and lib visible in your home, but we probably ultimately want this.

Having node_modules in $HOME is dangerous, because then any node modules can accidentally resolve a dependency out of there e.g.

$HOME/
  node_modules/
    lib_a/
    lib_b/
  some_project/
    package.json
    node_modules

The some_project is able to resolve lib_a and lib_b which we don't want.

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Jun 17, 2014

I started this page on the wiki https://github.com/NodeOS/NodeOS/wiki/FileSystem

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 17, 2014

I see in the wiki you are not adding /sbin, that already is empty and I was thinking it could be removed. I agree on that. On the other hand, you said /root comes prefiled with some basic services, would then /root/bin be doing its role? Also, why not /home/root or plainly remove root account? In other words: if all apps, libraries and services are in a per-user basis, do we still need a root user? Or could be mitigate its necesity? Maybe a wheel admins group is just enough?

In the same way of thinking, maybe / and the other private system directories could be hidden removing the -r permission, in a way of "your root is your home folder, you don't have nothing of your interest to see here". This could also apply with removing the -r permisson by default on the home folders, leaving to the user (and maybe also its installed programs?) the impression that in the computer "there's no place like $HOME" (pun intended :-P). You could say "what about /dev if I want to access to my CD-ROM?". Ok, valid argument. In this aspect, I would go to implement and make use of W3C DeviceStorage API and other similar ones for similar issues. I think this way we can give a really secure environment and OS by not letting anybody to go out of this sandbox that we are building on their $HOME folder. Also, this way we can get some interest in the proponents of open web standards :-) and also increase portability of the applications to other platforms.

On a side note, I have been thinking about some performance improvements we can achieve working this way.

First of all, moving all the commands and utilities to user folders would left us with a minimal / directory that mostly only have the kernel and the node.js binary. This will be used heavily, and probably will not be modified until a new version of one of the two gets published, so lets put them directly on a read-only ramdisk image and load in on memory on startup. This way we get a system really fast, and on hard-disk there's only a single compresed file (the ramdisk image) that can be move easily, and there's no more cryptic files flying around.

The second idea is about /tmp being on ram, too. This way we get a high performance, SSD disks don't get stressful unnecesarily, and also all the temporal data gets removed when the machine is switch-off, being more secure. I know @groundwater proposed that tmp lives in each user $HOME, but with correct permissions, they can all live in the same tmp folder without security implications. Also, if we create the "API sandbox" I said previously, this /tmp folder will be inacesible for regular users, so their only option would be to use libraries to create temporal files and folders.

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 17, 2014

Crazy & radical idea: let's go down all the road with the "your $home is you filesystem" thing! Let's isolate each user in their own chroot! What could be safer than that?! :-D

Each time I think about this, the most I like it: it's the best way to give the impression to the users that they have the computer only for themselves, something like NodeOS offer to the user an empty filesystem where to build the OS of their dreams, but in reality there would be thousand of other users with their "dream OSes" on the same machine, maybe running at the same time, and nobody else knows about them :-)

@distransient

This comment has been minimized.

Copy link

distransient commented Jun 17, 2014

Chroot can often be broken out of in a number of circumstances, though. It
would be a false sense of security that'd lead to bigger security flaws.

Michael Keating
https://github.com/mkgh
On Jun 17, 2014 1:04 PM, "Jesús Leganés Combarro" notifications@github.com
wrote:

Crazy & radical idea: let's go down all the road with the "your $home is
you filesystem" thing! Let's isolate each user in their own chroot! What
could be safer than that?! :-D

Each time I think about this, the most I like it: it's the best way to
give the impression to the users that they have the computer only for
themselves, something like NodeOS offer to the user an empty filesystem
where to build the OS of their dreams, but in reality there would be
thousand of other users with their "dream OSes" on the same machine, maybe
running at the same time, and nobody else knows about them :-)


Reply to this email directly or view it on GitHub
#19 (comment).

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 17, 2014

Are there other alternatives that could give the same functionality I intended to achieve (except virtual machines...)?

@distransient

This comment has been minimized.

Copy link

distransient commented Jun 17, 2014

You could still do a chroot user system, just do it on top of the
already-secure Unix user system and file permissions. However, it sounds
like something meant for user space only, not system space. Of course, a
utility to manage user chroots wouldn't be such a bad idea!

Michael Keating
https://github.com/mkgh
On Jun 17, 2014 1:10 PM, "Jesús Leganés Combarro" notifications@github.com
wrote:

Are there other alternatives that could give the same functionality I
intended to achieve (except virtual machines...)?


Reply to this email directly or view it on GitHub
#19 (comment).

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 17, 2014

It was mostly to hide the user any filesystem outside their $home, we can go for now to have a strict filesystem permissions policy and revisit this later :-)

@distransient

This comment has been minimized.

Copy link

distransient commented Jun 17, 2014

Just think, what good is hiding non-$HOME fs from users when the
directories there are useful? They don't suddenly disappear and the chroot
is only an illusion of protecting them. /etc, /sys, and many other
directories are very useful on the system-level for user utility.

Michael Keating
https://github.com/mkgh
On Jun 17, 2014 1:18 PM, "Jesús Leganés Combarro" notifications@github.com
wrote:

It was mostly to hide the user any filesystem outside their $home, we can
go for now to have a strict filesystem permissions policy and revisit this
later :-)


Reply to this email directly or view it on GitHub
#19 (comment).

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Jun 18, 2014

if all apps, libraries and services are in a per-user basis, do we still need a root user?

Yes, you need a user to boot the system. Classically this is done by root, but the programs and configurations are not stored in the root directory. The root node-os user boots the system, and everything necessary to accomplish that is contained in /root.

you said /root comes prefiled with some basic services, would then /root/bin be doing its role? Also, why not /home/root

Root's home directory could be moved into /home, but it's special enough that I don't really mind it being at /root. The /root directory may also be mounted in a unique way, because it's used immediately after the kernel is loaded. It may be that /home is mounted by a root service at startup.

Crazy & radical idea: let's go down all the road with the "your $home is you filesystem" thing! Let's isolate each user in their own chroot! What could be safer than that?! :-D

I think eventually we will want high isolation, but I think it's difficult to see what that may look like from here. I think true isolation will come from using containers.

The only point to sharing users on a system is to enable them to cooperate. Perhaps by shared access to directories etc. If we really want to 100% isolate users, we should give them their own container.

Chroot can often be broken out of in a number of circumstances

Also chroot is pretty much useless because of dynamic libraries. ELF binaries are almost certainly dynamically linked against libc. If your app is jailed when it's loaded, it will not be able to find the libc shared object. The real solution for isolation is unique containers.

lets put them directly on a read-only ramdisk image and load in on memory on startup. This way we get a system really fast, and on hard-disk there's only a single compresed file (the ramdisk image) that can be move easily, and there's no more cryptic files flying around.

I definitely like this idea, but it's also a pure optimization. I don't want to discard this idea, but it's going to get queued behind designing the user-facing product first.

This is a great discussion btw, thanks!

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 19, 2014

@mkgh, the idea behind hide non-$HOME directories comes from two points.

First of all, "global" binaries and libraries are going to be installed inside $HOME, there's no necesity to "go out", so users and apps could be cheated making them to believe that in fact the root of the filesystem is at $HOME, giving they the impression that they have all the computer/filesystem to themselves.

The second one, this directories and files that would be left outside $HOME are really minimal (mostly the kernel and the node binary, no more), somewhat special, for advance users only "who knows what they are doing", and OS dependent, so if there's a real necesity for them they could be virtualized using some APIs (the only thing I really think would be needed by a user to move out of $HOME is some shared folder, and this would be offered using DeviceStorage API or FileSystem API or something similar), leading to a safe sandbox in each user $HOME.

@groundwater, didn't know about Linux Containers, only of chroot, thanks! :-D Definitelly it's the way to go to isolate the users ant their servieces between them :-)

Regarding root user, my question was most related with the fact that if it's only used to init the system and later all the programs will be run with unpriviledges users and root user will be useless after that, why having a specific user for that with its $HOME and all the things? More than root user, it should be called boot user... :-P So that's why I ask about it: is it still necesary? Would a root user needed after booting to do administrative tasks, or users could manage themselves? If there are need for god-like administrative tasks, is not enough with sudo?

@groundwater

This comment has been minimized.

Copy link
Contributor

groundwater commented Jun 20, 2014

More than root user, it should be called boot user...

What we call it arbitrary because a user name is not a concept inherent to the kernel, but to libc. The kernel just knows about user numbers, and root has uid 0. This is important because uid 0 is allowed to do anything. Convention calls this user root, so we probably want to keep calling it that for now.

Would a root user needed after booting to do administrative tasks

In general you shouldn't need to be doing things as root, but root privileges are necessary to accomplish various system tasks, or to bind to port 80 etc.

The sudo command is just root in disguise. The sudo process has it's suid bit set, meaning it runs as root, even if an unprivileged user executes it.

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 20, 2014

In general you shouldn't need to be doing things as root, but root privileges are necessary to accomplish various system tasks, or to bind to port 80 etc.

So the explicit user could be removed... :-)

@jbenet

This comment has been minimized.

Copy link

jbenet commented Jun 20, 2014

In general you shouldn't need to be doing things as root, but root privileges are necessary to accomplish various system tasks, or to bind to port 80 etc.

So the explicit user could be removed... :-)

Technically yes. It may be more trouble than it's worth. Perhaps a suitable middle ground would be to differentiate between kernel users (privilege identities) and userland users (people's accounts).

Fighting linux's privilege model will probably be too much work and be bug-ridden-- at some point, using a better privilege system altogether (say, capabilities) becomes less, safer work.

@piranna piranna added this to the 0.1.0 milestone Jun 28, 2014

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jun 29, 2014

As suggested on #37 (comment), we could use a statically compiled Node.js binary so we could remove dependency of compiled libraries, having the filesystem more clean. Also, kernel-generated folders like /dev (devfs), /proc (procfs) and /sys (sysfs) are only there to show kernel internal structures so they can be desactivated, and probably /usr/bin/env could also hacked, so the root filesystem would only show the ramdisk image, the root user directory (where all boot and system tools are installed) and the users homes, that could also be directly on the root folder since there's no system files at all (only the just ones inside /root).

For a first release, we can try to have a filesystem similar to this:

/bin/node
/etc \
/usr  | stuff for libc
/lib /
/dev  (devfs)
/proc (procfs)
/sys  (sysfs)
/root/
  lib/node_modules/
    mini-init/
    asgard/
    nodeos-boot/
      node_modules/
        getty/
        dhcp-client/
  bin/
    asgard
    mini-init

but with this changes, we can try to have a filesystem like this:

/nodeos.img.gz
/root/
  bin/
    asgard
    mini-init
  lib/node_modules/
    asgard/
    mini-init/
    nodeos-boot/
      node_modules/
        dhcp-client/
        getty/
/piranna/
  bin/
    shareit
  lib/node_modules/
    shareit/
  Movies/
  Music/
  Pictures/
@piranna

This comment has been minimized.

Copy link
Member

piranna commented Jul 28, 2014

I've found some info about initramfs and rootfs and seems the way to go. This allow both to have a initramfs.gz file to give as parameter to the kernel, but also to add it on compile time to the kernel itself, and it works as the OS root filesystem by default. This would allow us to distribute just only one file ready to boot (the kernel + rootfs one) and totally isolate what's on NodeOS "OS" layer from what's on "user's space" layer (pun intended ;-) ).

@piranna

This comment has been minimized.

Copy link
Member

piranna commented Sep 11, 2014

After doing the port to QEmu, I've check there are some subtle differences on the filesystem. For example, Docker mount the ROOT filesystem, but also /proc and /sys, but no /dev (use a static, hardcoded one). On QEmu I'm doing just the oposite, but also /root is created automagically.

I'm thinking to use on QEmu AUFS (the same as Docker) for the readonly initramfs, and mount /proc and /sys to make them similar (although I don't like the idea to have more processes running...), and also use /root to have the basic, boot-time only things, so we could have:

  • layer2: REPL, just kernel and initramfs
  • layer3: REPL with basic init functionality (maybe up to getty?). kernel, initramfs and ROOT filesystem with /root
  • layer4: kernel, initramfs, ROOT filesystem with /root and USERS filesystem, user's services and shell according to each user (no REPL)
@piranna

This comment has been minimized.

Copy link
Member

piranna commented Apr 7, 2015

Each user has now it's own root filesystem thanks to OverlayFS that can modify at its will, so this can be clossed.

@piranna piranna closed this Apr 7, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment