Building Toasters

[sbuller]

As a potential user, I'm finding the focus of the documentation and tooling a bit different than what I'd hope for. I think my aims are still in line with the direction of the project though.

My hope is to have a tool—let's call it toast for arguments sake—that builds a bootable disk image when run in a node repository (i.e. a directory containing a package.json file). I don't particularly care if it's built from source—in fact, I'd probably rather that toast would download a pre-built image. The downloaded image should be combined with the contents of the repository such that the resultant image can be booted by qemu, and the js file pointed to by "main" in the package.json file should then be started automatically.

I specifically do not care about users, or any other direct form of interaction or administration. My goal in terms of administration is that when I want to change things, I modify the source code in my package, re-run toast generating a new image, and upload it—entirely replacing the old image. In view of this, I would recommend a read-only filesystem.

Beyond this, some facility or API for mounting additional disks would be nice, but if that proves insurmountable then I would rather store my state elsewhere on the network rather than store my state alongside my application. Perhaps arguments passed on the command line to toast could be stored in the image as well, and be referenced by the running script, allowing an application to be more easily instanced.

Example

main.js

var http = require('http');

var server = http.createServer(function (request, response) {
  response.writeHead(200, {"Content-Type": "text/plain"});
  response.end("Hello World\n");
});

server.listen(process.argv[2]);

package.json

{
  "main": "index.js"
}

Command Line

$ toast -o hello.img --ip=192.168.0.10/24 --args="80"
$ qemu -net ... hello.img

... I hope I'm not talking like a crazy person here.

[piranna]

... I hope I'm not talking like a crazy person here.

Not more than most of us here (or at least me :-P ). You have shown a really interesting use case that was between the initial ones when @groundwater started the project and also between mines when I got a ride, only that it hasn't got enough traction to work toward that idea :-) At this moment, NodeOS architecture is designed so you can custom it by forking the project and changuing the dependencies of the layers to the ones you want, in your case it would a custom nodeos-initramfs that instead of executing nodeos-mount-filesystems it would exec your app, and remove nodeos-rootfs and nodeos-usersfs ones. This way you could get a pure initram-based and diskless system. Problem is that your app would be running as root... that's not fine. A better alternative would be instead use a custom nodeos-usersfs with just a non-root user that exec your app, but this would require a disk partition and it's not yet documented, but has the advantage of execute it as regular user and allow data persistence. Your idea of a builder gor the initram is not bad and can be really useful, but nothing is done towards this. You could be able to create a toaster tool in an independent npm module that generate an initramfs and then use the just generated one to build your custom copy of NodeOS. In that way, it would be ideal to first split nodeos-mount-filesystems to an independent module with the basic system config, and later exec your app with restricted permission. How do you think about doing it? :-)

[sbuller]

I kind of want to do it, but I've got other stuff to do too. And right now I just need to get something working. Anyways, I don't see any problem with running as root, since there isn't going to be anything else on the system anyway. No point trying to mess around with permissions—the only thing they could do is restrict access to low number ports, which needs to be done on the firewall anyway. The only hardware that would be connected to the system would be hardware that my program would need. Making a user account just doesn't make sense.

I might try poking around a bit, but I can't commit a lot of time to this.

[piranna]

Then in your case the quick path is to fork nodeos-initramfs and replace
there the nodeos-mount-filesystems dependency by your app and change the
creation of the /sbin/init symlink to point to your app. After that, just
the tipical dance of npm install & npm run build & npm start. Could you be
able to do a pull-request on nodeos-initramfs with the steps that you've
followed to customize it?

And just curious... are you planning to use NodeOS on production? Where?
Can be able to tell it in a "projects and companies using NodeOS" section?
:-)
El 9/7/2016 5:03, "sbuller" notifications@github.com escribió:

I kind of want to do it, but I've got other stuff to do too. And right now
I just need to get something working. Anyways, I don't see any problem with
running as root, since there isn't going to be anything else on the system
anyway. No point trying to mess around with permissions—the only thing they
could do is restrict access to low number ports, which needs to be done on
the firewall anyway. The only hardware that would be connected to the
system would be hardware that my program would need. Making a user account
just doesn't make sense.

I might try poking around a bit, but I can't commit a lot of time to this.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#273 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAgfvoscz8vFaKfLvLHZYwcfQMH-Jj6-ks5qTw-dgaJpZM4JIbq3
.

[sbuller]

Well, it's a bit early to say if this will be put into production, but I'm looking to use it for running some internal automation where I work.

Since you already make pre-built images for the project, would it be possible to have barebones provided on its own? It looks like that will be all that I need to make me some toast.

[piranna]

Well, it's a bit early to say if this will be put into production, but
I'm looking to use it for running some internal automation where I work.

I would be really happy to hear that ;-)

Since you already make pre-built images for the project, would it be
possible to have barebones provided on its own? It looks like that will be
all that I need to make me some toast.

Yes, each layer is independent, so you can be able to use nodeos-barebones
standalone, although I recomend you to do your changes on nodeos-initramfa.

[piranna]

Hi @sbuller, did you get any chance to take a look on this, or was nodeos-barebones useful to you? If you have any question don't doubt to ask.

[sbuller]

I'm still poking around. I feel like I'm getting close to having something that does what I need.

[piranna]

I'm still poking around. I feel like I'm getting close to having something that does what I need.

I have described a possible solution at NodeOS/nodeos-initramfs#4, tell me if we have some points in common here.

[piranna]

@sbuller, did you get any progress on this?

[sbuller]

I've gotten a bit sidetracked. I put together nos-init to generate initramfs files in the manner I envision. I've played a bunch with syslinux and other bootloaders, as well as related filesystems. I can't say that it's truly worthwhile, but I had been focused on generating a bootable disk image without depending on tools external to NodeJS/NPM. I settled on syslinux with fat16 is the most realistic option. It took me a while to come to the realization that there was no practical way to produce the output entirely with streams. I've got a bunch of spaghetti code at this point that puts together a lot of this but it's super hairy, and it's still missing some important parts. The stupid thing is that it's probably just a couple lines of bash script using mkisofs. I'd just rather make program that could run everywhere. That's one of the main things I like about NodeJS.

Anyhow, it's still pinging around my brain at the moment, but hasn't seen any work for a week or two.

[piranna]

I've gotten a bit sidetracked. I put together nos-init to generate initramfs files in the manner I envision.

That's awesome! :-D A generator of nodeos-initramfs custom images, Very neat... :-) We could spand it later to make modules run inside an unpriviledged environment, but that's definitely a good starting point, good job! ;-)

I've played a bunch with syslinux and other bootloaders, as well as related filesystems. I can't say that it's truly worthwhile, but I had been focused on generating a bootable disk image without depending on tools external to NodeJS/NPM.

That would be cool if it helps to make NodeOS build environment more portable, although by far I have been seeing that the most portable language at least for low-level and platform independent components is bash. Look, it now has official suport on Windows 10, too... :-P

I settled on syslinux with fat16 is the most realistic option.

Is that the reason why you are doing the require() magic on your code? To evade problems with symlinks on FAT16?

A bootable NodeOS image on a FAT pen drive booting directly to a secure environment, that's directly the USB drive root directory, that would be interesting... OverlayFS would have problems with that, but maybe we could be able to override them with FUSE or an upgraded version of UMSDOS... Hum... :-)

[sbuller]

My intent is only to have the initramfs and barebones on the fat16 image. My use case is only for qemu. That said, I do try to keep things modular and general where I can. I can't remember my reasons for abandoning symlinking, but it's not because of fat16. The initramfs is still a cpio which populates a tmpfs.