NodeOS auto-updater

[mitsukaki]

I thought this might be a cool module to create. I wouldn't suggest we include it with NodeOS simply to remain simplistic but it could be installed by users. Optimally, the module would simply check for updates on the startup of a NodeOS session. This would be nice for users using a bare-metal installation. The installer would of course run within NodeOS.

I have a few questions however. Is this feasable without using Administrator Mode? Could the updates be cached and then implemented on a system restart? Wouldn't this require the module to be installed while in Administrator Mode?

I like the idea of this. What do y'all think?

[luii]

I kinda like the idea!
On system startup it just should check if newer version are available and notifiy the user but it should not install on its own!
It should install the updates when the system shuts down, but before shutting down it should ask if you want to install updates

But we should seperate system updates and package updates because the system updates like newer kernel and such things should only be installed on admin mode.
And package updates from npm should just be installed (but only when they're not system wide packages)

[piranna]

The more I think about it, the more I think it's a fantastic idea :-D Obviously this would not be on the core of NodeOS, but it would be nice to provide a module that users would install if they want.

I'm used to use the npm-check-updates module and works like a charm :-D so we could be able to use it to be able to check and update all the globally installed modules, and users could be able to exec it on their /init script. I think it would download the modules updated tarballk on the cache and later do the update from them. Obviously this would only be done for globally installed ones, since the locally ones can be installed anywhere :-P I'm not sure if the upgrade should be done on shutdown like Windows does or on boot by the same script, this would be easier since we don't need to update NodeOS to allow to exec scripts on shutdown, that also makes sense.

Global modules would be check and updated too, and would be done automatically on boot too, but this would need a careful thinking since this updates would run as root and so they could be an attack vector if a module gets an update with malware inside. It's an interesting point, but don't think of too much interest since scripts running as root should be minimal (mostly mounting the users partition).

Regarding the upgrade of the kernel... well, it could be done too!!! :-P I have been thinking if rootfs should be mount as /boot and upgrade the kernel version would be a valid use case, so who knows! :-D

[mitsukaki]

This is interesting discussion! Let's continue. As school comes to a close I will finally be able to continue contributing, and as I now have a few users who've opted in to beta test NodeOS-mobile the auto-updater seems more than reasonable.

On system startup it just should check if newer version are available and notifiy the user but it should not install on its own!
It should install the updates when the system shuts down, but before shutting down it should ask if you want to install updates

Interesting suggestion! With this autoupdater, I'm curious whether we ought to go the simplistic as possible route and set a bunch of defaults, or try and make everything configurable. Finding a nice inbetween is something we should pay a lot of attention to seeing that a lot of the people using it won't be advanced users.

But we should seperate system updates and package updates because the system updates like newer kernel and such things should only be installed on admin mode.

Yeah that makes sense. This leaves me with a question. Users can boot into administrator mode, but if an application, like the updater needs it, could it request a reboot into administrator mode? If a kernel update was available, the updater could simply reboot into administrator on shutdown, update the kernel, then turn off the device. (or boot into admin, update, the restart in normal mode if turning on the device and updating)

If we were to attempt to add this feature, it's important that users are querried first before restarting or that could definetely be a feature leveraged by malware.

I'm used to use the npm-check-updates module and works like a charm :-D so we could be able to use it to be able to check and update all the globally installed modules, and users could be able to exec it on their /init script.

Ah okay this sounds excellent! If we were to begin right away, we could build off this module to create the module updater. However, I do have a counter point. To voice simplicity, ought we include a module updater? Technically speaking, anyone wanting to update their modules on device should be using that module or something similar and all our updater would truly need to do is update NodeOS system related modules. What do you think?

Global modules would be check and updated too, and would be done automatically on boot too, but this would need a careful thinking since this updates would run as root and so they could be an attack vector if a module gets an update with malware inside.

Yes, auto updates are often a bad idea. I suggest we prompt users first, but this could definetely be a configurable feature. If a module got loaded while in administrator mode, who know's what could happen! (we all know).

Regarding the upgrade of the kernel... well, it could be done too!!! :-P I have been thinking if rootfs should be mount as /boot and upgrade the kernel version would be a valid use case, so who knows! :-D

This is an exciting idea! I do agree that the feature makes sense. Whether it is, or isn't practical to most users, that I don't know.

[piranna]

In this way, we could also configure nightly tasks to check external dependencies like new versions of the Linux kernel or of Node.js so new packages could be created and published automatically, and the auto-updater could be able to fetch them and have an up-to-date system without having any human harmed or involved in the process :-)

[mitsukaki]

human harmed or involved

haha, well said! Sounds like an excellent idea! Let's start at PR for it.

[mitsukaki]

And maybe we ought to create a thread too? We could just use the PR thread?

[piranna]

Maybe better an independent project, not inside NodeOS core. Remember, modularity and reusability ;-) This could be used on regular Node.js installations, too :-)

[piranna]

[piranna@Latitude:~/Proyectos/prebuild]
 (prerelease) > ncu -g -u

ncu cannot upgrade global packages. Run npm install -g [package] to update a global package

Seems it only last the last mile... :-) Two options, to do a pull-request on npm-check-updates to add that functionality (that since they only update the package.json file and don't do the updates themselves probably it would be discarded), or create a new project using it as dependency that does the real updates.

[mitsukaki]

@piranna I was referring to creating a PR for mounting rootfs as boot :P

[piranna]

Oh, I though you were talking about the auto-updater. They are two independent and unrelated things, and in fact mounting rootfs as /boot don't have too much mistery and it's just a matter of a single line of code, that's why I was not thinking about it... I will accept your pull-request when you have it ready, but we should focus on the auto-updater tool since it's needed first to have it working :-)

By the way, we have open other issues for 1.0 version like the nsh update (my fault...) and the git client, how are they going?

[mitsukaki]

By the way, we have open other issues for 1.0 version like the nsh update (my fault...) and the git client, how are they going?

Git has made no progress from my understanding. I'm contemplating taking it on in the summer.
Nsh update :P no idea!

And yes I'm working on the auto-updater as we speak on 'noUpdate' repo on my page. I know what needs to be done for mounting but if you have a free hand right now, we could save manpower if you went ahead and did it right now.

(I'd have to go hunt which file needs changes because I forget which script does the mounting...)

[piranna]

(I'd have to go hunt which file needs changes because I forget which script does the mounting...)

nodeos-mount-filesystems repo. It should be done only for root user, if you want I can be able to do this myself and you work on the noUpdate project.

[mitsukaki]

Alright, I've got a prototype of the updater on it's repo noUpdate. What I have so far, is it reads it's config (I'm not sure where you'd like that to go. Where did we put the PalmTree configuration again?) and updates all the packages listed in it.

It first runs ncu on the package.json, then runs an npm install to update it.

Possible adjustments could be having it scan and update everything and instead use a blacklist of what NOT to update. Any other suggestions??

P.S. yes I'm aware that Object.values() isn't available in Node... I have to go find a substitute.

[piranna]

I'm not sure where you'd like that to go. Where did we put the PalmTree configuration again?

/etc directory, as always.

Possible adjustments could be having it scan and update everything and instead use a blacklist of what NOT to update. Any other suggestions??

Far better. I would check if we are inside a project (there's a package.json file on this folder or previous ones, npm does it this way so there's a module for it but don't remember the name, maybe ncu does it itself) and update its dependencies, or if not update the global ones. To fetch the list of modules I would use the npm module to fetch just the first level of global dependencies (or one of its dependencies if the functionality is isolated) and apply ncu on them.

P.S. yes I'm aware that Object.values() isn't available in Node... I have to go find a substitute.

Why not iterate over the object itself? It's standard...

[mitsukaki]

/etc directory, as always.

Gotcha

Far better. I would check if we are inside a project (there's a package.json file on this folder or previous ones, npm does it this way so there's a module for it but don't remember the name, maybe ncu does it itself) and update its dependencies, or if not update the global ones. To fetch the list of modules I would use the npm module to fetch just the first level of global dependencies (or one of its dependencies if the functionality is isolated) and apply ncu on them.

Okay so you're suggesting to see if we're already in a package and if not, proceed to updating all the globals. Yeah that makes sense, okay. We could also add a parameter to specify which modules to update.

Why not iterate over the object itself? It's standard...

I wanted to see if there was an elegant one-line solution similar to Object.values() but there isn't so yeah I'll just iterate.

[piranna]

We could also add a parameter to specify which modules to update.

Yes, but don't obsess with that, focus first on the default functionality without parameters to upgrade the globally installed modules, that's what will be used more.

Oh, and please use the npm module (or the specific dependency) instead of enpeem to do the install.

[mitsukaki]

Oh, and please use the npm module (or the specific dependency) instead of enpeem to do the install.

I actually reverted to using the command-line because that's what npm said to do. (The api is apparently only to be used internally and will not remain stable)

[piranna]

Could you be able to point me where it says so? I think it's a bad design,
specially if they are splitting npm in a lot of micromodules...
El 18/6/2016 8:10, "Lite McFish" notifications@github.com escribió:

Oh, and please use the npm module (or the specific dependency) instead of
enpeem to do the install.

I actually reverted to using the command-line because that's what npm said
to do. (The api is apparently only to be used internally and will not
remain stable)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#247 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAgfvtxfzzACAdyiW6AKJ3nHnHoZIQ8Jks5qM4vLgaJpZM4Ivhnp
.

[mitsukaki]

@piranna here is one place where it's quoted from the doc. I could probably go find it in the docs but I'm about to go to sleep 😛

http://stackoverflow.com/questions/15957529/can-i-install-a-npm-package-from-javascript-running-in-node-js/15957574#15957574

[piranna]

Sh*t... Problem is NodeOS don't have npm installed by default (and
specially to root user), so we could not be able to use the command, we
need to set it programatically and download as dependency... :-/
El 19/6/2016 7:57, "Lite McFish" notifications@github.com escribió:

@piranna https://github.com/piranna here is one place where it's quoted
from the doc. I could probably go find it in the docs but I'm about to go
to sleep 😛

http://stackoverflow.com/questions/15957529/can-i-install-a-npm-package-from-javascript-running-in-node-js/15957574#15957574

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#247 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAgfvv17-uXW1A_k4IbvMfAY1J-oTutaks5qNNplgaJpZM4Ivhnp
.

[mitsukaki]

@piranna. Oh, no problem. I'll rip the install command right out of npm then and grab all it's dependencies.

[mitsukaki]

@piranna easier said than done. I'm trying to see if I can mimic npm's install command.

[piranna]

Seems npm update does almost all we need here... I think we should think
calmly what exact features and behaviour we want it to have and what use
cases it should fix.
El 20/6/2016 0:10, "Lite McFish" notifications@github.com escribió:

@piranna https://github.com/piranna easier said than done. I'm trying
to see if I can mimic npm's install command.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#247 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAgfvvvI2wiR-NgBuOdYD2i062X0bUPOks5qNb5GgaJpZM4Ivhnp
.

[mitsukaki]

@piranna well from my understanding, the goal of noUpdater is to create an auto-updater for npm packages. Originally I had intended for it to be used to update the base NodeOS system so we would have to keep fetching new releases from git, but have the system automatically update itself (just as Android updates itself).

That's why originally there was a list of programs to update; this way only required packages would be updated and nothing else. Auto-updating packages from other things on the system can be dangerous if the application breaks because of an incompatible update.

What do you think? I'll be on gitter.

[piranna]

I think all can be done with 'npm update -g', so what extra functionality
would it do? Ask to the user? Download at shutdown?

I think we have here four main use cases:

1. Users own global modules
2. Root own global modules
3. System core (kernel + initram)
4. Users local modules

1 & 2 are somewhat the same and feasible with 'npm update -g', and the same
happens with 4, only that from the docs I'm not sure if it update the
package.json file. Also, we should consider too if updates should be only
on the same minor or could change of major version or if this should be
configurable...

3 is more tricky since it's not on a npm standard place, but also it could
be done with 'npm update' by using a postinstall script. And there's also
the problem of the updater size with all its dependencies... I think we
should do a list of this features to discuss them. Maybe in the noUpdate
repo?

[mitsukaki]

Alright sounds good. I'll close this here, and let's open a new thread for each feature so they can be discussed in depth.