Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRFC 19 - Let's Stop Installing Packages #19

Open
jbenet opened this issue Jun 19, 2014 · 23 comments

Comments

@jbenet
Copy link
Owner

commented Jun 19, 2014

Today's paradigm includes installing software. It's really silly, having to go find a particular package, and then download it manually. Our package managers should just make the code available. If it can be found in the registry, it should be importable in the code.

How?

Easy, mount the registry:

/npm/<module-name>/<version>

Or, in my world:

/ipns/npmjs.org/<module-name>@<version>

Concerns

What about Security? is this safe!?

Security is not about installing software X at time Y, but about checking integrity (hash the code) and authenticity (sign the code). This could be done on import, every single time you run the code, which would be much safer than just hoping all your files are the same as when you last looked at them. You did look at all the modules you imported, right? You are sure that foobar module you used doesn't actually open a back door, right?

But what about production? Will I have to randomly download modules?

Not at all, things will be cached locally, and just make sure things stay local, why not pin them?

> mounted-npm pin module-i-care-about

Basically, make your "mounted registry" save things locally that you're going to use regularly. (IPFS will do this for you).

But, maybe I want to version lock?

And you should! Lock your local files to exactly the modules you want:

> echo bar@1.3 >> modules-i-want
> echo foo@1.0 >> modules-i-want
> cat modules-i-want | mounted-npm lock
> mounted-npm ls
bar@1.3
foo@1.0
@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

@groundwater you'll want this in NodeOS, i think :) -- give me two weeks and IPFS may be stable enough for it.

@groundwater

This comment has been minimized.

Copy link

commented Jun 19, 2014

This is kind of like a lazy install of modules.

I'm also interested in what things we can dream up when file-systems and mounts are easy and safe to hack on. Kinda like FUSE, but without the constant segfaults 😄

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

Kinda like FUSE, but without the constant segfaults

FUSE gives you segfaults!? I've yet to see this-- perhaps I'll run into it shortly. Is this osx or linux?

@groundwater

This comment has been minimized.

Copy link

commented Jun 19, 2014

It gives me segfaults because I'm causing them. Basically I'm a shitty FUSE developer =]

@mlovci

This comment has been minimized.

Copy link

commented Jun 19, 2014

http://xkcd.com/1367/

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

@mlovci precisely. The divide between web and fs is really stupid and it's time to end it. We don't do http requests on ever file open, and with good reason. Need a better protocol :)

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

@groundwater try fuse4js? shouldn't be able to segfault :)

@groundwater

This comment has been minimized.

Copy link

commented Jun 19, 2014

Retrieving and caching content is pretty straight forward. How would you deal with directory listing?

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

@groundwater I wouldn't. Don't list root (-r+x on dir). Do list under keywords/, author/<username>/, etc.

@groundwater

This comment has been minimized.

Copy link

commented Jun 19, 2014

Would you use FUSE?

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2014

More like, i will use FUSE ;)

@groundwater

This comment has been minimized.

Copy link

commented Jun 19, 2014

Too bad we can't just patch the fs module.

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 24, 2014

For @mikolalysenko:

Below, it could easily be foo@version or foo/version. I'm opting for foo@version because i like it more. You can also keep the node_modules/ but i took it out below. Con

/ipns/npmjs.org/package/foo@1.0.0
/ipns/npmjs.org/package/foo@1.0.1
/ipns/npmjs.org/package/bar@1.0.0 
/ipns/npmjs.org/package/bar@1.0.0/foo@1.0.0  (same object as .../foo@1.0.0 above)
/ipns/npmjs.org/package/baz@1.0.0 
/ipns/npmjs.org/package/baz@1.0.0/foo@1.0.1 (same as .../foo@1.0.1)
/ipns/npmjs.org/package/bork@1.0.0/bar@1.0.0 (same as .../bar@1.0.0)
/ipns/npmjs.org/package/bork@1.0.0/baz@1.0.0 (same as .../baz@1.0.0)
@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 24, 2014

@mafintosh

This comment has been minimized.

Copy link

commented Jun 24, 2014

@jbenet wouldn't I need to list the /ipns/npmjs.org/package (which can be HUGE) to match a semver using the above approach?

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 24, 2014

@mafintosh where match here means resolve something like >=1.2.3 ? Ah, then maybe we do want name/version in top level:

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/foo/1.0.1
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0  # only needs to list /ipns/npmjs.org/package/foo/
/ipns/npmjs.org/package/baz/1.0.0/foo@^1.0.1
@mafintosh

This comment has been minimized.

Copy link

commented Jun 24, 2014

Yes this makes it a lot easier :) I would probably also not include sub dependencies. I don't really see the benefit of having them.

@jbenet

This comment has been minimized.

Copy link
Owner Author

commented Jun 24, 2014

@mafintosh we get subdependencies for free.

/ipns/npmjs.org/package/foo/1.0.0
/ipns/npmjs.org/package/bar/1.0.0                         
/ipns/npmjs.org/package/bar/1.0.0/foo@>=1.0.0
/ipns/npmjs.org/package/bork/1.0.0/bar@>=1.0.0  
# \o this resolves to bar/1.0.0 above, which already has foo@>=1.0.0

content-addressed deduplication :) !

@mafintosh

This comment has been minimized.

Copy link

commented Jun 25, 2014

Nice!!

@jonathanmarvens

This comment has been minimized.

Copy link

commented Jun 30, 2014

Random stranger stopping by to say that these are awesome thoughts :) .

@jbenet

- Jonathan

@silky

This comment has been minimized.

Copy link

commented Jul 11, 2014

check the nix package managed and nixos - http://nixos.org/ - https://nixos.org/nix/

@jbenet jbenet changed the title Let's Stop Installing Packages JRFC 19: Let's Stop Installing Packages Apr 17, 2015

@jbenet jbenet changed the title JRFC 19: Let's Stop Installing Packages JRFC 19 - Let's Stop Installing Packages Apr 17, 2015

@BillDStrong

This comment has been minimized.

Copy link

commented Jun 4, 2016

Why would this only be at that layer?

If you want to go that route, create a Linux distro that mounts IPFS early enough in the chain, then symlink all the normal filenames to IPNS addresses that always have the latest released version. Instant always up to date system.

@burdges

This comment has been minimized.

Copy link

commented Jun 4, 2016

You need reproducible builds to do this securely, along with Guix or NixOS tool for keeping package options straight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.