Avoiding 404s

[KidkArolis]

Have you considered an optional server side component that could be started in the background that would provide the info about where all the modules are located which would avoid all of the 404s?

Alternatively, it could be a metadata file that's generated after npm install and placed at the root somewhere.

It kind of goes against rave's goal to remove machinery, but it could be optional.

[unscriptable]

Have you considered an optional server side component

The server-side topic comes up often. It's been very interesting to see how much we can do without it, but at some point we may collectively decide that we need a server-side helper. As employees of Pivotal's Frameworks and Runtimes group, we'd have to ensure that we have a solution for a Spring-based server-side helper, too.

it could be a metadata file that's generated after npm install

I like this line of thinking. I've been thinking similarly, but via a rave cli. A cli could do a lot of things like this. I'd insist that the use of the rave cli be optional and that the dev could just use their favorite tools (npm + gulp or bower + grunt, for examples). The rave/debug module would help detect errors that devs might cause when not using the cli.

[KidkArolis]

I wonder if npm would accept a patch that creates a node_modules/metadata file..

[unscriptable]

Interesting. Are you saying that npm would create some sort of app manifest file??? My experience with the npm folks tells me that they would not accept this kind of patch. :(

Also, to be clear, I have been thinking that a rave cli would collect metadata in the bower.json and/or package.json files, not a new metadata file or app manifest. I'm open to awesome ideas, though. :)

[KidkArolis]

Well, I imagined node_modules/.meta could contain the state of the node_modules every time npm modifies node_modules. By state I mean something like a list of what package have been installed in which location, etc. package.json's alone don't tell you where the packages are actually installed in (since npm sometimes moves them up the node_modules hierarchy). So in addition to reading package.json's, rave-cli, for example, would have to also travese the node_modules of the parent dirs until it finds the right location of each module (I thought that's what rave.js does now, which is why it's 404ing, or am I wrong?)

Of course, such file could easily get out of sync since there might be other tools that touch node_modules, etc. Which is why I also think npm folks wouldn't be in favour of adding such a file with info that can be easily derived.

rave-cli could generate such a file for sure. or a rave serve could provide an endpoint with the list of the locations of all packages..

[unscriptable]

I thought that's what rave.js does now, which is why it's 404ing, or am I wrong?

The 404s are caused by two things, atm, neither of which is caused by the node_modules heirarchy:

1. rave/auto tries to find both package.json and bower.json at the top level by default. This 404 can be eliminated via the data-rave-meta html attribute.
2. Bower allows package authors to register packages without bower.json. rave looks for package.json when bower.json is missing.

As it turns out, (2) is problematic since package.json typically points at node-specific dependencies. jspm looks for a "repository" field to decide whether the dependencies are found on npm or bower. The bower folks are also considering if/when to make bower.json mandatory in registered packages.

I'm seriously thinking about removing that "feature".

[KidkArolis]

1. that sounds reasonable. What about loading package.json if rave is installed via npm and using bower.json if rave is installed via bower :)
2. I would vote for removing the "feature"

[KidkArolis]

An example of where npm hierarchy 404s kick can be reproduced by installing

npm install react rave-node-process

Loading rave then throws the following 404
GET http://localhost:8900/node_modules/envify/node_modules/jstransform/node_modules/esprima-fb/package.json 404 (Not Found), because npm puts esprima-fb one level up.

And I assume that loading envify would fail for the same reason (atm this is not the best example, since loading envify fails due to missing node builtin stream and some other reasons).

But my main point is that the node_modules hierarchy will definitely become an issue (especially after calling npm dedupe?)

[KidkArolis]

Ah! I see you're actually looking up the node_modules hierarchy here https://github.com/RaveJS/rave/blob/master/lib/createVersionedIdTransform.js#L21-L23. Neat. Ignore my last message then.

[KidkArolis]

Well, actually, trying to fetch package.jsons for modules that live higher up in the node_modules is one of the reasons 404s are caused. And possibly it's something that could be slightly improved.

[unscriptable]

I agree. this could probably be improved

[unscriptable]

all of the current bower issues could be solved if .bower.json were accessible from the browser. here are some options to fix this:

1. urge the bower authors to stop hiding the generated metadata file.
2. tell bower users to generate a bower metadata file. see version attribute should always be required bower/spec#4 (comment)
3. force bower users to use a rave CLI that could generate or fix meta data.
4. create a web server that doesn't hide .bower.json from the browser.

I like option 1, but I'm not sure if we'll be successful. I know at least two of the bower committers will be open to the idea.