Option to use requirejs path resolution logic #4

Open
jrburke opened this Issue May 25, 2012 · 19 comments

Comments

Projects
None yet
4 participants
@jrburke
Owner

jrburke commented May 25, 2012

This came up from a use case from @chrispitzer:

If you have modules coded to use amdefine and they work in reqiurejs, if they need to run in something like a busterjs test, or anything that has a command line command that just jumps right into node (cannot insert requirejs in the middle), it would be nice to be able to have the same config used in requirejs loading to work in that scenario, since the modules may not be in the node_modules area for node to find.

First idea: amdefine could read an environment variable that specifies a file that has the requirejs config, and it would use that for loading modules before handing off to node's loader if it cannot find it at that config.

@smt

This comment has been minimized.

Show comment
Hide comment
@smt

smt Jun 21, 2012

+1, I'm running into this very situation when running a Mocha test from command line.

smt commented Jun 21, 2012

+1, I'm running into this very situation when running a Mocha test from command line.

@jgrund

This comment has been minimized.

Show comment
Hide comment
@jgrund

jgrund Jul 20, 2012

+1, Having this will make amdefine a usable option for me.

jgrund commented Jul 20, 2012

+1, Having this will make amdefine a usable option for me.

@anodynos

This comment has been minimized.

Show comment
Hide comment
@anodynos

anodynos Sep 5, 2012

+1, I have stumpled upon this and wondered if I am missing something...

As I understand it, our true options for node+browser AMD modules right now is amdefine and the define [dep], (dep)-> suggested format to support baseUrl & paths. Otherwise we'll have to be using requirejs=require 'requirejs'; requirejs.config {baseUrl: 'my/app/path/'} and requirejs [dep], (dep)-> in each module, after we check we are on node or browser etc etc, which wont make them very portable and boilerplate free... Or am I missing something else ?

I am not so sure about the environment variable though... what if you have multiple apps, with different paths ? Perhaps some convention over configuration is needed - something like
'on first use, visit each subpath backwards and read [requireJSConfig.js].' or something...

anodynos commented Sep 5, 2012

+1, I have stumpled upon this and wondered if I am missing something...

As I understand it, our true options for node+browser AMD modules right now is amdefine and the define [dep], (dep)-> suggested format to support baseUrl & paths. Otherwise we'll have to be using requirejs=require 'requirejs'; requirejs.config {baseUrl: 'my/app/path/'} and requirejs [dep], (dep)-> in each module, after we check we are on node or browser etc etc, which wont make them very portable and boilerplate free... Or am I missing something else ?

I am not so sure about the environment variable though... what if you have multiple apps, with different paths ? Perhaps some convention over configuration is needed - something like
'on first use, visit each subpath backwards and read [requireJSConfig.js].' or something...

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Sep 6, 2012

Owner

@anodynos the other option is to use a UMD approach to detect node vs. AMD when writing a library.

When loading code for tests, you can use requirejs in node to kick off module loading.

I suggest that last path if you have multiple apps. I think the env variable is useful just for the cases where something, like test frameworks, want to be the top level script executed by node, vs. a script that you make to load requirejs then load the test framework.

Owner

jrburke commented Sep 6, 2012

@anodynos the other option is to use a UMD approach to detect node vs. AMD when writing a library.

When loading code for tests, you can use requirejs in node to kick off module loading.

I suggest that last path if you have multiple apps. I think the env variable is useful just for the cases where something, like test frameworks, want to be the top level script executed by node, vs. a script that you make to load requirejs then load the test framework.

@anodynos

This comment has been minimized.

Show comment
Hide comment
@anodynos

anodynos Sep 11, 2012

UMD could be a solution, but it has one great obstacle and a ugly caveat.

  • The caveat is that it looks really awful, its way TOO much boilerplate for anyone's real usage. People are cursing it, and I can see why. They will start calling it MUD. I see it being used only as an after step in some automated build that is added around the non-amd code to modularize and amdify the deployment (like this screencast), rather than being used to modularize code structure during development.

Coming from the java territory, I want to be able to write every logically different code chunk on a different file, without so much boilerplate. And trust me, I 've done my boilerplate with java :-)

  • The obstacle with is that when running on browser/AMD you can use 'package-relative' dependencies views/generic/PersonView', but when running on node you can only use file-relative../../../PersonView` which is awful. What does that last ../../PersonView point at? it blows my mind!

If you have to keep the two 'formats' in sync everytime you add a dependency, I am sorry, its not usable.

Using requirejs on node confuses me : where is define defined ? How can I define a module that is executed on node ?

In overall, how can I write once and without having any other concern, have it run seamlessly on both browser and node (which is what most people use anyway). It's such a shame that so much work has been done with AMD (& node), but they still can't really work together. See more on this issue

So, after realizing the void here, I am in the middle of writing a simple converter, where you write AMD modules and it translates them to UMD that run everywhere, taking care of the details.. I will update when v0.0.1 is done :-)

UMD could be a solution, but it has one great obstacle and a ugly caveat.

  • The caveat is that it looks really awful, its way TOO much boilerplate for anyone's real usage. People are cursing it, and I can see why. They will start calling it MUD. I see it being used only as an after step in some automated build that is added around the non-amd code to modularize and amdify the deployment (like this screencast), rather than being used to modularize code structure during development.

Coming from the java territory, I want to be able to write every logically different code chunk on a different file, without so much boilerplate. And trust me, I 've done my boilerplate with java :-)

  • The obstacle with is that when running on browser/AMD you can use 'package-relative' dependencies views/generic/PersonView', but when running on node you can only use file-relative../../../PersonView` which is awful. What does that last ../../PersonView point at? it blows my mind!

If you have to keep the two 'formats' in sync everytime you add a dependency, I am sorry, its not usable.

Using requirejs on node confuses me : where is define defined ? How can I define a module that is executed on node ?

In overall, how can I write once and without having any other concern, have it run seamlessly on both browser and node (which is what most people use anyway). It's such a shame that so much work has been done with AMD (& node), but they still can't really work together. See more on this issue

So, after realizing the void here, I am in the middle of writing a simple converter, where you write AMD modules and it translates them to UMD that run everywhere, taking care of the details.. I will update when v0.0.1 is done :-)

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Sep 11, 2012

Owner

A path that may be workable, which I have not tried because I did not want to hijack that lower level part of node, is to just replace the require.extensions entry for 'js' and have it auto-inject the define() definition for that module. I would not recommend that for a generic module that should be used in node, but for applications that want to use AMD, allow them to require('') this thing in their top-level script, and then amdefine's version of define() would be available for all modules loaded, even if they do not require('amdefine'). So for something like mocha tests, at the top of your test file if you do a require('auto-amdefine') (name TBD), it would do this require.extensions change in node.

I'll look into that, to see what it feels like, but will likely do it as a separate module/repo that uses amdefine as part of it.

@anodynos I appreciate the frustration though. I think this is just a transition period as javascript works out a good general module solution. Even when if it gets sorted, say in ECMAScript modules, I do expect some roughness around paths assumptions in different environments, but we'll see how it evolves.

Owner

jrburke commented Sep 11, 2012

A path that may be workable, which I have not tried because I did not want to hijack that lower level part of node, is to just replace the require.extensions entry for 'js' and have it auto-inject the define() definition for that module. I would not recommend that for a generic module that should be used in node, but for applications that want to use AMD, allow them to require('') this thing in their top-level script, and then amdefine's version of define() would be available for all modules loaded, even if they do not require('amdefine'). So for something like mocha tests, at the top of your test file if you do a require('auto-amdefine') (name TBD), it would do this require.extensions change in node.

I'll look into that, to see what it feels like, but will likely do it as a separate module/repo that uses amdefine as part of it.

@anodynos I appreciate the frustration though. I think this is just a transition period as javascript works out a good general module solution. Even when if it gets sorted, say in ECMAScript modules, I do expect some roughness around paths assumptions in different environments, but we'll see how it evolves.

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Sep 11, 2012

Owner

Thinking more about this, that require.extensions path is not really an improvement over just doing a require('requirejs') in that top level script, it will amount to the same effect.

So I think the trick is to work out examples to show to set up a module for mocha tests that does that require, sets up the requirejs config for that set of tests, then exports that requirejs require out of that setup module for use by the mocha tests.

That would result in all your modules just using define(). Then, amdefine is just for create a lib that stands alone in node, and that may inline other modules as part of that file, but if just a simple one module script, the UMD pattern may work better.

Owner

jrburke commented Sep 11, 2012

Thinking more about this, that require.extensions path is not really an improvement over just doing a require('requirejs') in that top level script, it will amount to the same effect.

So I think the trick is to work out examples to show to set up a module for mocha tests that does that require, sets up the requirejs config for that set of tests, then exports that requirejs require out of that setup module for use by the mocha tests.

That would result in all your modules just using define(). Then, amdefine is just for create a lib that stands alone in node, and that may inline other modules as part of that file, but if just a simple one module script, the UMD pattern may work better.

@anodynos

This comment has been minimized.

Show comment
Hide comment
@anodynos

anodynos Sep 11, 2012

What skips me though is how one can use both define [], -> and require [],-> on the same file on node.
Excuse my coffescript :-)

What skips me though is how one can use both define [], -> and require [],-> on the same file on node.
Excuse my coffescript :-)

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Sep 11, 2012

Owner

If it is code you control, it is app code, and not something you want to publish as an npm-installabled package for other people who may not be using requirejs/AMD, then loading requirejs then having it load the modules in node works out. If you are using coffeescript, you can use the coffeescript AMD loader plugin to first convert to JS, then have requirejs load it as a module.

However, if it will be a reusable piece of code that is npm-installable into possibly non-AMD projects, then amdefine or the UMD pattern is a better fit.

Owner

jrburke commented Sep 11, 2012

If it is code you control, it is app code, and not something you want to publish as an npm-installabled package for other people who may not be using requirejs/AMD, then loading requirejs then having it load the modules in node works out. If you are using coffeescript, you can use the coffeescript AMD loader plugin to first convert to JS, then have requirejs load it as a module.

However, if it will be a reusable piece of code that is npm-installable into possibly non-AMD projects, then amdefine or the UMD pattern is a better fit.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jul 30, 2013

@jrburke I'd like to return to @anodynos's question: where is define defined when using requirejs for Node, or IOW, how can I have both requirejs and define in node at once (same as in browser)? So that I can load modules from configured paths and at the same time have possibility to define them with id and having require catch them?

Once crazy idea seems to be include both and requirejs.config({nodeRequire:define.require});, is this how it should be done?

ghost commented Jul 30, 2013

@jrburke I'd like to return to @anodynos's question: where is define defined when using requirejs for Node, or IOW, how can I have both requirejs and define in node at once (same as in browser)? So that I can load modules from configured paths and at the same time have possibility to define them with id and having require catch them?

Once crazy idea seems to be include both and requirejs.config({nodeRequire:define.require});, is this how it should be done?

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Aug 2, 2013

Owner

@Herby: if the .js file is loaded via a top-level requirejs() call, then requirejs loads it and define is available to it. If the .js file is loaded with node's loader, then there is no define, and amdefine can be used for that case.

Once node' require() is used to load the file, I cannot inject any define() for that file, as the define() needs to be specific to the module being loaded. However, node does not give enough hooks to make this possible.

Owner

jrburke commented Aug 2, 2013

@Herby: if the .js file is loaded via a top-level requirejs() call, then requirejs loads it and define is available to it. If the .js file is loaded with node's loader, then there is no define, and amdefine can be used for that case.

Once node' require() is used to load the file, I cannot inject any define() for that file, as the define() needs to be specific to the module being loaded. However, node does not give enough hooks to make this possible.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 2, 2013

No, I want to be able to use it to embed amd modules, with specific id. But it seems that amdefine's and r.js's worlds are different. What I define using amdefine, that I cannot later load with requirejs. I want to be able to user requirejs to load external resources using its own logic, but also load internally defined ones. Unluckily, r.js does not provide any define and amdefine's one is not known by requirejs. I'd like to be able to connect these two worlds. Best solution would be if r.js would export define (maybe internally included amdefine, wrapped over it's caches or something).

James Burke wrote:

@Herby https://github.com/herby: if the .js file is loaded via a
top-level requirejs() call, then requirejs loads it and |define| is
available to it. If the .js file is loaded with node's loader, then
there is no define, and amdefine can be used for that case.

Once node' require() is used to load the file, I cannot inject any
define(
) for that file, as the define() needs to be specific to the
module being loaded. However, node does not give enough hooks to make
this possible.


Reply to this email directly or view it on GitHub
#4 (comment).

ghost commented Aug 2, 2013

No, I want to be able to use it to embed amd modules, with specific id. But it seems that amdefine's and r.js's worlds are different. What I define using amdefine, that I cannot later load with requirejs. I want to be able to user requirejs to load external resources using its own logic, but also load internally defined ones. Unluckily, r.js does not provide any define and amdefine's one is not known by requirejs. I'd like to be able to connect these two worlds. Best solution would be if r.js would export define (maybe internally included amdefine, wrapped over it's caches or something).

James Burke wrote:

@Herby https://github.com/herby: if the .js file is loaded via a
top-level requirejs() call, then requirejs loads it and |define| is
available to it. If the .js file is loaded with node's loader, then
there is no define, and amdefine can be used for that case.

Once node' require() is used to load the file, I cannot inject any
define(
) for that file, as the define() needs to be specific to the
module being loaded. However, node does not give enough hooks to make
this possible.


Reply to this email directly or view it on GitHub
#4 (comment).

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Aug 2, 2013

Owner

@Herby if amdefine is conditionally set to define mentioned in the usage docs, then I expect that to work out. If you are inlining amdefine in the file with your define()'d modules, and always calling it, I expect there will be problems.

Owner

jrburke commented Aug 2, 2013

@Herby if amdefine is conditionally set to define mentioned in the usage docs, then I expect that to work out. If you are inlining amdefine in the file with your define()'d modules, and always calling it, I expect there will be problems.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 3, 2013

I am afraid we're not understanding each other. The question is, how to
define module with known id (not putting in in file, doing it
programmatically) in conbination with r.js, using the same (r.js's)
require? That is, how can I define("foo", [...], ....) and later in
that same or different file be able to define(["f, ... or
requirejs(["foo", ...while still using other r.js modules, path
resolution, async loading etc., but also having ability to deifne foo?
It is not really an amdefine question, it is more a question of "where
is my define in r.js", but it is here, because r.js has no define and
amdefine claims to provide one... .but alas, incompatible.

So, again, shorter, the use case is to be able to
define("foo", { foo: "bar" })
in a file and later be able to use
requirejs(["foo", "with/resolved/path"], function (...) { ... })

But maybe I do not understand something... if a file is required by
r.js, it has define from somewhere. My use case is, to have def
ine in a file that does require("requirejs"), that is, to have define
also outside, not just inside. Similar to browser where I have define in
a page that has <script src=".../require.js"> (and so I can define
some modules inline). I only ended up using amdefine because r.js does
not have outsude define.

James Burke wrote:

@Herby https://github.com/herby if amdefine is conditionally set to
define mentioned in the usage docs
https://github.com/jrburke/amdefine#usage, then I expect that to
work out. If you are inlining amdefine in the file with your
define()'d modules, and always calling it, I expect there will be
problems.


Reply to this email directly or view it on GitHub
#4 (comment).

ghost commented Aug 3, 2013

I am afraid we're not understanding each other. The question is, how to
define module with known id (not putting in in file, doing it
programmatically) in conbination with r.js, using the same (r.js's)
require? That is, how can I define("foo", [...], ....) and later in
that same or different file be able to define(["f, ... or
requirejs(["foo", ...while still using other r.js modules, path
resolution, async loading etc., but also having ability to deifne foo?
It is not really an amdefine question, it is more a question of "where
is my define in r.js", but it is here, because r.js has no define and
amdefine claims to provide one... .but alas, incompatible.

So, again, shorter, the use case is to be able to
define("foo", { foo: "bar" })
in a file and later be able to use
requirejs(["foo", "with/resolved/path"], function (...) { ... })

But maybe I do not understand something... if a file is required by
r.js, it has define from somewhere. My use case is, to have def
ine in a file that does require("requirejs"), that is, to have define
also outside, not just inside. Similar to browser where I have define in
a page that has <script src=".../require.js"> (and so I can define
some modules inline). I only ended up using amdefine because r.js does
not have outsude define.

James Burke wrote:

@Herby https://github.com/herby if amdefine is conditionally set to
define mentioned in the usage docs
https://github.com/jrburke/amdefine#usage, then I expect that to
work out. If you are inlining amdefine in the file with your
define()'d modules, and always calling it, I expect there will be
problems.


Reply to this email directly or view it on GitHub
#4 (comment).

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Aug 13, 2013

Owner

Sorry I missed this in my earlier pass. I think I understand, trying a full code block, so that hopefully it reflects what I think the question is. Say I have main.js and it is run in node.js via node main.js, and main.js looks like this:

var requirejs = require('requirejs');

//Define a named module before starting other module loading
requirejs.define('foo', function(){});

//Later in main.js, this should work, used the named define above
requirejs(['foo'], function (foo){});
Owner

jrburke commented Aug 13, 2013

Sorry I missed this in my earlier pass. I think I understand, trying a full code block, so that hopefully it reflects what I think the question is. Say I have main.js and it is run in node.js via node main.js, and main.js looks like this:

var requirejs = require('requirejs');

//Define a named module before starting other module loading
requirejs.define('foo', function(){});

//Later in main.js, this should work, used the named define above
requirejs(['foo'], function (foo){});
@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 13, 2013

James Burke wrote:

Sorry I missed this in my earlier pass. I think I understand, trying a
full code block, so that hopefully it reflects what I think the
question is. Say I have main.js and it is run in node.js via |node
main.js|, and main.js looks like this:

var requirejs = require('requirejs');

//Define a named module before starting other module loading
requirejs.define('foo', function(){});

//Later in main.js, this should work, used the named define above
requirejs(['foo'], function (foo){});

Yes, except it should work in other files as well, if they are loaded by requirejs from main.js (directly or indeirectly), they should also recognize foo.


Reply to this email directly or view it on GitHub
#4 (comment).

ghost commented Aug 13, 2013

James Burke wrote:

Sorry I missed this in my earlier pass. I think I understand, trying a
full code block, so that hopefully it reflects what I think the
question is. Say I have main.js and it is run in node.js via |node
main.js|, and main.js looks like this:

var requirejs = require('requirejs');

//Define a named module before starting other module loading
requirejs.define('foo', function(){});

//Later in main.js, this should work, used the named define above
requirejs(['foo'], function (foo){});

Yes, except it should work in other files as well, if they are loaded by requirejs from main.js (directly or indeirectly), they should also recognize foo.


Reply to this email directly or view it on GitHub
#4 (comment).

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Aug 13, 2013

Owner

If main.js uses requirejs to load the file, then define() should be available to that other file, and 'foo' should be visible to it. So, if in the above main.js, it did requirejs(['something']) and then something.js was found via the requirejs config, then this would work:

//in something.js loaded by requirejs
define(['foo'], function(foo){});

However, what will not work is if 'something' was a node module that was loaded by a requirejs() call, but not found via requirejs config. In those cases, requirejs delegates to node's require() to load it. That is a fundamental limitation when working with Node. I have no plans to do some sort of deep injection hack into the Node module system to add things to their namespace as there is not guarantee that would work in the future if they make changes, or even disallow it if it worked today.

But if 'something' is loaded by requirejs and found via its config (so that it exposes define() and the set of modules it knows about) then the above should work.

Owner

jrburke commented Aug 13, 2013

If main.js uses requirejs to load the file, then define() should be available to that other file, and 'foo' should be visible to it. So, if in the above main.js, it did requirejs(['something']) and then something.js was found via the requirejs config, then this would work:

//in something.js loaded by requirejs
define(['foo'], function(foo){});

However, what will not work is if 'something' was a node module that was loaded by a requirejs() call, but not found via requirejs config. In those cases, requirejs delegates to node's require() to load it. That is a fundamental limitation when working with Node. I have no plans to do some sort of deep injection hack into the Node module system to add things to their namespace as there is not guarantee that would work in the future if they make changes, or even disallow it if it worked today.

But if 'something' is loaded by requirejs and found via its config (so that it exposes define() and the set of modules it knows about) then the above should work.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 13, 2013

Oh, this again looks like misunderstanding... of course I want to define foo in main.js. Just it should be requireable anywhere in the require tree, nor just in main.js. Which is what one awaits from define. Which is not present in main.js (such that requirejs honours it, which is not true in case of amdefine's one). Which is the only problem.

James Burke wrote:

If main.js uses requirejs to load the file, then define() should be
available to that other file, and 'foo' should be visible to it. So,
if in the above main.js, it did |requirejs(['something'])| and then
something.js was found via the requirejs config, then this would work:

//in something.js loaded by requirejs
define(['foo'], function(foo){});

However, what will not work is if 'something' was a node module that
was loaded by a requirejs() call, but not found via requirejs config.
In those cases, requirejs delegates to node's require() to load it.
That is a fundamental limitation when
working with Node. I have no
plans to do some sort of deep injection hack into the Node module
system to add things to their namespace as there is not guarantee that
would work in the future if they make changes, or even disallow it if
it worked today.

But if 'something' is loaded by requirejs and found via its config (so
that it exposes define() and the set of modules it knows about) then
the above should work.


Reply to this email directly or view it on GitHub
#4 (comment).

ghost commented Aug 13, 2013

Oh, this again looks like misunderstanding... of course I want to define foo in main.js. Just it should be requireable anywhere in the require tree, nor just in main.js. Which is what one awaits from define. Which is not present in main.js (such that requirejs honours it, which is not true in case of amdefine's one). Which is the only problem.

James Burke wrote:

If main.js uses requirejs to load the file, then define() should be
available to that other file, and 'foo' should be visible to it. So,
if in the above main.js, it did |requirejs(['something'])| and then
something.js was found via the requirejs config, then this would work:

//in something.js loaded by requirejs
define(['foo'], function(foo){});

However, what will not work is if 'something' was a node module that
was loaded by a requirejs() call, but not found via requirejs config.
In those cases, requirejs delegates to node's require() to load it.
That is a fundamental limitation when
working with Node. I have no
plans to do some sort of deep injection hack into the Node module
system to add things to their namespace as there is not guarantee that
would work in the future if they make changes, or even disallow it if
it worked today.

But if 'something' is loaded by requirejs and found via its config (so
that it exposes define() and the set of modules it knows about) then
the above should work.


Reply to this email directly or view it on GitHub
#4 (comment).

@jrburke

This comment has been minimized.

Show comment
Hide comment
@jrburke

jrburke Aug 13, 2013

Owner

Oh, this again looks like misunderstanding... of course I want to define foo in main.js. Just it should be requireable anywhere in the require tree, nor just in main.js. Which is what one awaits from define. Which is not present in main.js (such that requirejs honours it, which is not true in case of amdefine's one). Which is the only problem.

Sorry, yes I am getting confused. If you wanted to get it sorted out, then it will be best if you could construct a test that demonstrates the problem, what you would like to work, and then I could probably provide better feedback.

Owner

jrburke commented Aug 13, 2013

Oh, this again looks like misunderstanding... of course I want to define foo in main.js. Just it should be requireable anywhere in the require tree, nor just in main.js. Which is what one awaits from define. Which is not present in main.js (such that requirejs honours it, which is not true in case of amdefine's one). Which is the only problem.

Sorry, yes I am getting confused. If you wanted to get it sorted out, then it will be best if you could construct a test that demonstrates the problem, what you would like to work, and then I could probably provide better feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment