define() & paths relativity & resolution with requirejs on nodejs

[anodynos]

I am eager to write reusable modules, spanned in multiple files, that run & test on both the browser and nodejs.

According to the docs, I can use requirejs on node like this

  var requirejs = require("requirejs");
  requirejs.config({
    baseUrl: __dirname + "../my/deps/path",
    nodeRequire: require
  })

but then I can't understand how I can define modules that run on node.js, since define() is not available.

I tried to do a

define = requirejs.define;

but then with the call

define(['dep'], function(dep) {....mycode..; return myModule; })

nothing is happening, 'my code' never gets executed. Is this a bug ? Or is define() not available on node?

I also tried to use requirejs along with amdefine

if (typeof define !== 'function') {
  var define = require('amdefine')(module);
  var requirejs = require("requirejs");
  requirejs.config({
      baseUrl: __dirname + "/../../../build/code/deps",
    nodeRequire: require
  });
}

which seems to work, but is a bit of a frankestein: the amdefine version of define() doesn't respect the baseUrl set on requirejs. Specifically the amddefine define() delegates to node's require() and its resolution of paths that are relative to the file.js and not some 'baseUrl'. On top, node's require() requires ./ for files on the same directory, and there is no 'base', so even this doesn't work smoothly. How can we run code (eg mocha tests) with a reference to a base directory with requirejs based modules, on node ?

So far, my reusable modules work on the browser but not on node, unless I do all kinds of non-resuable tricks & tests with the path names.

Is umdjs format the only way to achieve true cross-executability ? But even then, the path relativity semantics are not the same....

[Devric]

I've trouble here too, I'm trying to use mocha to test in the command line.
but the define() is not defined unless i put require('amdefine')(module) within the start of each of my main js files.

But than that means i need to remove it after testing.
Looking for a way so it only need to define() once in the test file.

[anodynos]

What's even more confusing is that if you want to define your dependencies with the module-relative path (eg.
define(["my/module/path"], ....), it will work fine on the browser (with the baseUrl set to the containing directory) but will fail on node since a) there is no way with amdefine to set baseUrl and b) nodejs uses a file-relative path resolution, so you would need to use define([../../somepath],...) depending on where the requiring file resides in relation to the required file...

So you can't easily test or execute you scripts in the browser and node, unless you use ONLY the file-relative path resolution (starting with ./ for this directory, ../ to go back etc) which works the same for browser and node.

Alternativelly you use umdjs, pick a template and use module-relative on the browser and calculate the path resolution manually for each dependency on the node side. The downside is too much boilerplate, and optimization with r.js wont work. I am actually thinking of drafting a utility that will do the 'conversion' automatically :-) Hope its not an overkill!

[jrburke]

Some background: If you load requirejs via `require('requirejs') as done on the requirejs-node doc page, then you would not need amdefine in that script. amdefine is an option to using a UMD pattern in a particularly library module.

Ideally you would just use the require('requirejs') approach and load modules that way. The trick is that for test frameworks, they like to be run via a top-level node file they define, where the requirejs path wants to be in that top level file.

An option could be to figure out how to load mocha via a require() call in a node program, and then drive mocha via JS instead of a command line call.

It is all a bit tricky because node does not understand define() and getting the right pathing behavior is tricky too. There is an amdefine ticket asking for it to use requirejs loading logic when it can, but I'm on the fence about it. I think it will just blur things even further.

[anodynos]

@jrburke, thank you for your response.

I believe having to tweak mocha or any other framework just to make it work with AMD, completely breaks the reusability and modularity concerns that modules come to solve. Using modules should be as unobstrusive as possible, like using an import in other languages. In current state, this is not the case.

For this reason, after having tried all possible valid solutions (I hope), I have drafted out a simple source converter that I believe will be useful and I would like to hear your thoughts.
I know you are against build tools as a mandatory requirement for AMD, but still 'building' tools (grunt, make, ant) are inevitable for serious development. And most of all, I dont really care what the automated tools will do, but only the code I have to write (and later look at). Even still, my converter will be an optional step :-)

The main aim is to write once (with AMD only syntax) and run on both browser & node, using some UMD template, without any other hassle.

The core usage pattern will be:

• Use only the standard AMD syntax define(['dep1', 'dep2'], function(dep1, dep2) {}).
  Inside your module, if you want to conditionally load myhugeOptionalModule you again use the anyschronous AMD require syntax only require(['dep1', 'dep2'], function(dep1, dep2) {}).
• Use 'package' relative notation for dependencies `mydepdir1/mydepdir2/myModule' which is more natural (esp for us x-java devs). Optionally you will still be able to use the 'file-relative' (if it starts with ./ or ../)
• The converter extracts the dependencies & parameters of your code, and rewrites you body (factory method) around a UMD template, injecting require as the first param. When running on node, this injected require will be aware of the relative path of your file inside your 'package' and will be resolving/loading your dependencies (asynchronously, to match browser behaviour) before calling your factory method.
• And finally, yes, node's require will be locally hidden from you - you can only use AMDs require format, which is effectivelly a wrapper for nodes' native require.

This way, your code is guaranteed(!) to run everywhere, but written consistently with the exact same authoring semantics (provided async requires are also used on the node side), and testers like mocha wouldn't even know the difference. Think of it as the oposite of browserify: it takes AMD-browser modules and wraps them around UMD so they can run on node.

Its a trivial thought really, but can help many people quickly use AMD over UMD without even bothering with it. Ultimatelly I hope it wil raise AMD adoption :-)

My first tests seem to work and I should have a first alpha-preview version hopefully by the end of this week for you to reflect on, but I'd appreciate you first thoughts on it.

[jrburke]

@anodynos see this comment in the amdefine issue: I think a converter that taps into node's require.extensions system may be the way to go. But it will be difficult, just on the path resolution front to bridge both node and AMD conventions.

[anodynos]

I haven't looked at require.extensions, but I will if they deem useful. So far I don't really need and haven't looked at any low-level node specific stuff.

What I have in the works is just a proof of concept of a generic AMD-to-UMD converter that allows AMD written code (using both define and 'require`) to be executed without any (manual) modification on node. Since it will be using templates, you can provide different templates that will wrap your code around require('requirejs') or require('amdefine') or something else entirely.

Thus, it can be incorporated in your usual chain of build (eg. grunt) and then be run, deployed and tested on both browser and node (as long as its not using any dom or node's specific stuff).

[jrburke]

Converters are difficult, in that not all the same concepts are in both envs, in particular the require([]) capability and loader plugin resource IDs in AMD. For simpler modules it is possible though.

[anodynos]

If you write against the require([]) convention only, then I believe it should be ok. Most business logic code is fairly non-low level, 'simpler modules' as you put it. Developers should be able to run and share code on both environments without hassle. That's my aim anyway, a quick converter/generator that takes the yak out of UMD, but embraces it under the hood :-)

[anodynos]

As promised, I have released an early version of uRequire - https://github.com/anodynos/uRequire

uRequire converts AMD/commonjs modules to the UMD format. It allows you to write against a strict AMD or a more 'relaxed' notation.

Any feedback is welcome.