Search Paths do not work with RequireJS

[jasonswearingen]

EDIT: summary of issue (20120430)

okay, i got a repro with the Karma+Requirejs example.

basically it comes down to a problem with having the app.js and appSpec.js in the same dir, and having that NOT be the baseUrl.

you can get the repro code here: https://bitbucket.org/novaleaf-apps/karma-requirejs/overview

fyi my version also includes a test for having the code in a subdir (also not baseUrl) which is also broken, but the reason for that seems to be fixed by @kjbekkelund 's workaround (i don't include the workaround in my version though)

to be clear there seems to be two issues here.

1. the test url path issue @kjbekkelund solves with his workaround,
2. the broken search paths for tests in the same folder as their dependencies

ORIGINAL ISSUE BELOW

summary

loading a dependency with relative paths results in script error due to the file being loaded without the proper ".js" extension.

repro steps

modify line 1 in karma requirejs example, appSpec.js from:

define(['app', 'jquery', 'underscore'], function(App, $, _) {

to:

define(['../src/app', 'jquery', 'underscore'], function(App, $, _) {

results

the app will still run successfully, but running the Karma tests will result in the following error:

Chrome 26.0 (Windows) ERROR
Uncaught Error: Script error
http://requirejs.org/docs/errors.html#scripterror
at http://localhost:9877/adapter/lib/require.js?1366192555000:1746
Chrome 26.0 (Windows): Executed 0 of 0 ERROR (0.67 secs / 0 secs)

workaround for the issue:

add the following to my test-runner (shim the define() method) to truncate relative paths before sending to requirejs

function createDefineShim(originalDefine) {
    return function (deps, callback) {
        for(var i = 0; i < deps.length; i++) {
            deps[i] = deps[i].substr(deps[i].lastIndexOf("/") + 1);
        }
        originalDefine(deps, callback);
    };
}
define = createDefineShim(define);

other things i tried that don't work

edit the (requirejs monkeypatch adapter.js)[https://github.com/karma-runner/karma-requirejs/blob/master/src/adapter.js] by adding the following line between lines 5 and 6 (add this as the first line of the normalizePath() method)

path += (/\.js/.test(path) ? '' : '.js'); //halfway works, but breaks dependency mapping

this will add the .js extension to any file that does not already have it.

this didn't work because....

because it seems that via Karma, requirejs doesn't understand that "app" is the same as "../src/app" ** please note that executing the app normally, requirejs is able to map the dependencies correctly **

root cause

I'm not sure what the root cause is.
as for why this issue occurs in the karma and not in requirejs itself, i see some wacky execution flow in (require.js)[https://github.com/jrburke/requirejs/blob/master/require.js] online 1555, which is probably contributing to the issue, however as i'm a javascript beginner i haven't figured out why it works ok in requirejs itself. (sorry, i spent a day tracking this down)

why this bugfix is important

I am using typescript, which automatically generates the AMD define() call that requirejs uses, and it ALWAYS includes the relative path. I can not control the generated code.

[kimjoar]

Require.js appears to have a problem with mixing absolute and relative requires. This change in test-main.js should fix it:

var tests = Object.keys(window.__karma__.files).filter(function (file) {
    return /Spec\.js$/.test(file);
}).map(function(file) {
    // create relative path from `baseUrl` for specs, without `.js`
    // i.e., instead of requiring `/base/test/appSpec.js`
    // we want to require `../test/appSpec` when the
    // baseUrl is `/base/src`
    return '../' + file.replace(/^\/base\//, '').replace(/\.js$/, '');
});

But this really isn't nice and it has to be changed in every project according to the where the test and production files are placed. I'll see if I find better ways of handling this.

[jasonswearingen]

hello, fyi I tested your modification to test-main.js in other of my projects and it works, thank you for the help, a better solution than my half-assed workaround :)

[jasonswearingen]

oops, sorry your solution has a problem,

references break if you add a requirejs.config based shim dependency. (my workaround works in this circumstance)

for example, if you take (the karma+require example)[https://github.com/kjbekkelund/karma-requirejs] and add dependencies to "app" in the test-main.js requirejs.config call:

requirejs.config({
    // Karma serves files from '/base'
    baseUrl: '/base/src',
    paths: {
        'jquery': '../lib/jquery',
        'underscore': '../lib/underscore',
        //"app":"../src/app",
    },
    shim: {
        'underscore': {
            exports: '_'
        },
        'app': {
            deps: ['jquery', 'underscore'],
        }
    },
    // ask Require.js to load these files (all our tests)
    deps: tests,
    // start test run, once Require.js is done
    callback: window.__karma__.start
});

then change the define line in appSpec.js to

define(['../src/app'], function (App) {

then jquery and underscore should be globally loaded prior to appSpec.

again, this is a common workflow for typescript, as it's easier to have frameworks as global because the language currently only allows explicit imports for code files.

[kimjoar]

I really don't like overriding define in Require.js, as hacking libraries always come back to bite me at some point. I guess the problem is that Require.js has some problems when the dependencies are specified relatively, absolutely and based on "module name".

This works:

var tests = Object.keys(window.__karma__.files).filter(function (file) {
    return /Spec\.js$/.test(file);
}).map(function(file) {
    // create relative path from `baseUrl` for specs, without `.js`
    // i.e., instead of requiring `/base/test/appSpec.js`
    // we want to require `../test/appSpec`
    return '../' + file.replace(/^\/base\//, '').replace(/\.js$/, '');
});

requirejs.config({
    // Karma serves files from '/base'
    baseUrl: '/base/src',

    paths: {
        'jquery': '../lib/jquery',
        'underscore': '../lib/underscore',
    },

    shim: {
        'underscore': {
            exports: '_'
        },
        '../src/app': {
          'deps': ['jquery', 'underscore']
        }
    },

    // ask Require.js to load these files (all our tests)
    deps: tests,

    // start test run, once Require.js is done
    callback: window.__karma__.start
});

And:

define(['../src/app'], function(App) {

I.e. use the same string to the shim and the require. But, this feels quite nasty and hacky. You should probably create a TypeScript issue on this (or a Require.js issue for that matter).

[jasonswearingen]

unfortunatly, the issue doesn't exist without Karma in the picture. And
while I don't know what the actual issue is, I did enough investigation to
know that Require.js is pretty hacky regarding dependency paths. I'll
file an issue with requirejs and reference this thread.

• Jason

[jasonswearingen]

also, from what I see, the "../src/app" named module would break if we have
another dependency in a different folder.

so it seems my define shim workaround is still the way to go.... FYI i
only need to do this in my test-runner (no shim in my real app) because
this issue only exists when using Karma

[vojtajina]

@jasons-novaleaf can you go to http://localhost:9876/debug.html and see what is the wrong path that require is trying to fetch ? That should tell us what the issue is.

I can only think about two things that are different when running with Karma:

• normalizing the path (eg. 'a/../b' -> 'b') https://github.com/karma-runner/karma-requirejs/blob/master/src/adapter.js#L27
• using basePath

I don't think the normalization should cause any troubles, but you can try to comment it out.

It's more likely caused because of the baseUrl...

[jasonswearingen]

if my tests and referenced dependencies are not in the same folder as my basePath, then my work-around does not work.

So I am trying to investigate more: it appears that karma's "base" path breaks things

here's an example:

my directory structure

• app/
  • egghead.js
  • egghead-spec.js

egghead-spec.js has a dependency on egghead.js,
i set up my requireConfig baseUrl as "", and I get the error:

GET http://localhost:9876/app/egghead-spec.js 404 (Not Found)

• missing the "base" so can't load my test as the entry point

i set up my requireConfig baseUrl as "base", and I get the error:

GET http://localhost:9876/base/egghead.js 404 (Not Found)

• loaded my test ok, but now can't figure out that egghead.js is in the same folder as the test

it looks like whatever karma does to setup the url destroy's requirejs's ability to figure out paths. sorry i can't help beyond this.

FYI i have no problem setting this up with normal requirejs and a browser.if you want code to reproduce this problem on, a good place would be the Karma+requirejs example, as described in my original post.

[kimjoar]

@jasons-novaleaf That's interesting. How do you include egghead-spec.js? Is it included as app/egghead-spec or as /base/app/egghead-spec.js? I've had some problems with Require.js when using the latter version, especially when depending on another file in the same folder. Currently the karma+require.js example includes using the entire path, so relative requires fails. I've fixed it using something like this (which isn't very nice, I should find a better solution):

var tests = Object.keys(window.__karma__.files).filter(function (file) {
    return /Spec\.js$/.test(file);
}).map(function(file) {
    // create relative path from `baseUrl` for specs, without `.js`
    // i.e., instead of requiring `/base/test/appSpec.js`
    // we want to require `../test/appSpec`
    return '../' + file.replace(/^\/base\//, '').replace(/\.js$/, '');
});

Does that work for you, or does it create other problems?

[jasonswearingen]

for testing purposes i explicitly set the tests in the require dependencies

i tried various combinations:

• require.deps =["/app/egghead-spec.js"];
• require.deps =["app/egghead-spec"];
• require.deps =["egghead-spec"];
• etc etc etc...

the ones where it's able to see egghead-spec properly fail because it's looking for "egghead.js" in the base folder, not in the location it actually is. (base/app/), as i described in my previous post

[nuweb]

I have the same problem with Karma and RequireJS.

Versions using:
Karma version: 0.8.5
Node version: 0.8.19

Git Ref:
https://github.com/nuweb/angular-home.git

Karma config ref:
https://github.com/nuweb/angular-home/blob/master/karma.conf.js

RequireJS, main.js ref:
https://github.com/nuweb/angular-home/blob/master/src/js/main.js

The app will run successfully, but running the Karma tests will result in the following error:
karma start karma.conf.js
INFO [karma]: Karma server started at http://localhost:9876/
INFO [launcher]: Starting browser ChromeCanary
INFO [Chrome 28.0 (Mac)]: Connected on socket id yoLzyPllTDVLc5aubUH6
Chrome 28.0 (Mac) ERROR
Uncaught Error: Script error
http://requirejs.org/docs/errors.html#scripterror
at http://localhost:9876/adapter/lib/require.js?1365839185000:1746
Chrome 28.0 (Mac): Executed 0 of 0 ERROR (0.101 secs / 0 secs)

If you look at the above screenshot, running the test runner will download angular.js twice, one reference is from karma.conf.js and the another reference specified in main.js.

The reference from main.js is incorrectly loaded (gives 404) cause the test runner is trying to load it relative to the root, while it is relative to the main.js file.

[jasonswearingen]

i tried reproing this bug with the karma+require example project: https://github.com/karma-runner/karma-requirejs/blob/master/ by setting the base path to "/base" instead of "/base/src",

but i could not repro the bug (it seems to work)

the main difference between my app and this is that I pre-set the require.config before loading require. I'll try modifying my app to fit the same workflow and write back on the results.

[jasonswearingen]

okay, i got a repro with the Karma+Requirejs example.

basically it comes down to a problem with having the app.js and appSpec.js in the same dir, and having that NOT be the baseUrl.

you can get the repro code here: https://bitbucket.org/novaleaf-apps/karma-requirejs/overview

fyi my version also includes a test for having the code in a subdir (also not baseUrl) which is also broken, but the reason for that seems to be fixed by @kjbekkelund 's workaround (i don't include the workaround in my version though)

to be clear there seems to be two issues here.

1. the test url path issue @kjbekkelund solves with his workaround,
2. the broken search paths for tests in the same folder as their dependencies

[nuweb]

I setup test-main.js for karma, but couldn't get karma load the angularjs files.
https://github.com/nuweb/angular-home/blob/master/test/test-main.js

[nuweb]

Ok, finally I was able to get this working after specifying the shim for dependencies.
https://github.com/nuweb/angular-home/blob/master/test/test-main.js

[vojtajina]

@nuweb don't use shim for loading all of your code, rather define baseUrl properly. Why are you bootstrapping angular in main-test.js ? That makes no sense to me. Check out the requirejs example, your project seems to be a typical one...

@jasons-novaleaf @kjbekkelund I will look into the project you have on bitbucket...

[vojtajina]

@jasons-novaleaf @kjbekkelund I looked into https://bitbucket.org/novaleaf-apps/karma-requirejs/overview:

• if you wanna reference stuff from src/ without the src prefix, you need to set baseUrl: '/base/src.
• the you need to set the shim properly, eg. jquery to /base/lib/jquery or ../lib/jquery.
• when you reference stuff with ../ require does not append .js, so in the subDirAppSpec.js, you need to require either ../app.js, or just app

Note, that I'm not requirejs user, but I don't see anything wrong on this. I'm happy to make changes to karma or requirejs adapter in order to make this more simple and straightforward, however I don't know how.

[vojtajina]

Basically the changes I described seem to be correct fixes for the project.

Please re-open this issue, if you think there is something we can improve...

[ddlsmurf]

I ran into the same issue. Requirejs doesn't need the extension, even for relative paths, perhaps in the specs, but from experience it doesn't bother it. I am sure because sometimes I let requirejs load the coffee files and sometimes the js files depending on context (loading into node or browser), so specifying extensions would prevent either of working.
Here is the code I added to test-main.js to get it to work for me:

window.requirejs.load = do (previous_load = window.requirejs.load) ->
  (context, moduleName, url) ->
    url += ".js" unless /\.js$/.test(url)
    previous_load.call(@, context, moduleName, url)

[trevithj]

I have just spent some time getting requirejs tests running under karma. While I'm glad I took the time (solves a lot of dev problems for me) I would like to suggest a change in the documentation to clarify what I think is a common newbie problem. On the http://karma-runner.github.io/0.8/plus/RequireJS.html page:

For the question "Which files do you want to test?", we choose all the files we want to load with require.js.

Fair enough, but it assumes knowledge about the default served=true attribute. Someone like myself with a little require.js experience may find this contradictory - why include files that require is going to load anyway?
Some extra comments in the sample karma.conf.js listing would have helped a lot. For example:

{pattern: 'test/*_/_Spec.js', included: false, served: true}, //require loads the files using karma's localhost server

[vojtajina]

@trevithj could you send a PR with improving the requirejs docs ? The source is in https://github.com/karma-runner/karma/blob/master/docs/plus/01-RequireJS.md

Feel free to change anything in order to make it more easier to understand.

[duro]

@vojtajina I am still running into this issue. I have reviewed your comment here:

#513 (comment)

What you are saying seems to be a workaround, and I am wondering what actions need to be taken to find a real solution. When I use RequireJS in my app, outside of Karma, all of my relative paths work just fine, both in the config shim, as well as in AMD module definitions.

The problem only seems to arise when my app is running inside a Karma environment.

Is this a problem on the RequireJS end, or is a a problem on the Karma end? If my app runs 100% with relative paths outside of Karma, but not inside Karma, it would seem that Karma is problem, no?

I'd like to push to see this issue resolved, as it is the only thing holding me back from integrating Karma seamlessly into my workflow.

Thanks for your follow up.

[chrisprice]

As best as I can tell, this is what's happening -

• RequireJS is being configured such that deps includes URLs of the form /base/foo/bar.js
• RequireJS is triggering an alternative "as-is" loading strategy for these files (and importantly all of their dependencies) as described in the documentation indicated by a dependency string starting with a slash, has a protocol, or ends in .js.

This is how I solved the problem -

require.config({
    // Karma serves files from '/base'
    baseUrl: '/base',

    paths: {
        'jquery': 'jquery-2.1.1'
    },

    shim: {
        'jquery': {
            exports: '$'
        }
    },

    // ask Require.js to load these files (all our tests)
    deps: Object.keys(window['__karma__'].files)
    // only include *Spec.js files
        .filter(f => /Spec\.js$/.test(f))
    // remove leading /base
        .map(f => f.replace(/^\/base/, '.'))
    // remove .js extension
        .map(f => f.replace(/\.js$/, '')),

    // start test run, once Require.js is done
    callback: window['__karma__'].start
});