Handling scripts that assume nodejs modules are available if in CommonJS environment

[tauren]

The xls.js script found at https://github.com/SheetJS/js-xls assumes it can use fs and other node.js modules if it is within a CommonJS environment. This means that running it within a browser environment with JSPM is throwing errors such as fs.js not found. The readme indicates the script supports running in a browser and will not load nodejs modules or extra functionality such as cpexcel.

Here are some code snippets:

if(typeof module !== "undefined" && typeof require !== 'undefined') {
    if(typeof cptable === 'undefined') cptable = require('./dist/cpexcel');
    current_cptable = cptable[current_codepage];
}
...
function readFileSync(filename, options) {
    if(fs === undefined) fs = require('fs');
    return parse(fs.readFileSync(filename), options);
}

It seems to me the code would have to be changed to work within JSPM and that simple overrides will not solve this problem. Is this true, and if not, any ideas how to make it work?

[guybedford]

Perhaps try using jspm install npm:xlsjs rather than the github version?

[tauren]

I actually tried that first. Both the npm and github versions work the
same. Both the root xls.js and the dist/xls.js try to use node dependencies
when used in the browser via JSPM.

On Fri, Dec 5, 2014, 2:58 AM Guy Bedford notifications@github.com wrote:

Perhaps try using jspm install npm:xlsjs rather than the github version?

—
Reply to this email directly or view it on GitHub
#79 (comment).

[andreasgrimm]

along the lines of https://stackoverflow.com/questions/23038483/how-to-detect-when-browserify-is-being-run ... maybe js-xls should just use if (typeof window === 'undefined') to determine it's runtime environment.

having taken a slight look in https://github.com/SheetJS/js-xls/blob/master/xls.js it seems they don't use browserify, so checking for window shouldn't be a problem.
I guess it'd be better if they actuallly would use browserify because it can make things easier when developing cross-environment modules.

This code looks like once being a global in the browser got shoved in the ability to run in node rather quick and dirty .. not regarding practices like https://github.com/substack/browserify-handbook or https://gist.github.com/defunctzombie/4339901

I know all this doesn't solve your current problem, it's just some things I noticed.

edit: I'm guessing if they'd change their module format from global to CommonJS they'd encounter the same problem ;)

[andreasgrimm]

taking a slightly deeper look in the code base, https://github.com/SheetJS/js-xls/blob/master/dist/xls.js#L1687 seems the only place where the above mentioned error could happen.

I didn't find out though, who or what is calling the public readSync funciton https://github.com/SheetJS/js-xls/blob/master/dist/xls.js#L1725
it seems it's options (https://github.com/SheetJS/js-xls/blob/master/dist/xls.js#L1691) determine, if fs = require('fs') gets executed at all.

require('./dist/cpexcel') shouldn't be a problem as far as I can tell.

[guybedford]

Apologies for the delayed response on this. I see it is a global actually - yes loading from GitHub would make sense then and you can do:

jspm install github:SheetJS/js-xls -o "{ 'format': 'global' }"

Just let me know if you want to do a PR on the override or if you need help with it at all.

[andreasgrimm]

I didn't use SheetJS myself, but @tauren, isn't it true that the error fs.js not found gets thrown only when the function readSync get's called?
I don't know though in what cases it would get called. I couldn't find a case where it would be called within the module itselft .. assuming you had to call it manually as a consumer. Do you know if that's the case?
If yes the code implies to me that require('fs') would get called nevertheless. Then it'd depend on how SystemJS treats the require() statement.

(edit) maybe you should ad nodelibs package to your app in that case (at least until the loader supports conditionally loading).

[tauren]

@guybedford Thanks! I didn't realize that JSPM could work that way. It worked perfectly with your suggested override, Bravo! I'll create a PR.

@andreasgrimm Thanks for your investigation. I do believe that when I was experiencing the fs.js not found error, it was where you indicate. There is only one require('fs') in the source. I also haven't investigated too closely, but I would guess that the switch case is not 'file' when running in a browser, so that code would not get called in a browser:
https://github.com/SheetJS/js-xls/blob/master/dist/xls.js#L1725
https://github.com/SheetJS/js-xls/blob/master/dist/xls.js#L1693

[guybedford]

@tauren glad to hear that! That is actually one of the main use cases for overrides! Setting how to interpret a module, as the module environment the module is supposed runs in is what jspm needs to know.

[andreasgrimm]

Just for clarification .. @tauren you said that "Both the root xls.js and the dist/xls.js try to use node dependencies when used in the browser via JSPM".
How can the node core deps (e.g. require('fs')) not be a problem anymore just by handling the module as global? .. especially when not having nodelibs as a dependency that can be used as a replacement for node core libs when running in the browser.

(edit) I guess I'm missing something here.
What's the difference between interpreting CommonJS modules as global vs CommonJS? Is it just the global environment variables that a module can ask for that differ?
I guess that the "loading" of dependent code (e.g. ES6 import vs CommonJS require) get's translated to ES6/SystemJS in either case anyways ?

[guybedford]

@andreasgrimm I assume the code does environment detection something like:

if (typeof require != 'undefined') {
  var fs = require('fs');
}

By forcing jspm to treat it as a global and not CommonJS (which it would be detected as by default in jspm), it would never run that code.

[andreasgrimm]

that's exactly what I mean and what confused me, because the code actually is

if(fs === undefined) fs = require('fs');

and that code gets executed depending on external data (input).

[tauren]

@andreasgrimm I believe that line of code can only be executed if you use XLS.readFile(). I imagine if you try that in a browser you'll see an error. The XLS docs show code samples that use XLS.read() in a browser instead which should not execute require('fs').

[andreasgrimm]

@tauren That'd make sense.

But then on the other hand I wonder why you get that error when the module doesn't get interpreted as global but as CommonJS instead. Because you didn't call that function explicitly but just loaded the module, right?

[andreasgrimm]

I guess it's because in CommonJS "mode" (as in ES6 and AMD as well) at startup time all the code gets parsed for require() (or import / define) expressions so that dependencies can be loaded appropriately (and that's where the error happens) .. whereas a global module's dependencies need to be specified outside of the module code itself via shim configuarion in config.js.

@guybedford Can you confirm this? Sorry if it's too obvious.

In that case adding nodelibs as a dependency to your app should have solved that problem, too, right?

[tauren]

@andreasgrimm I believe you are correct. In "global" mode, pre-parsing the file for require commands and then loading those files must not happen or a failure would have occurred. This is obvious now that we are discussing it, but I didn't really understand how it worked until now. It would be great if the JSPM documentation could do a deeper dive into how the different "modes" work.

[guybedford]

Great to hear these issues are making sense without further explanation
this side. Although certainly agreed more high-level documentation
discussing the implementation approaches and decisions are needed.

Will certainly bear this in mind - I'm hoping to write more articles about
this stuff in the new year.

On 12 December 2014 at 10:15, Tauren Mills notifications@github.com wrote:

@andreasgrimm https://github.com/andreasgrimm I believe you are
correct. In "global" mode, pre-parsing the file for require commands and
then loading those files must not happen or a failure would have occurred.
This is obvious now that we are discussing it, but I didn't really
understand how it worked until now. It would be great if the JSPM
documentation could do a deeper dive into how the different "modes" work.

Reply to this email directly or view it on GitHub
#79 (comment).