Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Please document how to generate a commonJS/systemJS/AMD JS module from python using transcrypt for loading using require.js or node/etc. #495

Closed
shlomif opened this issue Mar 6, 2018 · 26 comments
Closed

Please document how to generate a commonJS/systemJS/AMD JS module from python using transcrypt for loading using require.js or node/etc. #495

shlomif opened this issue Mar 6, 2018 · 26 comments
Labels
IS: adapt doc NEED: help SUB: documentation

Comments

@shlomif
Copy link

shlomif commented Mar 6, 2018

Hi!

Please document how to generate a commonJS/systemJS/AMD JS module from python using transcrypt for loading using require.js or node/etc.

I saw this discussion - #94 - but I do not understand how to do it based on that.
@JdeH
Copy link
Collaborator

JdeH commented Mar 8, 2018 •
edited

I don't currently don't know enough from JS module systems to describe this properly.
Can anyone give it a try, e.g. with some examples.
I am more a Python than a JS guy...

@doconix
Copy link
Contributor

doconix commented Mar 8, 2018 •
edited

I know enough (I think :) to write it up. However, there are a couple things that transcrypt needs before it can happen.

  1. We need to be able to do JS imports and exports at the top level of the file. Right now, transcrypt doesn't provide a way to place code outside the containing function. The JS (ES6) standard for the import statement says it has to happen outside any function--in the main space of the .js file.

  2. We need to be able to disable placing the function in the global window object (very last line of the generated JS) or in any other parent. In other words, we need to disable calling the function entirely. Just create the function and let the exports code (that goes in the top level of the file (Few questions #1 above) export the function according to the AMD, CommonJS, etc. ways.

From what I can see, the two above items are everything needed to make module creation happen.

@shlomif
Copy link
Author

shlomif commented Mar 8, 2018 via email

@JdeH
Copy link
Collaborator

JdeH commented Mar 8, 2018 •
edited

@doconix

Could you manually modify a simple, generated JS file, e.g. hello.js (compiled from hello.py in the demos) to lay down exactly what is needed.

You can use the -p .user switch to avoid the main function to be called.
If you manually insert the needed export / import code, we can later on create a facility to insert it at will, e.g. as a free to choose prologue and epilogue by means of a pragma.

@doconix
Copy link
Contributor

doconix commented Mar 8, 2018

Will do!

@doconix
Copy link
Contributor

doconix commented Mar 8, 2018 •
edited

Wow. So I totally forgot that I already did this with an earlier project and published the plugin on npm. I'm getting old.

Here's the repo: https://github.com/doconix/autotimers
In particular, check out the exports at the bottom of the main file: https://github.com/doconix/autotimers/blob/master/src/main.py#L117

The code is a little different than the common pattern used in JS. Most modules follow this pattern: https://github.com/umdjs/umd/blob/master/templates/returnExports.js. I modified it because using Python code syntax makes it quite a bit simpler and easier. To my view, my code does the exact thing, just with a refactored way.

The good news for @JdeH is no changes are needed in Transcrypt. Although I will note that using npm modules from Transcrypt code is difficult because of #1 above. The workaround is to use the deprecated require() function. Since the npm repository seems to be the future of JS modules, it might be a good thing to support import in Transcrypt. The issue deserves its own page instead of this paragraph, but I'll leave any development at @JdeH discretion.

@JdeH
Copy link
Collaborator

JdeH commented Mar 9, 2018 •
edited

@doconix
What do you mean by #1? (It links to the very first issue, which is probably not what is meant.)

Do you mean the following point of @shlomif :

1. We need to be able to do JS imports and exports at the top level of the
 file.  Right now, transcrypt doesn't provide a way to place code outside the
 containing function.  The JS (ES6) standard for the import statement says it
 has to happen *outside* any function--in the main space of the .js file.

So if Transcrypt would offer the possibility to have code before and after the main function definition, that would help, right?

@shlomif
Copy link
Author

shlomif commented Mar 9, 2018 via email

@JdeH
Copy link
Collaborator

JdeH commented Mar 9, 2018

Investigating adding ES6 modules to TS.
Testing with Chrome.
Still some trouble in paradise...

https://stackoverflow.com/questions/49190897/cant-get-export-as-default-to-work-on-chrome-version-64-0-3282-186-official-bu

@doconix
Copy link
Contributor

doconix commented Mar 9, 2018 •
edited

Regarding "#1" above, we need to be able to add the import statement at the top of the following example. It is outside the dev function below -- at the top level of the file.

The import $ as 'jquery' line below is what we need to be able to inject.

"use strict";
import $ as 'jquery';
// Transcrypt'ed from Python, 2017-11-17 21:55:04
function dev () {
    ...
}
dev ();
//# sourceMappingURL=extra/sourcemap/dev.js.map

AFAIK, the following is not valid in JS (although it is just fine in regular python):

"use strict";
// Transcrypt'ed from Python, 2017-11-17 21:55:04
function dev () {
    // can't do the following within the function:
    import $ as 'jquery';
    ...
}
dev ();
//# sourceMappingURL=extra/sourcemap/dev.js.map

@doconix
Copy link
Contributor

doconix commented Mar 9, 2018

I see from your SO question that you are working on exporting ES6 modules. That's the right approach to work for.

The solution I referenced above (https://github.com/doconix/autotimers/blob/master/src/main.py#L117) implements the "universal module definition". It supports the different methods that have been competing for the last few years: CommonJS, AMD (requirejs), etc. ES6 modules are the result of this competition and is the future.

So while my solution works well right now, the way you are going is the standards approach. I should have suggested this rather than the universal one I did. Just wanted to validate where you are headed with it.

@JdeH
Copy link
Collaborator

JdeH commented Mar 9, 2018 •
edited

Your solution is very useful, since it addresses the currently popular modules systems.
However the "official" JS6 one is the one I want to include by default in Transcrypt.
I've been experimenting with it, and it alreay works, but it doesn't please me, since it is orthogonal to TS/Python imports.
What I'm after is a mode where every Python import automatically is a JS6 import.
So pervasive transparent on-demand module loading.
I expect that it could kind of supersede the unit mechanism.
What I've been running into is that the support in e.g. Chrome doesn't always seem to function right.
So I'm not sure it's all going to work out.
But trying...
Will be some time, this is a long term thing, most browsers do not yet support it properly.
But if possible it is the way to go.

@doconix
Copy link
Contributor

doconix commented Mar 9, 2018

That's beautiful. Imports aren't fully into the browsers yet, so that's one possible reason it's difficult. In any case, almost all people working with ES6+ code right now are transpiling back to 5 with webpack, babel, etc. So in my mind, the real question is whether babel can handle the imports.

I'm sure you know, but note also that the new import() function (not the statement) is being integrated into the JS standard right now. So that's a possibility for making it work as well.

Tough to hit a moving target...

@metamarcdw
Copy link
Contributor

I would definitely love to see the prologue/epilogue pragmas as a stopgap until the ES6 module stuff is done. Any pointers on how I could get started on that?

@JdeH
Copy link
Collaborator

JdeH commented Mar 12, 2018 •
edited

I've already built in and removed it again, since it messed up the sourcemaps. (linenumbers are offset).
Currently the best thing to do is probably:

  • If you want to avoid the main encapsulator function to be called and its result assigned to window, use the -p .user switch:

From the code:
self.argParser.add_argument ('-p', '--parent', nargs='?', help = "object that will hold application, default is window. Use -p .none to generate orphan application, e.g. for use in node.js. Use -p .user to generate an application that has to be explicitly initialized by calling <application name> (), e.g. after the full page has loaded")

  • To add a prologue and epilogue, the best thing to do (since it's only temporary) is to read the generated .js file into a small postprocessing application and glue something before and after it, then write the result.

Releasing a pragma suggests it will be a permanent facility, which I want to avoid, because taking it out again will break user code.

@shlomif
Copy link
Author

shlomif commented Mar 13, 2018

Can someone provide a script or whatever to automate the process as a gist or similar? I don't fully understand what to do.

@doconix
Copy link
Contributor

doconix commented Mar 13, 2018

@shlomif I scaled down my project and posted to https://github.com/doconix/transcrypt_module/. Please see the src/main.py for the code on making a UMD.

This should work with 1) browser, 2) AMD, and 3) CommonJS. I have tested my original project with these three, but I've only had time today to test the first. It should work with all three, though, because the code is essentially the same.

Note that this does NOT make the module work with the ES6 standard. UMD and ES6 are not really compatible, from what I see. An ES6 module would need to export default .... I'm not sure if that can be done with Transcrypt yet.

But in any case, the code I posted will work with today's tech: webpack, gulp, browserify, node, etc.

@doconix
Copy link
Contributor

doconix commented Mar 13, 2018

Also note that it needs to be built with specific transcrypt options. I created the build.py file to do it the right way.

@shlomif
Copy link
Author

shlomif commented Mar 13, 2018 via email

@shlomif
Copy link
Author

shlomif commented Mar 13, 2018

@doconix : can you please add a LICENSE/etc. file to the repo so it won't be ARR? See https://github.com/shlomif/Freenode-programming-channel-FAQ/blob/master/FAQ.mdwn#i-want-to-release-my-code---which-open-source-licence-should-i-use . I hope I don't seem too needy and picky.

@doconix
Copy link
Contributor

doconix commented Mar 13, 2018

@shlomif Done

@shlomif
Copy link
Author

shlomif commented Mar 13, 2018 via email

shlomif added a commit to shlomif/fc-solve that referenced this issue Mar 13, 2018
@shlomif
Integrate the CommonJS-based transcrypt output.
e95cc63 
See https://github.com/doconix/transcrypt_module/ and
TranscryptOrg/Transcrypt#495 . Thanks to @doconix and
@JdeH .
@shlomif
Copy link
Author

shlomif commented Mar 13, 2018

@doconix : it looks very good, thanks - I'll let you know what evolves next.

@shlomif
Copy link
Author

shlomif commented Mar 13, 2018

@doconix : thanks again, the code is now used live at http://fc-solve.shlomifish.org/js-fc-solve/find-deal/ . There still is some polish to be done , but it is not related to transcrypt.

@JdeH
Copy link
Collaborator

JdeH commented Apr 9, 2018 •
edited

ES6 modules are now available in the 3.7.1 alpha prerelease.

@JdeH JdeH closed this as completed Apr 9, 2018
@shlomif
Copy link
Author

shlomif commented Apr 9, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
IS: adapt doc NEED: help SUB: documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@shlomif @JdeH @doconix @metamarcdw