Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ES6 modules don't work in 2.0.0-beta.1 #12011

Closed
jarek-foksa opened this issue Feb 22, 2018 · 25 comments · Fixed by electron/libchromiumcontent#459

Comments

@jarek-foksa
Copy link

@jarek-foksa jarek-foksa commented Feb 22, 2018

  • Electron version: 2.0.0-beta.1
  • Operating system: macOS 10.13.3

<script src="script.js" type="module"></script> should execute the code in script.js file. Instead, the following error is thrown:

Failed to load module script: The server responded with a non-JavaScript MIME type of "". Strict MIME type checking is enforced for module scripts per HTML spec.

NW.js used to be affected by exactly the same issue: nwjs/nw.js#6106

@codebytere

This comment has been minimized.

Copy link
Member

@codebytere codebytere commented Feb 26, 2018

this is a chromium 61 issue; i'll see if there's a patch i can cherry-pick into libcc

@deepak1556

This comment has been minimized.

Copy link
Member

@deepak1556 deepak1556 commented Mar 1, 2018

Loading es6 modules over null origin (file:// scheme) is not supported by standard https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script, we would like to follow similar behavior. At the moment we are setting CORS header for file:// to support service workers in electron, but this will be removed in the upcoming version. Closing this as expected behavior. Thanks!

@deepak1556 deepak1556 closed this Mar 1, 2018
@jarek-foksa

This comment has been minimized.

Copy link
Author

@jarek-foksa jarek-foksa commented Mar 1, 2018

@deepak1556 Do you mean that Electron will not support ES6 modules unless they are loaded from server?

@MarshallOfSound

This comment has been minimized.

Copy link
Member

@MarshallOfSound MarshallOfSound commented Mar 1, 2018

@jarek-foksa No, you can just use a custom protocol to provide your files (you should probably do this anyway, the file:// protocol doesn't follow the same rules as other protocols)

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Mar 1, 2018

@MarshallOfSound can you elaborate a little on this please... A link on setting up a custom protocol in such a way that it would work would be really helpful.

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Mar 1, 2018

Just to confirm the following (crude) test case works:

Assuming the conventional node requires like const {URL} = require('url')… per electron guides.

function createProtocol(name = 'app', {
  prefix = `${name}:///`, basepath = __dirname
} = {}) {
  protocol.registerStandardSchemes([name], { secure: true });
  app.on('ready', () => {
    protocol.registerBufferProtocol(name, (request, callback) => {
      const url = new URL(request.url.replace(/^.*?:[/]*/, prefix));
      const { pathname, hash, searchParams } = url;
      const filename = path.resolve(basepath, pathname.slice(1));
      const extension = path.extname(pathname);
      const scope = { request, url, pathname, filename, extension };

      try {
        const content = `${readFile(filename)}` || '';
        const data = scope.data = Buffer.from(content);
        const mimeType = scope.mimeType = mimeTypeFor(extension);
        callback({ mimeType, data });
      } catch (exception) {
        scope.exception = exception;
        console.error(exception, scope);
      }
    }, (error) => {
      if (error) console.error('Failed to register protocol')
    });
  });

  const mimeTypeFor = (extension) =>
    mimeTypeFor[`${/^.\w+$/.test(extension) && extension || ''}`.toLowerCase()];

  mimeTypeFor[''] =
    mimeTypeFor['.js'] =
    mimeTypeFor['.ts'] =
    mimeTypeFor['.mjs'] = 'text/javascript',
    mimeTypeFor['.html'] =
    mimeTypeFor['.htm'] = 'text/html',
    mimeTypeFor['.json'] = 'application/json',
    mimeTypeFor['.css'] = 'text/css',
    mimeTypeFor['.svg'] = 'application/svg+xml';
}
@jarek-foksa

This comment has been minimized.

Copy link
Author

@jarek-foksa jarek-foksa commented Mar 1, 2018

Could this protocol thing be somehow abstracted away by Electron? Most developers (including me) don't even know what custom protocols are and I would expect a core language feature such as modules to work out of the box.

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Mar 6, 2018

@jarek-foksa After a bit of testing, I think @deepak1556's view on keeping with standards re file: is absolutely right. Though this does complicate things a little for prototyping, the idea that you can load content from a site inside a BrowserView means that blessing the file: protocol simply opens up a new potential for vulnerabilities.

That said, I think for non-web BrowserViews with node integration, it can be a lot more useful to consider NodeJS's new Loader.

Here is a quick example with a preload script:

// preload.js
const { NativeModule, mainModule, mainModule: { filename } } = process;

{ /* Loader */

  const module = mainModule;

  const base = module.base =
    `file://${filename.replace(/[^/]*([#?].*)?$/, '')}`;

  const Loader = NativeModule.require('internal/loader/Loader');

  const AsyncModule = 'async module'; // [Symbol.for('async module')];

  class NodeLoader extends Loader {
    async import(specifier, parentURL = this.base) {
      const job = await this.getModuleJob(specifier, parentURL);
      const module = await (job[AsyncModule] || (job[AsyncModule] = job.run()));
      return module.namespace();
    }
  }

  const loader = module.loader = new NodeLoader(base);

  module.import = (specifier, referrer) =>
    loader.import(specifier, referrer);
}

And to use it in your page:

<script>
  { /* Load a "valid" Web Component */
    const { log, error } = console;
    const specifier = './tests/static/custom-element.mjs';
    (async () => {
      try {
        const a = await module.import(specifier); // .catch(error);
        const b = await module.import(specifier);
        log('import(%o) => [#1 %O] === [#2 %O] %O', specifier, a, b, a === b);
      } catch (exception) {
        error(exception);
      }
    })();
  }
</script>

Node's loader is a far more flexible option for desktop applications in my opinion. It has a learning curve, and it is technically still experimental, but the things it can do and how fast it can load (and even transpile on-demand) stuff puts it in a whole new category of loaders (if you are patient enough to give it a shot and figure it out). Just keep in mind that "by default" it expects modules to be awkwardly .mjs because someone apparently invented something called CommonJS modules with .js 😉 before JavaScript so that extension was taken.

Future:

Once Electron hits Chromium 63 and NodeJS 10, it will even be possible to bind the loader to the BrowserView's dynamic import and literally defer all import(…) statements (but not static imports) to the loader.

@jarek-foksa

This comment has been minimized.

Copy link
Author

@jarek-foksa jarek-foksa commented Mar 6, 2018

@SMotaal Thanks for providing all the examples. In your first example (with protocols), how do you load the app page? After registering the app: protocol I was initially trying to load the page as follows, but the ES6 modules inside the page would still fail to load:

appWindow.loadURL(`file://${__dirname}/index.html`);

After that I switched to the code below, but it would just load a blank/stub page instead of index.html:

appWindow.loadURL(`app://index.html`);
@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Mar 6, 2018

@jarek-foksa I worked on a gist, let's continue this discussion there:
https://gist.github.com/smotaal/f1e6dbb5c0420bfd585874bd29f11c43

@jarrodek

This comment has been minimized.

Copy link

@jarrodek jarrodek commented Sep 1, 2018

I am trying to run electron app with Polymer 3 modules. I have difficulties getting it right.

With given gist I can make import modules from the HTML page but imported modules can't resolve their dependencies.

This is an example setup (from Polymer documentation):

(demo/demo.html and main is in demo/main.js)

<script src="../node_modules/@webcomponents/webcomponentsjs/webcomponents-loader.js"></script>
<script type="module">
  // case 1
  import '../node_modules/@polymer/paper-styles/typography.js';
  // case 2
  import 'app:@polymer/paper-styles/shadow.js';
</script>

In case 1 it throws the error with mime type check.

In case 2 the create protocol function fails to load the module.
It says the request.url is app://polymer/paper-styles/typography.js which is already not true because it is missing @ of Polymer scope. Eventually it resolves the pathname to ./polymer/paper-styles/typography.js.
I never work with custom protocols so I am not sure what to do next. Could you advise?

@dravenk

This comment has been minimized.

Copy link

@dravenk dravenk commented Sep 27, 2018

I tried running polymer 3 on electron 3, but failed. I saw the release log. Electron 3 using chrome 66, which is much higher than 61. Why is this problem?

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Sep 27, 2018

@jarrodek sorry for not seeing this before was it resolved?

Unfortunately I have pulled back on my polymer experiments a bit, but in a more general sense, can you guys try troubleshooting (in your current setup).

First the try following try in console, if in doubt, try them in a script tag (not module).

  • import().then(console.log, console.warn) for each of:

    • the module that you imported
    • the resolved path to one or two of that module’s own dependencies
  • also do the same with fetch( … then … etc.

  • If it works at least in script, add type=module

  • If it works, save that to a separate file right next to the page (like import-test.js) and add src=./import-test.js to the now empty script tag

  • try both with and without type=model.

  • finally, rewrite those to static imports import * as x from '';

  • let me know

@jarrodek

This comment has been minimized.

Copy link

@jarrodek jarrodek commented Sep 27, 2018

@SMotaal It's not about polymer itself but how web components works. Polymer just adds some sugar on top of the spec.

My problem is about making it work when imported module has more imports. With the custom protocol I cannot properly resolve those dependencies. Or at least I am not sure how to do it properly.

I was working on a demo app some time ago. My main app still uses HTML imports that doesn't use modules imports. However in the beginning of the next year I would probably switch to module imports. When I do this I would like to know how to do it properly.

I may try to do this demo app again this weekend. If I won't be able to then I will try in about 2 weeks as I am going to vacations next week.

Thanks

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Sep 27, 2018

@jarrodek I don't think it is worth the effort, honestly, given the state modules are in for the next couple of months I think it is best to focus on fixing the odd issue in something working well otherwise. I know for a fact that Electron awaits Node awaits TC39 (relevant proposals this week).

If you have a way to demo this quickly, and only if it helps you and not simply pull you into more experimental dilemmas, I can try (always find those interesting for some reason). But, please, don't waste your time, we are almost across the threshold — after ~4 years of eagerly motivated efforts wasted due to implementation delays maybe for good reasons.

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Sep 27, 2018

A trick to resolve modules from relative specifiers:

HTML

<script>
  { // This block scope simply prevents making resolve a global constant

    // Classic script tags can use window.location as the referrer
    const resolve = (specifier, referrer = location)  => `${new URL(specifier, referrer)}`;

    // Resolves the parent directory of this page
    console.log(resolve('.')); 
    // Resolves "./index.js" relative to this page
    console.log(resolve('./index.js')); 
    // Resolves "./modules/module-a/index.js" relative to this page
    console.log(resolve('./modules/module-a/index.js')); 
    // Resolves "./modules/module-a/submodule-b/index.js" relative to this page
    console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

  }
</script>

<script type=module>

  // Module script tags can use import.meta.url (if in doubt you can just use window.location)
  const resolve = (specifier, referrer = import.meta.url || location)  => `${new URL(specifier, referrer)}`;

  // Resolves the parent directory of this page
  console.log(resolve('.')); 
  // Resolves "./index.js" relative to this page
  console.log(resolve('./index.js')); 
  // Resolves "./modules/module-a/index.js" relative to this page
  console.log(resolve('./modules/module-a/index.js')); 
  // Resolves "./modules/module-a/submodule-b/index.js" relative to this page
  console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

</script>

ES Modules Only

<script type=module src="./path/to/the/module/that/resolves/another.js"></script>
// Module script tags can use import.meta.url (never use window.location here)
const resolve = (specifier, referrer = import.meta.url)  => `${new URL(specifier, referrer)}`;

// Resolves the parent directory of this module
console.log(resolve('.')); 
// Resolves "./index.js" relative to this module
console.log(resolve('./index.js')); 
// Resolves "./modules/module-a/index.js" relative to this module
console.log(resolve('./modules/module-a/index.js')); 
// Resolves "./modules/module-a/submodule-b/index.js" relative to this module
console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

export {resolve}

Trick to help debug how modules would resolve

<script type=module>

  import { resolve } from './path/to/the/module/that/resolves/another.js';

  // Resolves the parent directory of the module NOT this page
  console.log(resolve('.'));

  // So now we can check where that module resolves module-a from exactly
  console.log(resolve('../../../../../../modules/module-a/index.js'));

</script>
@jarrodek

This comment has been minimized.

Copy link

@jarrodek jarrodek commented Sep 27, 2018

Thanks for the example @SMotaal. I will experiment with it.

@dsanders11

This comment has been minimized.

Copy link
Contributor

@dsanders11 dsanders11 commented Oct 27, 2018

I don't think it is worth the effort, honestly, given the state modules are in for the next couple of months...But, please, don't waste your time, we are almost across the threshold — after ~4 years of eagerly motivated efforts wasted due to implementation delays maybe for good reasons.

@SMotaal, could you elaborate on that or provide a link where I could read about it? It sounds like you're saying it'll soon (relatively?) be easier to use ES6 modules in Electron?

@SMotaal

This comment has been minimized.

Copy link

@SMotaal SMotaal commented Oct 27, 2018

@dsanders11 from my own perspective as a user, I know for a fact that ES module support in Electron overlaps with support in Chromium and support in Node.

For over a year, support through Chromium (ie script type module) was available. But personally, I have not used it in a while, and when I did, it was clearly browser-centric (may have changed).

For the Node.js part, which only offers an experimental implementation via flags. This part is not meant for production, comes with frequent breaking changes, and is expected to be replaced with a separate implementation all together once the modules team designs a new module system based on gained insights from the experimental one.

A while back I tried to use experimental-modules in Electron, and I ran into some compatibility issues which were premature to address in either Node.js or Electron. Fixing it was not hard, however, it became clear to me that experimental patching across projects is far more complicated, and instead it makes sense to factor such findings into the design of the new module system instead.

Obviously there is a lot going on there, which takes place at nodejs/modules. Just a heads up, it can be very hard to track discussion threads there, as a lot of discussions involve a lot of context, brainstorming and so on. The actual design is now in Phase 2, and there is a lot more work that needs to take place to bring it to a level that compares with and goes beyond the features in experimental before they land in Node.js.

Until then, it is hard to predict how this work will integrate in Electron, which I am sure is something that Electron folks are keeping track of and are best able to elaborate on.

@stefaneidelloth

This comment has been minimized.

Copy link

@stefaneidelloth stefaneidelloth commented Feb 15, 2019

Related issue: #13402

@jarrodek

This comment has been minimized.

Copy link

@jarrodek jarrodek commented Mar 5, 2019

I have finally figured out what is happening with Polymer elements. The thing is that all paper-* and iron-* elements have node like paths when using imports (starts with @polymer/ for example) instead of using relative paths. When trying to load any of Google's components into Electron app it fails giving the error message I described above. In the development process you should use library like polyserve to transform the components to final and working form and build the application the Polymer way before publishing it.
It is inconvenient but I guess the Polymer team had good reason to do this. I am upgrading my set of components and after upgrading just a few I run into problem with dependency management. I ended up doing the same thing as the Polymer team did (I think).

So to work with some libraries based on web components (especially Google's) you have to register a protocol for the components and use tools to transform the output to correct syntax (Polymer CLI, polyserve, own solution) or import files directly from node_modules but using polyserve at the same time. I had to do similar thing recently for another (not Electron) app: https://github.com/advanced-rest-client/api-components-apps/blob/master/ci-status-app/app.js#L40
In first step I run polymer serve command to run the UI server and then a proxy module when running the application. This way when requesting the UI in the application the request is finally proxied to Polymer transformed components.

frederiksen pushed a commit to frederiksen/angular-electron-boilerplate that referenced this issue Jul 23, 2019
@Lonniebiz

This comment has been minimized.

Copy link

@Lonniebiz Lonniebiz commented Jul 29, 2019

This is disappointing. Electron should abstract away all of this custom protocol complexity and make ES Modules something that just works.

@codebytere codebytere removed their assignment Jul 29, 2019
@jarrodek

This comment has been minimized.

Copy link

@jarrodek jarrodek commented Jul 29, 2019

@Lonniebiz It's more than that. I do agree that ES modules should return proper content type headers so it can be executed without any additional setup. However authors tend to not to comply with ES spec and use node like paths when importing other modules. It is because they assume a some kind pre-processing in development. In electron you have to take care of it yourself and I am not sure if that's the role of Electron to abstract this.

@Lonniebiz

This comment has been minimized.

Copy link

@Lonniebiz Lonniebiz commented Jul 29, 2019

@jarrodek If Chromium needs a server to serve ES modules, instead of the direct filesystem, then Electron should be a server (as far as Chromium is concerned).

@xenomarz

This comment has been minimized.

Copy link

@xenomarz xenomarz commented Aug 5, 2019

I have finally figured out what is happening with Polymer elements. The thing is that all paper-* and iron-* elements have node like paths when using imports (starts with @polymer/ for example) instead of using relative paths. When trying to load any of Google's components into Electron app it fails giving the error message I described above. In the development process you should use library like polyserve to transform the components to final and working form and build the application the Polymer way before publishing it.
It is inconvenient but I guess the Polymer team had good reason to do this. I am upgrading my set of components and after upgrading just a few I run into problem with dependency management. I ended up doing the same thing as the Polymer team did (I think).

So to work with some libraries based on web components (especially Google's) you have to register a protocol for the components and use tools to transform the output to correct syntax (Polymer CLI, polyserve, own solution) or import files directly from node_modules but using polyserve at the same time. I had to do similar thing recently for another (not Electron) app: https://github.com/advanced-rest-client/api-components-apps/blob/master/ci-status-app/app.js#L40
In first step I run polymer serve command to run the UI server and then a proxy module when running the application. This way when requesting the UI in the application the request is finally proxied to Polymer transformed components.

I just use pika-web in order to transform paths - a great solution for electron apps that use web components. it saves the hassle of performing a build step every time you run your app. However, the protocol hack is still needed.

https://spectrum.chat/pika/pika-web/pika-web-and-web-components~c2647922-d8d7-457d-993f-0dd38198a749

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.