New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do i mark some modules to external? #144

Open
garrydzeng opened this Issue Dec 8, 2017 · 69 comments

Comments

Projects
None yet
@garrydzeng
Copy link

garrydzeng commented Dec 8, 2017

Choose one: is this a 馃悰 bug report or 馃檵 feature request?

馃檵 feature request

馃 Expected Behavior

Don't include external module in bundled file everywhere.
Like rollup globals option.
https://rollupjs.org/#big-list-of-options

馃實 Your Environment

Software Version(s)
Parcel 1.0.3
Node 9.2.0
npm/Yarn Yarn 1.2.1
Operating System Windows 10
@devongovett

This comment has been minimized.

Copy link
Member

devongovett commented Dec 20, 2017

Can you elaborate on the behavior you'd like to see here? I'm not sure how the option as documented by rollup is useful. Why not just use the global variable? Why is it an import at all?

@garrydzeng

This comment has been minimized.

Copy link
Author

garrydzeng commented Dec 20, 2017

@devongovett

Let me take React as an example.

I can use React as global variable in my project, but many third-party library imported React in their code and I can't control that, so, if I include React through a CDN, like <script src="//cdnjs.cloudflare.com/ajax/libs/react/16.2.0/umd/react.production.min.js"></script> and the bundle in my index.html, then browser has to download React twice.

  • one copy in the bundle
  • one from CDN

In order to avoid this problem, I have to import React in my project. If Parcel can replace React as global variable when it handling third-party library, then it can reduce bundle size & I can use CDN to speed up my website. React can be any library also.

I made an example: parcel-example.zip.
Hope this help!

@devongovett

This comment has been minimized.

Copy link
Member

devongovett commented Dec 23, 2017

I see. This seems somewhat related to aliasing actually. See #25.

@MalcolmDwyer

This comment has been minimized.

Copy link

MalcolmDwyer commented Feb 12, 2018

Another vote for this issue... I really like what you've done here with Parcel. It was super-easy to get my project 95% set up. This issue, (external resources), along with dev-server proxying are the two things holding me back.

Motivation for both is that I'm developing components that will be used within a larger application. There are external dependencies (that I don't want packaged up in the component build), and there are external APIs I want to hit while developing. In both cases, these are things that will be there when the project is deployed, but which are not a part of this component.

garrydzeng mention Rollup's "globals" option. Webpack similarly has "externals" (https://webpack.js.org/configuration/externals/).

You mentioned babel aliasing. That, I think, is a separate issue, because it's after the packager has decided to include a file, you can tell babel an alternate path to get it. There doesn't appear to be a way to tell babel that it should be ignored entirely. (At least not in a way that will keep other things happy). The "external" concept would really need to be configured at the packager.

I'd be happy to help out and contribute code to solving these issues, but given the 'zero-configuration' mantra, it's not clear to me where or how either could/should be done in Parcel. If you have ideas along those lines, please let me know. --Thanks

@starkwang

This comment has been minimized.

Copy link
Contributor

starkwang commented Feb 27, 2018

+1 on this issue. The aliases implemented in #850 can not solve this issue. Just like globals in Rollup and externals in webpack, Parcel also need an option to map modules to a global value.

I'd like to help with this issue if needed : )

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 4, 2018

I have a patched version that allows external scripts. The idea is to mark scripts as 鈥渃opy鈥 and provide a mapping of globale to modules that. But that still is kind of configuration...

@davidnagli

This comment has been minimized.

Copy link
Member

davidnagli commented Mar 4, 2018

@starkwang Cool! I鈥檇 be great if you can take this issue 馃槂

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Mar 4, 2018

@derolf @davidnagli Perhaps change the aliasing a slight bit and add something like this:

aliases {
  "react": false // This will ignore the package
}

This would cause it to skip without adding extra complex configurations. (Of course we still have to implement it and i'm not sure if it's sorta allowed by the standards)
This behaviour would be kinda similar to how browser.fs === false works for fs resolves

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 4, 2018

I have a proposal to extend the "alias" syntax to support externals:

"<module>": "<file>!<evaluated export term>"

Example (for cesiumjs):

"cesium": "./node_modules/cesium/Build/Cesium/Cesium.js!Cesium"

Semantics:

  • copy ./node_modules/cesium/Build/Cesium/** into the dest folder
  • lazy load or include in bundle (depending on how it is used)
  • create fake module "cesium" with cesium.exports = Cesium

Usage:
HTML inline load: <script src="cesium"></script>

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Mar 4, 2018

@derolf isn't this different than the described issue? The issue is about using external packages from cdn's wouldn't this just be lazyloading everything from the local server or cdn?

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 4, 2018

@DeMoorJasper

For CDN:

"cesium": "http://cdn.xyz.com/foo/bar/Cesium.js!Cesium"

could work the same way. But instead of copying into dest folder it's is directly linked.

I think the main idea is to somehow refer to "prebuilt" folders -- disable parcel parsing -- and link them to a fake module.

So the exclamation mark disables parcel parsing of the content and the term afterwards creates the export directive for the fake module.

@woubuc

This comment has been minimized.

Copy link

woubuc commented Mar 5, 2018

@derolf I think the ! syntax is fairly non-obvious, at least I haven't seen it used before. Is this 'standard' across similar implementations? Why not just use the key?

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

@woubuc So, if you look at webpack's external, you need a key plus which global symbol to bind to that key. So, we need to tell parcel which symbol to bind to the fake module.

Example: three-js exports a global symbol called THREE, so the syntax would be:

"three": "https://cdnjs.cloudflare.com/ajax/libs/three.js/90/three.min.js!THREE"

This follows the same approach in webpack.

@MalcolmDwyer

This comment has been minimized.

Copy link

MalcolmDwyer commented Mar 5, 2018

@DeMoorJasper 's <PackageName>: false makes sense to me (but misses renaming... which may be important in certain cases).
I don't see the advantage of @derolf's <PackageName>: <CDN path>:!<import name>, unless it's solving a different problem than what I was talking about.

Could the webpack externals concept be followed directly, except inside package.json. Then it's a 1-to-1 with a well-known feature, and keeps the ability to rename the import.

package.json:

  "dependencies": { },
  "parcel": { /* Or should externals be at the top level? */
    "externals": {
      "MyProjectConfig": "config",
      "react": "react"
    }
  }
}

Unlike the aliases: {packageName: false}, this allows for renaming. You can import from the proper name of the package (as if you were really packaging it up), but still have the code get linked into whatever the actual named variable is in the CDN/inline script.

My deployed index.html/jsp that pulls in the parceled project...

   <script>
      config = {
        someValue: 123
      }
   </script>
   <script src="://some-cdn/react.min.js"></script>
   <script src="./build/parceled-up-project.js></script>

And finally... some random source file in my project:

import MyProjectConfig from 'MyProjectConfig'
import React, { Component } from 'react'

For both those imports, the packager can find the 'from' in the externals list, and know that it doesn't need to actually fetch/import anything. And the actual variable names can get assigned with the desired renaming.

Something like this makes the most sense to me... but it's obviously getting away from 'zero-config', so I assume it would need some consensus before building it in. What do you think?

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

Package.json is not available to the parser. Because resolution is done afterwards.

So, you need two things:

  • disable parcel鈥檚 parsing if you pull in a prebundled script
  • link a module to an exported variable
@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

@MalcomDwyer okay, I can implement your 鈥渆xternals鈥 in package.json proposal and create module stubs the same way that Webpack does.

But, what is a good syntax to tell parcel to NOT bundle a script that is already in the public folder?

Like: <script parcel-ignore src=鈥榓bc.js>

@MalcolmDwyer

This comment has been minimized.

Copy link

MalcolmDwyer commented Mar 5, 2018

But, what is a good syntax to tell parcel to NOT bundle a script

In my example, that "deployed index.html/jsp" would not even be something that parcel ever sees...

I have my frontend project codebase that I use to develop, and test the javascript project. Then in a separate downstream project, I just pick up the bundle.js (src="./build/parceled-up-project.js in my example above) as a pre-built entity. It is in that downstream file that those external imports actually get resolved. (So they are external to parcel, but are part of the downstream project).

So I would probably have parcel building the real bundle.js (which ignores those externals), but also for development, I'd have parcel build a demo/index.html which would have the same imports and should include those files. I can't think of a time I'd be pointing parcel at an html with <script src="x"> and not want it to import "x".

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

I start bundling from a central HTML file and need to pull in stuff to parcel and stuff that is prebuilt (three js and cesium js).

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Mar 5, 2018

Package.json is not available to the parser. Because resolution is done afterwards.

You shouldn't wait this long right?
You just append the externals to the resolver, this way u can detect it before it even gets to the parser

<script parcel-ignore src=鈥榓bc.js>

No big fan of this, it would add a new sort of config to the ecosystem and limit the abilities it has, this seems to be more a use-case for the false key I described in my first comment on this issue, as this would extend beyond just html (adding the possibility to ignore in JS, for example optional dependencies parcel tries to resolve)

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

As far as I saw, parsing is done before resolving.

So, how can I tell the parser of the index.html not to bundle a script, but just leave it untouched.

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Mar 5, 2018

In the current codebase u should look at asset.addUrlDep... and the Resolver as those are the only places we use to actually resolve assets.

I'll try to make it a bit more clear, the parser.getAsset() never gets called if the dependency never gets added to the asset in the first place. The package is known to the parent asset, so you have full access to package.json

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

@DeMoorJasper I looked a lot at the code this weekend and found the asset.addUrlDep. The problem is that the parsing of the asset, processing, and generating the output is done in a Worker. This Worker has NO access to the resolver, but just creates put dependency-links to the bundles that will contain these dependencies into the processed files.

Then later, the bundler picks up the queued dependencies and resolves (including aliases) them and starts another Worker process.

Check this Bundler: loadAsset(asset)

async loadAsset(asset) {
... FIRST PROCESS
    let processed = this.cache && (await this.cache.read(asset.name));
    if (!processed || asset.shouldInvalidate(processed.cacheData)) {
      processed = await this.farm.run(asset.name, asset.package, this.options);
...
    }
... THEN RESOLVE
    // Resolve and load asset dependencies
    let assetDeps = await Promise.all(
      dependencies.map(async dep => {
...
          let assetDep = await this.resolveDep(asset, dep);
...
      })
    );

...
}

To fix this, we would need to expose the resolver to the Worker and let resolution happen there???

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

Ah, you mean Assert.pkg? You are right, it's having full access to the package.json!

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Mar 5, 2018

I'm pretty sure if the dependency never gets added, the bundler never picks them up anyway? The resolver changes would just be in case the bundler adds some dependencies.
Now that I looked at it resolver changes aren't even neccessary unless u wanna move the responsibility to the bundler, I haven't tested any of the things i say i'm suggesting as a fix.

And the code sample you show makes sense but doesn't have much to do with the ignore or this issue. It will never get to loadAsset if the asset is ignored, as that happens on the loadAsset of the previous/parent asset.

And if it's really necessary u could spin up a resolver per worker similar to how the parser is being used inside workers, although it would be nice to prevent this.

EDIT: Yes i do mean Asset.pkg (of the parent of the ignored/external package)

@derolf

This comment has been minimized.

Copy link

derolf commented Mar 5, 2018

So, which syntax do we follow now? An external needs TWO things: where is the file located and what does it export.

Example with cesium js:

"externals": {
 "cesium": {
   "exports": "Cesium", // --> this generates a module stub with cesium.exports = Cesium;
   "file": "node_modules/cesium/Build/Cesium/Cesium.js", // ---> that is the file to be src for the script
   "copy": "node_modules/cesium/Build/Cesium", // ---> that tells parcel to copy (or symlink) all that content into the public dir
 }
}

"copy" here is important since Cesium.js includes other files from that folder. So, the WHOLE folder needs to be present!

Makes sense?

@sgf

This comment has been minimized.

Copy link

sgf commented Aug 22, 2018

@DeMoorJasper @externals
i have the same problem.i talking about it in another issue with u.

I have a related issue, with TypeScript config.
My goal is simple: I want to use TypeScript types for Firebase in VS Code, and load Firebase from CDN.

But I couldn't get it to work with Parcel, so I'm not using Parcel even though I love the 0-config idea and using .ts directly.

Setup:
In index.ts:

import * as firebase from "firebase";
In index.pug:

html
head
script(src='https://cdn.jsdelivr.net/npm/firebase@4.13.1/firebase.min.js')
Expected:
During bundling, parcel.js does not double-include firebase

Actual:
During bundling, there's an extra 1MB+ file (thus my total website goes from ~40K to ~1MB size) containing second copy of firebase

just i need exclude the Vue.js when deploy.
but when in develop i need do import.

 "exclude": {
      "react": "react.js",
      "vue": "vue.js"
}

if Parcel support,that means Parcel will also better Freindly for Develop MPA(Multi Page App).

Daniel15 added a commit to Daniel15/SrcBrowse that referenced this issue Sep 5, 2018

Add Flow typing
 - Includes workaround for `config-chain` including intentionally broken test files in its repo: facebook/flow#869
 - Includes Hack for externals with Parcel: parcel-bundler/parcel#144
@Daniel15

This comment has been minimized.

Copy link

Daniel15 commented Sep 6, 2018

Aliasing (as mentioned by @fathyb above) partially solves this, which is great. Having to create the stub externals modules is a bit annoying, but doable.

The ideal scenario would be to also have some sort of HTML helper that supports external URLs, with a fallback to local URLs if the external URL fails to load. Of course, you can write the code for that manually, but it's useful for the framework to handle it.

For example, ASP.NET Core MVC has a script tag helper that supports specifying a fallback:

	<script 
		src="https://fb.me/react-0.14.3.min.js"
		asp-fallback-src="~/Content/js/lib/react-0.14.3.min.js"
		asp-fallback-test="window.React">
	</script>

This renders the following HTML:

<script src="https://fb.me/react-0.14.3.min.js"></script>
<script>(window.React||document.write("\u003Cscript src=\u0022\/Content\/js\/lib\/react-0.14.3.min.js\u0022\u003E\u003C\/script\u003E"));</script>

That is, try to load it from the CDN, but fall back to the local copy if the CDN load fails (based on the presence of the window.React global).

Something similar in Parcel could likely be done as a config option, say something like:

"externals": {
  "react": {
    "url": "https://examplecdn.com/react-1.2.3.min.js",
    "global": "window.React"
  }
}

As part of the build, this could compile the version of react in node_modules. At runtime, it could update the <script> tag in the HTML to load the CDN URL, and if it fails to load, use the fallback local version.

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Sep 6, 2018

@Daniel15 this isn't a widely supported syntax, it's probably not a good idea to introduce custom syntax.

You could however just create a js file that does that for you and include that, that way you would not even need externals.

Inlined JS will be supported in Parcel 2 as far as I know. So you could inline it in the future.

@Daniel15

This comment has been minimized.

Copy link

Daniel15 commented Sep 8, 2018

@DeMoorJasper Sure, I didn't mean that specific syntax, but just the idea in general (having some way to fall back to a local build if the CDN URL fails to load).

@tomasdev

This comment has been minimized.

Copy link

tomasdev commented Oct 2, 2018

Also useful when your html has scripts like FB SDK or Twitter pixel tags, etc.

@quanganhtran

This comment has been minimized.

Copy link

quanganhtran commented Oct 14, 2018

Is this following use case doable with Parcel using alias?
lib.js

// needs to be processed by Parcel to resolve libraries e.g. lodash then assigned to global variable "globalLib", which is already supported by Parcel
module.exports = {
    "require": (function(x0) {
        return {
            "lodash": require("lodash"),
        }[x0]
    })
};

loader.js

const require = globalLib.require;

index.js

// output of another builder so it needs to be ignored by Parcel
const _ = require('lodash');
console.log(_.map([0, 1, 2], x => x * 2));

index.html

    <script src="lib.js"></script>
    <script src="loader.js"></script>
    <script src="index.js"></script>

It would be quite simple if file 2 and 3 is concatenated but they are kept separately, and then Parcel bundled the libraries twice in lib.js and index.js

@DeMoorJasper

This comment has been minimized.

Copy link
Member

DeMoorJasper commented Oct 14, 2018

@quanganhtran I'm pretty sure parcel will always run over require statements, therefore this isn't really possible unless require is named something else, like webpackRequire for example

@rixrix

This comment has been minimized.

Copy link

rixrix commented Oct 15, 2018

Hello, thanks for this wonderful project. It saved me about 95% of my dev/configuration time, and we're talking about hours here (or could be days). However we hit a scenario where we needed the external library (pre-compiled, we only get the compiled version - we do/work with integration stuff from 3rd party providers/systems) to be marked as external - to avoid dual transpilation, but can't find a better way - hacks? possibly but we can't afford at this time.

I guess in this case, I don't mind parcel to have an option to do minor tweaks of the build process eg. parcelignore or something like that.

I'm in the process of moving back to babel "for the meantime" but happy/looking forward to use parcel in the coming weeks (?). Otherwise happy for any good suggestions. thanks

@bard

This comment has been minimized.

Copy link

bard commented Oct 15, 2018

A dirty hack I'm using is "hiding" the scripts from parcel by loading them through an inline script:

    <script src="lib.js"></script>
    <script src="loader.js"></script>
    <script>
      function loadNonLocalScriptSync(href) {
        var f=document.getElementsByTagName('script')[0],
            j=document.createElement('script');
        // Simulate <script> faithfully
        j.async=false;
        j.src=href;
        f.parentNode.insertBefore(j,f);
      };

      loadNonLocalScriptSync('index.js');
    </script>

But yeah, for me this has been the major pain point with parcel so far.

@figalex

This comment has been minimized.

Copy link

figalex commented Oct 17, 2018

I tried @DeMoorJasper aliases workaround and it works OK for developing locally but as soon as you do parcel build to get a production build seems like it ignores the aliases and includes the whole external library again. Not sure if this is how is supposed to work or if I could be doing something wrong.

@Daniel15

This comment has been minimized.

Copy link

Daniel15 commented Oct 17, 2018

as soon as you do parcel build to get a production build seems like it ignores the aliases and includes the whole external library again.

Hmm, that's strange. I'm using that externals hack on one of my sites (https://srcbrowse.com/) and it's working fine for the production build.

Here's the code:
https://github.com/Daniel15/SrcBrowse/blob/d91a6b70060ef6d6cbdf6a3607ae5169d637d504/js/externals/prism.js
https://github.com/Daniel15/SrcBrowse/blob/d91a6b70060ef6d6cbdf6a3607ae5169d637d504/package.json#L13

@eirikb

This comment has been minimized.

Copy link

eirikb commented Nov 16, 2018

Not sure if I'm having the exact same problem, but I struggle to import modules which only have vars (global variables, no exports).
With help from @nicolo-ribaudo I've looked into writing a babel plugin to convert such modules into having an export. He wrote this astexplorer for me.
It converts

var X = {};

into

var X = {};export default X;

This plugin works, but it doesn't seem to be run in parcel, but that might be expected.
Is there any way I can force the plugin to run?

Update:

If I run my plugin directly inside parcel (by modifying JSAsset, some hacking, and changing plugin to add module.exports) it actually works, and global vars are exported. Couldn't this be a visitor built into parcel? It shouldn't cause harm, and it could run only when nothing else is exported.

Possible to test by replacing async transform()-method in node_modules/parcel-bundler/src/assets/JSAsset.js with this gist.

@jpmonette

This comment has been minimized.

Copy link

jpmonette commented Nov 22, 2018

Having the same issue has many in this thread - I need to include this file as an asset (https://cdnmc1.vod309.com/clm/release/veeva-library.js) that is exposing some functionality on the window global, but Parcel seems to tweak the output code once compiled, making it impossible to use.

Current options I found to workaround this at the moment are:

  1. Include the script using the remote address <script src="https://cdnmc1.vod309.com/clm/release/veeva-library.js"></script>
    1. This won't work if the compiled version is executed offline (which is our case)
  2. Save the asset in the codebase + update the compiled JavaScript with the original one <script src="./assets/veeva-library.js"></script>
    • Obviously a dirty workaround I don't want to go through

Seems to be the only thing preventing me from using Parcel today for production purposes versus Webpack. There are a couple of interesting suggestions in this thread to work around this, looking forward to see this added part of Parcel.

@felipemullen

This comment has been minimized.

Copy link

felipemullen commented Nov 26, 2018

A dirty hack I'm using is "hiding" the scripts from parcel by loading them through an inline script:
...

While this is a good idea, unfortunately it seems that this is still an issue when attempting to load dependencies that need to be loaded in a particular order. In my case, it is a policy to self-host all scripts, so bringing in some minified dependency that uses window (like jQuery) causes issues when bundling.

However, I still want my own scripts to be bundled. An attempt at mixing the two approaches gets me

<script>
   loadScriptSync('/scripts/jquery.min.js'); // Hides from parcel
</script>
<script type="text/javascript" src="/scripts/depends-on/jquery-lib.js"></script>

The script that depends on the lib does not encounter it due to the execution order for the dynamically loaded script happening afterwards (even though async = false)

In summary:
Some passthrough/ignore would be really useful, despite some of the mentions discussing otherwise

@bard

This comment has been minimized.

Copy link

bard commented Nov 29, 2018

I re-tested this recently and confirm what @felipemullen wrote. Either something changed since I first used the hack or my code was not really depending on strict execution order.

@antonedvard

This comment has been minimized.

Copy link

antonedvard commented Dec 8, 2018

Is there any solution to this?
Trying to include socket.io js file that obviously comes from server, how would i write it in the file and let parcel just include the script tag and not try to bundle the file ?

script(src="/socket.io/socket.io.js")

**Using PUG

@uglycoyote

This comment has been minimized.

Copy link

uglycoyote commented Dec 16, 2018

@derolf Did you find a solution to pulling in parts of Three.js that are not part of the core? That was my exact question over in a new issue (#2412), seems people on webpack are using imports-loader, e.g.

require('imports-loader?THREE=three!three/examples/js/loaders/OBJLoader.js');

where the ?THREE=three part means to pass in the local variable three in to the imported script as the name THREE

I'm a bit late to this thread so I'm having a hard time digesting whether there's any workaround that people are currently using to pass globals into imported libraries (e.g. the THREE global that's needed by OBJLoader.js). I'm figuring that since the issue is still open that Parcel doesn't directly support this.

@moos

This comment has been minimized.

Copy link

moos commented Jan 25, 2019

Possible solution to use peerDependencies in package.json to signal "external" dependencies in #2578.

Having a peer dependency means that your package needs a dependency that is the same exact dependency as the person installing your package. This is useful for packages like react that need to have a single copy of react-dom that is also used by the person installing it. (yarn)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment