New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: explicit named imports for non-JS/CSS assets #3722

Open
gaearon opened this Issue Jan 9, 2018 · 51 comments

Comments

Projects
None yet
@gaearon
Copy link
Member

gaearon commented Jan 9, 2018

Problem

We currently allow you to do this:

import logo from './logo.png';

After getting used to it, you’ll probably be comfortable with this giving you a URL.

But what about other types? For example, what should this return?

import doc from './doc.md';

Markdown source? Compiled HTML? An AST?

What about this?

import Icon from './icon.svg';

Should this give you a link? The SVG content? A React component?

The usual answer is “decide it for yourself in the configuration file”. However, that doesn’t work for CRA so we decided to treat all unknown extensions as URL imports. This is not ideal because in some cases it just doesn’t make any sense, and in others there are advanced (but still relatively common) use cases that aren’t satisfied.

Proposal

What if we allowed to user to pick what they want, from a limited supported subset per filetype?

import { url as logoUrl } from './logo.png';
import { html as docHtml } from './doc.md';
import { ReactComponent as Icon } from './icon.svg';

Named imports are checked by webpack so you’d get a compile error if you use an unsupported one.

Things that are unused will be tree shaken so if you only use e.g. HTML of Markdown files, their source won’t be bundled. Same for SVGs (whether you consume them as raw source, URLs, or React components).

Other zero-configuration tools can also adopt this approach.

Concerns

  • What do we do with the default import? Ideally I’d like to forbid it for anything other than JS/CSS because the intent is not clear for asset files (which version do you get?) We could do this with a lint rule.
  • If we make the breaking change, how do we update the consumers? We could write a codemod (it should be very simple).
  • It would be nice to coordinate this across at least a few other projects (e.g. @ndelangen Storybook). Maybe @KyleAMathews (Gatsby) @devongovett (Parcel) @rauchg (Next) would also be interested? I imagine we’ll need to write a multi-file Webpack loader for this, but I don’t see why other bundlers couldn’t adopt a similar convention.
  • Build performance: generating all possible content variations can be too slow. Ideally it would be nice if loaders had information about which import was used.

Thoughts?

@neoziro

This comment has been minimized.

Copy link

neoziro commented Jan 9, 2018

I think it is a very good thing to adopt a convention on it.

Even without talking about zero-configuration, today we can't use several Webpack loaders safely. We had the problem with svgr + url-loader. I made a workaround to solve it but it should not be a workaround.

I think we should add @sokra in this discussion, Webpack could force it by design.

@dmitriid

This comment has been minimized.

Copy link

dmitriid commented Jan 9, 2018

What do we do with the default import? Ideally I’d like to forbid it for anything other than JS/CSS because the intent is not clear for asset files (which version do you get?)

We may want to define the rules for the default as and extension of "pick what they want, from a limited supported subset per filetype".

import { url } from './logo.png'

import Logo from './logo.png'

Logo.url === url

Will this work in practice though?

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

I think we should add @sokra in this discussion, Webpack could force it by design.

I think this needs to be a grassroots effort because enforcing anything like this from webpack’s side is going to take a super long time, even if they do it.

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

@dmitriid

My thinking is that your example shouldn't build because it has a default import (import Icon). I think that if we adopt this approach we should just forbid default import and ask people to be explicit about what they want.

We could maybe keep default import for compatibility reasons for a while.

@Munter

This comment has been minimized.

Copy link

Munter commented Jan 9, 2018

If you want to foster a culture of zero config tooling, then it would be nice to come up with a convention that doesn't break module imports by default. Using the import statement for anything non-module is already forcing a specific tool chain dependency and thus a default config requirement on the developer.

I'd really love to see a syntax that was actually JS compatible by not overloading the import statement

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

@Munter I understand this sentiment but it’s not very practical in my experience. We’re trying to do the best with what we got. FWIW, my proposal is actually closer to no lock-in than syntax like raw!./file.md because at least it is possible to auto-generate file.md.js that contains those exports. So in practice it can even work on Node if you have a codegen step.

Let’s not derail this thread with a general discussion whether import for assets is a good idea. The fact is that it solves many real use cases, and people want to do it. So the question here is how to make that more ergonomic and less confusing.

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

I think in order to adopt this we’ll have to keep supporting default import for a while. This will make it possible for tools like Storybook to catch up. Then we can decide whether we want to deprecate support for the default import or not.

@dmitriid

This comment has been minimized.

Copy link

dmitriid commented Jan 9, 2018

@gaearon

I think I agree with you in principle, but I think there are potential downsides. Though as I'm writing them down, I feel like I might be wrong about some of my assumptions.

  • import * as Icon from 'icon.svg'

In the absence of a default export devs will still attempt the above, which will lead to multiple questions similar to "Why import * as React from 'react' and not import React from 'react'".

Why would dev need * from an svg? Because sometimes they just do :)

  • assumption that a default export is just an object with exposed/exported properties

Even though this assumption is incorrect due to how import/loading works, many devs (including me :) ) will assume the following code to be equivalent:

import {prop} from 'something';

// many will assume it to be equivalent to

import Obj from 'something';
const prop = Obj.prop

It doesn't help that a lot of transpiled code looks that way.

@ndelangen

This comment has been minimized.

Copy link
Contributor

ndelangen commented Jan 9, 2018

My first proposal would be to create a multi-markdown-loader that would export all the formats we want. and it could even have a default export. What this default would be exactly, I don't know.

I also see potential for this idea to take hold for other files then markdown.

import { url } from './my.svg';
import { raw } from './my.txt';
@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

Note there’s also a build performance concern: https://mobile.twitter.com/wSokra/status/950713344163446785.

@ranyitz

This comment has been minimized.

Copy link
Contributor

ranyitz commented Jan 9, 2018

@gaearon, That sounds like an awesome idea!

Supporting the default imports would only be less performant (from bundle size, and compile time aspects) and as you said, a lint rule could do the job for the depreciation period.

About the build performance problem, the loader will process all the possibilities, usually with no need, That might increase the build time.
Ideally, we would like the loader to be triggered by a combination of the extension and the named export, for example:

svg + url => use url-loader

import { url } from './logo.svg';

svg + ReactComponent => use react-svg-loader

import { ReactComponent } from './logo.svg';

I understand that there is a problem to do it in webpack because we only get what the user imported after the loaders ran.

When using static import it should be possible to get that information at an earlier stage.

@sokra, is there a plan of doing something like this?

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 9, 2018

Unfortunately @sokra says it’s not easy due to the current architecture: https://mobile.twitter.com/wSokra/status/950728052442529793

@viankakrisna

This comment has been minimized.

Copy link
Contributor

viankakrisna commented Jan 9, 2018

maybe it can be implemented as a babel-plugin?

import { raw } from './file.md';

becomes

import raw from 'raw!./file.md';

I think i've seen similar approach (rewriting imports) with babel-plugin-lodash. Can it be applied to loaders too?

@Andarist

This comment has been minimized.

Copy link

Andarist commented Jan 9, 2018

@viankakrisna it could, from the babel's perspective this is just a js syntac and it could transpile it to the custom loader format (import raw from 'raw!./file.md';)

@devongovett

This comment has been minimized.

Copy link

devongovett commented Jan 9, 2018

Yep, Parcel has had requests to get the URL for officially supported formats like JSON instead of inlining into the JS bundle.

The proposed syntax is potentially a bit problematic I think. Should it work for JS imports as well? i.e. import {url} from './my-script.js'. There would be no way to know statically whether url refers to a legit JS export or the URL to the file produced by the bundler, since the file extension is optional to imports.

In Parcel, and I think in webpack too, resolution of dependencies happens later - after loaders have already run. So in order to implement this syntax the loader would need to do its own resolution to check if it's a JS file. Also, I don't think we should treat JS files specially - you should be able to import a URL to a JS file as well.

@neoziro

This comment has been minimized.

Copy link

neoziro commented Jan 9, 2018

@devongovett the problem is already present with JS files. If I apply url-loader on JS files, I can't access the JS code itself any more.

But thinking about the JS file case, I don't know how to solve this issue. It seems more complicated than it appears.

@FWeinb

This comment has been minimized.

Copy link

FWeinb commented Jan 9, 2018

@viankakrisna
Can be done with a babel-plugin, quick test here.

@sompylasar

This comment has been minimized.

Copy link

sompylasar commented Jan 9, 2018

import { [Symbol.url] as jsUrl } from './foo.js'; — can this work? Getting a JS URL is a rare case if we have dynamic import().

RE: loaders do their own resolving, if resolving is exposed as a function (memoized somehow?), it can be reused from a loader.

@yyx990803

This comment has been minimized.

Copy link
Contributor

yyx990803 commented Jan 9, 2018

For the record, I'm interested in implementing this is the new vue-cli default configuration.

I was about to propose doing this via a babel-plugin but someone beat me to it :)

I think babel plugin is a good idea:

  • applicable to any file type, no need to implement a mega-loader or different loaders for different file types.
  • no tree-shaking needed
  • no perf concerns

What @FWeinb showed is already promising.

Note: the babel plugin is webpack-specific, implementation for Parcel probably has to be done in a completely different fashion.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Jan 9, 2018

Is there any problems with doing this via a URL Query Parameter so that it could also function in other places?

@stereobooster

This comment has been minimized.

Copy link
Contributor

stereobooster commented Jan 9, 2018

Also want to let you know about another potential use case (from svelte)

import MyComponent from './MyComponent.html';
@sompylasar

This comment has been minimized.

Copy link

sompylasar commented Jan 9, 2018

Is there any problems with doing this via a URL Query Parameter so that it could also function in other places?

If by URL Query Parameter you mean something like import foo from './foo.svg?ReactComponent' (altering the module request syntax) then it's as not standardized and webpack-specific as import foo from 'react-svg-loader!./foo.svg'.

Having a set of named exports from modules is not standardized as well, but may be easier to standardize across bundlers, also may be easier to provide type definitions.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Jan 9, 2018

@sompylasar Query Parameters are supported by browsers and servers, I'm not sure how this specific name list style of solution could be supported in those environments.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Jan 10, 2018

@stereobooster why? Query String has a different URL and can compose any type of Module Namespace shape it wants.

@stereobooster

This comment has been minimized.

Copy link
Contributor

stereobooster commented Jan 10, 2018

@bmeck I'm talking about type signatures like TypeScript or Flow. I know how to create type signatures based on extension of a file. Not sure if this is possible with query params. Maybe.

@FWeinb

This comment has been minimized.

Copy link

FWeinb commented Jan 10, 2018

I had some time and did some experimenting and build a proof of concept babel plugin. It's not published and very much WIP but I liked the idea and needed to try how far this could be pushed.

Some random thoughts I had while working on this:

  • Configuration of the loaders in the webpack.config will be tedious
  • Some loaders will probably need some special options passed to work
  • Maintaining a list of valid file extensions here is not optimal.
  • I don't know if the proposed syntax is as intuitive as a I previously though
@michael-ciniawsky

This comment has been minimized.

Copy link

michael-ciniawsky commented Jan 11, 2018

loader.js

const path = require('path')
const { getOptions, interpolateName } = require('loader-utils')

module.exports = function loader (src, map, meta) {
  const options = Object.assign({}, getOptions(this))

  const ctx = this.rootContext || this.context // (webpack 4 || webpack 3)
  const url = interpolateName(this, '[hash].[ext]', {
    content: src,
    context: ctx
  })
  const file = path.relative(ctx, this.resourcePath)

  // meta is/can be set by a previous loader
  const isAsset = options.isAsset || meta.isAsset || false;
  const transform = options.transform || false;

  if (isAsset) {
    // Raw {Buffer} (Loader Mode)
    module.exports.raw = true;

    this.emitFile(...)
  }
  
  if (transform) {
    src = transfrom(src)
  }

  const result = [
    `export const url = __webpack_public_path__ + '${url}'`,
    `export const file = '${file}'`,
    '',
    isAsset ? `export default \`${src}\`` : false
  ].filter(Boolean).join('\n')

  const cb = this.async()

  cb(null, result)

  return null
}

webpack.config,js

[
  {
    test: /\.txt/
    use: [  'loader' ]
  },
  {
     test: /\.md/
     use: [  
       { 
         loader: 'loader'
         options: {
            transfrom (src) {
              return markdown.render(src)
            }
         }
      }
     ]
   },
   {
     test: /\.svg/
     use: [  
       {
         loader: 'loader', 
         options {
           isAsset: true 
         } 
       },
       'svgo-loader' 
     ]
  }
]
import raw from './file.ext' // Content (raw-loader/url-loader)
import { url, file } from './file.ext' // Asset (file-loader)
import raw, { url, file } from './file.ext' // (both)
@sompylasar

This comment has been minimized.

Copy link

sompylasar commented Jan 11, 2018

@michael-ciniawsky 👍 but the version of the implementation I'm reading now (I see you editing, maybe move to a gist with version control?) doesn't let you export additional types, e.g. a React component. Also, `export default \`${src}\`` is unsafe to generate, src may contain string interpolation sequences ${...} which will be interpreted.

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 18, 2018

Related: we bumped into a problem implementing a limited subset of this

import url, { ReactComponent } from './file.svg';

The problem is that it breaks importing CSS from SVG, and even if we plug the hole by also exporting toString, an unused component is still not being treeshaken because the CSS/asset pipeline still relies on CommonJS.

If you're interesting in solving this problem, please chime in: #3856.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Jan 18, 2018

@gaearon is https://github.com/indutny/webpack-common-shake unable to shake your CommonJS?

@gaearon

This comment has been minimized.

Copy link
Member

gaearon commented Jan 18, 2018

Interesting, maybe it can. We could consider this as another option.

@rgilling

This comment has been minimized.

Copy link

rgilling commented Apr 29, 2018

Found this issue trying to isomorphically load inline md files in a react app because GA was complaining about loading the resources through isomorphic fetch. Thought I'd share my 2c on it: I got it working for my use case by using babel-plugin-static-fs which converts the node fs calls in the babel pipeline. This worked well on both the SSR server which doesn't have webpack, but is using babel-register (because I haven't managed to get ignore-styles to work with babel-cli), and the client which is using non-ejected CRA.

@lifeiscontent

This comment has been minimized.

Copy link
Contributor

lifeiscontent commented May 21, 2018

@gaearon what if you just returned the urls? and then used babel macros for all the things?

turns out there's even a macro for svgr. https://github.com/evenchange4/svgr.macro. I would imagine we could do the same for markdown and others alike. e.g. https://github.com/pveyes/raw.macro

@iansu

This comment has been minimized.

Copy link
Collaborator

iansu commented May 22, 2018

We considered using babel macros to implement this feature. However, the SVGs end up inlined in the code so if you include the same SVG more than once it will be duplicated, resulting in an unnecessarily large bundle. You can see the discussion here: #3856 (comment)

@lifeiscontent

This comment has been minimized.

Copy link
Contributor

lifeiscontent commented May 23, 2018

@iansu thanks for the post, makes sense. 👍

@ndelangen

This comment has been minimized.

Copy link
Contributor

ndelangen commented Jul 27, 2018

@iansu that should pretty much be gone after compression / gzipping though?

@iansu

This comment has been minimized.

Copy link
Collaborator

iansu commented Jul 27, 2018

Gzip would likely eliminate transferring most of that duplicate code. However, it will still increase the bundle size once decompressed and it will still need to be parsed by the client.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment