Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON "modules" #770

Closed
annevk opened this issue Oct 29, 2018 · 15 comments · Fixed by whatwg/html#4407
Closed

JSON "modules" #770

annevk opened this issue Oct 29, 2018 · 15 comments · Fixed by whatwg/html#4407
Labels

Comments

@annevk
Copy link
Collaborator

annevk commented Oct 29, 2018

At TPAC someone (sorry, forgot) brought up that import json from "./data.json"; would be neat. Should be pretty easy to branch on https://mimesniff.spec.whatwg.org/#json-mime-type and UTF-8 decode and JSON parse something...

@annevk annevk added the modules label Oct 29, 2018
@CarterLi
Copy link

CarterLi commented Oct 29, 2018

It's nice to have but we can already handle it today. See my post

The same thing can be done with scss, typescript, images (maybe convert them to base64 or data-uri), and whatever.

The idea comes from webpack (what webpack does is converting everything into a js script). All you need to do is to find a loader.

It's much more reliable to handle it at server side, you know, for browser compatibility, unless browser send a special Accept header to indicate its support.

The same problem applies to <script type=module>. Browser still sends Accept: */* for <script type=module> so <script nomodule> exists

@annevk
Copy link
Collaborator Author

annevk commented Oct 29, 2018

Perhaps you missed that this effort is about defining the (default) loader for the browser. So yes, some things can be done through a more complicated setup, but it'd be nice if in the future that could be avoided altogether.

@CarterLi
Copy link

CarterLi commented Oct 29, 2018

Perhaps you missed that this effort is about defining the (default) loader for the browser. So yes, some things can be done through a more complicated setup, but it'd be nice if in the future that could be avoided altogether.

See my edited post

It's much more reliable to handle it at server side, you know, for browser compatibility, unless browser send a special Accept header to indicate its support.

The same problem applies to <script type=module>. Browser still sends Accept: */* for <script type=module> so <script nomodule> exists

But you can't assume ./data.json is a json file. Browsers never guess file type based on its extension. That's how browsers work

Different syntax like import_json data from './data.json' may work IDK. It's a syntax error without browser support, which may end up with webpack.

In addition, It's not polyfill, compile friendly. All you can choose is to compile it into a synced xhr, which is a big NONO

@matthewp
Copy link

@annevk knows how browsers work, he's worked on them for a long time. The idea here isn't to parse specifiers ending in .json, it's to do so based on the Content-Type.

@CarterLi
Copy link

CarterLi commented Oct 29, 2018

I mean that server dare not return a json document without an browser support indicator. @matthewp

Browser wont guess what .json is so they wont send a special header

@Jamesernator
Copy link

@CarterLi Is this about the time period where some browsers support JSON-modules whereas others do not? And so might want to send the .json file to browsers that support it.

In the long term this won't matter as all browsers would eventually support the feature if it was added, but I definitely see that having browsers use an Accept header would make it simpler as new formats are added as you could just give a webpack-ed module to a browser with the header Accept: text/javascript but provide JSON directly to a browser with Accept: text/javascript, application/json.

While it only really benefits the transitional periods, it may still be worthwhile as more module formats (e.g. Wasm, HTML modules, etc) are in development so there's likely to be a long set of transitional periods for various module formats.

@andyearnshaw
Copy link

@Jamesernator I've opened the issue above to add a default Accept header of JavaScript MIME types to JS module requests, which would encourage being overridden with values that make sense once the browser supports other MIME types.

@jfbrennan
Copy link

jfbrennan commented Feb 19, 2019

Pardon my ignorance, but I'm curious why this is needed?

file-a.js

export = {
  a: 'a'
}

file-b.js

export = {
  "b": "b"
}

Importing both of those result in the same thing...doesn't it? Why would json need any special treatment? Eliminates the need for export = ? or supports the .json file extension?? Interested in learning more if someone can explain...

@jkrems
Copy link

jkrems commented Feb 19, 2019

@jfbrennan Did you mean export default { "b": "b" }? In which case the advantage of true JSON modules would be that people can use existing tools for generating and processing the files without worrying about adding or stripping a magical prefix.

@jfbrennan
Copy link

jfbrennan commented Feb 19, 2019

Yeah, both examples - export = {} and export default {} - are valid js, but not valid JSON because of the export statement.

From my limited knowledge on this topic you can already successfully import JSON. I import package.json with no problems. Although I guess I'm require()ing it and that's servers-side only...anyway, just trying learn and was curious why @annevk and others apparently see this as a need.

@rniwa
Copy link
Collaborator

rniwa commented Feb 19, 2019

Independent of whether this particular proposal is good or not, JSON files are better than JS files in terms of security because its content can't be loaded cross origin without CORS, and it's a CORB protected MIME type.

If, for example, the JSON contains information that should be only visible to a logged in user, for example, then it's not sound to turn it into a JS file without a Cross-Origin-Resource-Policy header because such a file could be loaded cross origin without CORS.

@annevk
Copy link
Collaborator Author

annevk commented Feb 20, 2019

My primary motivation for this is convenience. If you want to use a JSON resource as some kind of input you should not have to turn it into JavaScript in order to be able to make use of it. And requiring fetch() for JSON resources instead would require significant concessions when setting up a module graph (basically cannot use the declarative form as I understand it).

@jfbrennan
Copy link

jfbrennan commented Feb 21, 2019

So import json from "./data.json" doesn't work the way you'd expect it to? With just that one line json is still a string that needs JSON.parse(json) or something? Haven't tried that before...

Ah! Never mind. It fails completely: "Failed to load module script: The server responded with a non-JavaScript MIME type of "application/json". Strict MIME type checking is enforced for module scripts per HTML spec."

Now I see. Thanks for helping explain

MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Feb 26, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
@MylesBorins
Copy link

We are exploring an implementation in Node.js in nodejs/ecmascript-modules#43

MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Feb 27, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
@annevk
Copy link
Collaborator Author

annevk commented Mar 4, 2019

FYI: this has also been discussed in the HTML Standard's repository at whatwg/html#4315 and there's now a PR at whatwg/html#4407. There's a couple minor issues still to be sorted, but thus far it seems highly likely this feature will make it.

MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 6, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 6, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 7, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 18, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 18, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 18, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 19, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 20, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 21, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 21, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 21, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 22, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 23, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 24, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins added a commit to nodejs/ecmascript-modules that referenced this issue Mar 26, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
nodejs-ci pushed a commit to nodejs/ecmascript-modules that referenced this issue Mar 27, 2019
With the new flag `--experimental-json-modules` it is now possible
to import .json files. It piggy backs on the current cjs loader
implementation, so it only exports a default. This is a bit of a
hack, and it should potentially have it's own loader, especially
if we change the cjs loader at all.

The behavior for .json in the cjs loader matches the current
planned behavior if json modules were to be standardized, specifically
that a .json module only exports a default.

Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
MylesBorins pushed a commit to MylesBorins/node that referenced this issue Mar 27, 2019
This PR updates the current `--experimental-modules` implementation
based on the work of the modules team  and reflects Phase 2 of our
new modules plan.

The largest differences from the current implementation include

* `packge.type` which can be either `module` or `commonjs`
  - `type: "commonjs"`:
    - `.js` is parsed as commonjs
    - default for entry point without an extension is commonjs
  - `type: "module"`:
    - `.js` is parsed as esm
    - does not support loading JSON or Native Module by default
    - default for entry point without an extension is esm
* `--entry-type=[mode]`
  - allows you set the type on entry point.
* A new file extension `.cjs`.
  - this is specifically to support importing commonjs in the
    `module` mode.
  - this is only in the esm loader, the commonjs loader remains
    untouched, but the extension will work in the old loader if you use
    the full file path.
* `--es-module-specifier-resolution=[type]`
  - options are `explicit` (default) and `node`
  - by default our loader will not allow for optional extensions in
    the import, the path for a module must include the extension if
    there is one
  - by default our loader will not allow for importing directories that
    have an index file
  - developers can use `--es-module-specifier-resolution=node` to
    enable the commonjs specifier resolution algorithm
  - This is not a “feature” but rather an implementation for
    experimentation. It is expected to change before the flag is
    removed
* `--experimental-json-loader`
  - the only way to import json when `"type": "module"`
  - when enable all `import 'thing.json'` will go through the
    experimental loader independent of mode
  - based on whatwg/html#4315
* You can use `package.main` to set an entry point for a module
  - the file extensions used in main will be resolved based on the
    `type` of the module

Refs: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md
Refs: https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal
Refs: nodejs/modules#180
Refs: nodejs/ecmascript-modules#6
Refs: nodejs/ecmascript-modules#12
Refs: nodejs/ecmascript-modules#28
Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
Co-authored-by: Myles Borins <MylesBorins@google.com>
Co-authored-by: John-David Dalton <john.david.dalton@gmail.com>
Co-authored-by: Evan Plaice <evanplaice@gmail.com>
Co-authored-by: Geoffrey Booth <webmaster@geoffreybooth.com>
Co-authored-by: Michaël Zasso <targos@protonmail.com>

PR-URL: nodejs#26745
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
BethGriggs pushed a commit to nodejs/node that referenced this issue Apr 5, 2019
This PR updates the current `--experimental-modules` implementation
based on the work of the modules team  and reflects Phase 2 of our
new modules plan.

The largest differences from the current implementation include

* `packge.type` which can be either `module` or `commonjs`
  - `type: "commonjs"`:
    - `.js` is parsed as commonjs
    - default for entry point without an extension is commonjs
  - `type: "module"`:
    - `.js` is parsed as esm
    - does not support loading JSON or Native Module by default
    - default for entry point without an extension is esm
* `--entry-type=[mode]`
  - allows you set the type on entry point.
* A new file extension `.cjs`.
  - this is specifically to support importing commonjs in the
    `module` mode.
  - this is only in the esm loader, the commonjs loader remains
    untouched, but the extension will work in the old loader if you use
    the full file path.
* `--es-module-specifier-resolution=[type]`
  - options are `explicit` (default) and `node`
  - by default our loader will not allow for optional extensions in
    the import, the path for a module must include the extension if
    there is one
  - by default our loader will not allow for importing directories that
    have an index file
  - developers can use `--es-module-specifier-resolution=node` to
    enable the commonjs specifier resolution algorithm
  - This is not a “feature” but rather an implementation for
    experimentation. It is expected to change before the flag is
    removed
* `--experimental-json-loader`
  - the only way to import json when `"type": "module"`
  - when enable all `import 'thing.json'` will go through the
    experimental loader independent of mode
  - based on whatwg/html#4315
* You can use `package.main` to set an entry point for a module
  - the file extensions used in main will be resolved based on the
    `type` of the module

Refs: https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md
Refs: https://github.com/GeoffreyBooth/node-import-file-specifier-resolution-proposal
Refs: nodejs/modules#180
Refs: nodejs/ecmascript-modules#6
Refs: nodejs/ecmascript-modules#12
Refs: nodejs/ecmascript-modules#28
Refs: nodejs/modules#255
Refs: whatwg/html#4315
Refs: WICG/webcomponents#770
Co-authored-by: Myles Borins <MylesBorins@google.com>
Co-authored-by: John-David Dalton <john.david.dalton@gmail.com>
Co-authored-by: Evan Plaice <evanplaice@gmail.com>
Co-authored-by: Geoffrey Booth <webmaster@geoffreybooth.com>
Co-authored-by: Michaël Zasso <targos@protonmail.com>

PR-URL: #26745
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
domenic pushed a commit to whatwg/html that referenced this issue May 17, 2019
This commit adds JSON modules as a single default export, with parse
errors checked before instantiating the module graph.

As infrastructure, this divides the "module script" concept into
"JavaScript module scripts" and "JSON module scripts". Most of the
spec's existing uses of "module script" become "JavaScript module
script".

JSON module scripts are fetched in the same way as JavaScript module
scripts, e.g. with the "cors" mode and using strict MIME type checking.
They use the Synthetic Module Record defined in
whatwg/webidl#722.

Closes #4315.
Closes WICG/webcomponents#770.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants