Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Convert all our config files to JSON5 #1088

Open
octogonz opened this issue Feb 10, 2019 · 8 comments

Comments

@octogonz
Copy link
Collaborator

commented Feb 10, 2019

Problem

The web-build-tools repo has used .json config files for a long time. We've always put // comments in these files, because the proposition of an undocumented config file is preposterous and unprofessional. :-P

However, GitHub's syntax highlighter has the differing opinion that JSON standard (despite ironically being a REQUEST FOR COMMENTS) does not allow comments. And despite the father of JSON himself saying that comments are fine. It's making us look bad, because all our config files get red highlighting everywhere, like this:

/**
 * This is the main configuration file for Rush.
 * For full documentation, please see https://rushjs.io
 */
{
  "$schema": "https://developer.microsoft.com/json-schemas/rush/v5/rush.schema.json",

  /**
   * (Required) This specifies the version of the Rush engine to be used in this repo.
   * Rush's "version selector" feature ensures that the globally installed tool will
   * behave like this release, regardless of which version is installed globally.
   *
   * The common/scripts/install-run-rush.js automation script also uses this version.
   *
   * NOTE: If you upgrade to a new major version of Rush, you should replace the "v5"
   * path segment in the "$schema" field for all your Rush config files.  This will ensure
   * correct error-underlining and tab-completion for editors such as VS Code.
   */
  "rushVersion": "5.5.4",

Investigation

Why doesn't JSON allow comments? It was originally intended as a machine-to-machine data format. Back when JSON was proposed, XML was already the accepted standard for large config files intended to be maintained by humans. Humans authoring JSON is a relatively new usage scenario that arose mainly with NodeJS, due to its lack of a decent XML library.

It seems like the JSON standard isn't going to fix this problem. I did some research, and here's the most obvious alternatives:

  • YAML: I don't like YAML because (1) the syntax has many redundant ways to express a given thing, so I've observed people constantly having trouble remembering exactly what the different symbols mean, and (2) parsing all this YAML grammar requires a library that is large and slow, and (3) YAML doesn't have a mature schema facility; they expect you to convert YAML to JSON and then use a JSON schema validator. I will say that I do like YAML as an output format (e.g. shrinkwrap.yaml) because it is concise and easy to diff, and the library allows you to disable the weird notations.

  • Hjson: This seems somewhat popular, however the syntax seems pretty counterintuitive and error-prone to me. For example, x: ab ef is translates to "x": "ab ef", whereas x: [ab ef] does not translate to "x": ["ab ef"] or "x": ["ab", "ef"].

  • jsonc: Not very popular. Its syntax is different from Hjson, and also seemed counterintuitive and error-prone. For example hey: [this needs no commas] translates to "hey": ["this", "needs", "no", "commas"] and {a : b c : d} translates to {"a": "b", "c": "d"}. (Also the name is confusingly similar to https://github.com/json-c/json-c which is a JSON parser written in C.)

  • JSON5: Out of the gate, JSON5 has the advantage that it's not trying to invent anything new. Their idea is to carve out a JSON-like subset of the ECMAScript 5 grammar that is already standardized and familiar to everyone. It's better than ECMAScript because this is a pure data representation (e.g. that can be described by a conventional JSON schema), and it can be read and rewritten with a relatively simple parser. Conveniently, the @microsoft/node-core-library JsonFile API is based on jju which already supports JSON5. (I didn't realize this, but you can already put JSON5 notation in rush.json and Rush doesn't complain. Oops!)

    The one concern I had with JSON5 was that ECMAScript is continually evolving: Their file extension is .json5 -- will we have to come back every year and rename all our file extensions to .json6 or .json7 or .json8 with each new JavaScript standard? Nobody does that with .js files. I opened an issue to ask about this, and they had a reasonable argument that newer ECMAScript versions have relatively little to contribute to the syntaxes relevant to JSON.

Proposal

  1. Let's convert all our config files to use JSON5.
  2. Let's rename the file extension to .json5 so GitHub highlights it correctly
  3. Let's fix the places where we write .json with non-JSON content (e.g. issue #996)

What do you think? Comments welcome...

@mike-north @hogmoru @iclanton @daspek @patmill @dzearing @cliffkoh @kenotron

@octogonz

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 10, 2019

BTW I noticed that VS Code's JSON language service supports comments using a library called jsonc-parser that accepts "JSON with JavaScript style comments" and no other syntax extensions besides that. (Not to be confused with jsonc which I mentioned above.)

@octogonz

This comment has been minimized.

Copy link
Collaborator Author

commented Feb 10, 2019

If you want JSON5 highlighting in VS Code, apparently it's not built-in. You have to install the JSON5 syntax extension. If you want this highlighter to operation on file with the .json file extension, you can add this to your VS Code settings.json:

"files.associations": { "*.json": "json5" }
@daspek

This comment has been minimized.

Copy link

commented Feb 10, 2019

How big of a problem is the syntax highlighting in github? I would think that changing our file extension would be more confusing than leaving things as they are.

This problem can't be unique to us. I just perused the repos of a handful of popular open source projects. Most had comments in their .json files somewhere, complete with syntax highlighting errors. None had a json5 or jsonc or hjson file.

@octogonz

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 11, 2019

How big of a problem is the syntax highlighting in github? I would think that changing our file extension would be more confusing than leaving things as they are.

VS Code is now following GitHub's convention and displaying rush.json with tons of red underlines everywhere. It seems like the community is deciding that "JSON is for machines," and we really should be using a different file format for human-editable config files.

JSON5 has all the right characteristics:

  • It preserves the character of JSON, and adds only a minimal set of enhancements
  • The enhancements are all 100% standardized by ECMAScript 5
  • The "5" in the file extension protects against further feature creep (if they make a JSON6, we'll probably choose not to adopt it, because JSON5 is good enough)
  • JSON5 is already fully implemented by our JSON parser and works with our existing schemas
  • There's a growing community of JavaScript developers adopting it
@daspek

This comment has been minimized.

Copy link

commented Mar 11, 2019

I haven't personally seen evidence of the growing community of developers adopting json5. I do believe that most folks agree with the spirit, but i have never seen a "json5" file in the wild. and according to google trends there doesn't seem to be any appreciable movement. also reference this comparison of xml, json, and json5 over the last 5 years.

i still believe that the disruption from changing our file extension far outweighs any of the benefits.

@octogonz

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 11, 2019

I think my "stake in the ground" is that my config files are going to have comments in them. And the syntax highlighters are not going to make them appear to be full of errors.

So it seems like the options are:

  • change the file extension to a JSON variant that allows comments
  • abandon JSON entirely
  • change VS Code and GitHub to accept comments in JSON files (both a matter of persuading some people at Microsoft)

I think I'd be okay with any of these options. Which one are you advocating?

@octogonz

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 30, 2019

@zkochan just posted a Twitter poll announcing that PNPM will support package.json5. :-)

@chaseholdren

This comment has been minimized.

Copy link

commented Aug 11, 2019

.gitattributes file contents:

*.json                       linguist-language=JSON-with-Comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.