Options for preserving comments when editing a JSON5 file by script

[msikma]

I really like the improvements that JSON5 have over regular JSON. Out of personal interest I've been looking into whether I can use this for configuration files that are touched both manually and automatically. An important requirement is that it's possible for everything, e.g. comments and lines split by escaping the newline, to be preserved on save losslessly.

As far as I can tell, this cannot be done with this library. When I run JSON5.parse(), it returns a plain object. What I want would probably require a parser more similar to how XML parsers work, iterating with more complex objects that contain metadata. That way I could iterate past everything I need, add a new node somewhere without disturbing the rest of the data, and write back the file without any of the formatting or manual choices being altered.

Are there any options for this at the moment? Or is anyone working on such a parser? Or better yet, maybe I'm mistaken and there is a way to do this with this library.

[jordanbtucker]

A JSON5 reader is a good idea. For version 3 of this library, I'm working on moving the stateful lexer to its own class. This will help facilitate an XML reader style class for JSON5. I don't have an ETA on when version 3 will be out. Version 2 is mostly internal changes and drop for support for Node.js 4.

[msikma]

@jordanbtucker I was wondering, is there any update on v3 of this library? I'm still available to work on the kind of parser I mentioned.

[fabiospampinato]

I'm thinking maybe there could be a json5-ast library, which would provide the following methods:

• parse: generate an AST out of a JSON5 string.
• stringify: generate a JSON5 string out of an AST.

So that preserving comments when updating an object could be implemented this way:

1. Call AST.parse to get the AST of the current object/string.
2. Make a patch object, which when merged with the current object will update it the way we want it. Basically if our API call looks like this: .set ( 'key', true ) then the patch object would look like this: { "key": true }.
3. Call AST.parse on the patch object.
4. Merge the 2 ASTs together, this would merge our patch while preserving comments.
5. Call AST.stringify on the merged AST to get string we wanted back.

Does that sound reasonable? Being able to get an AST from existing objects/strings may enable other features as well, and it's a pretty generic solution.

Maybe merging the two ASTs wouldn't be that easy though 🤔

What do you guys think?

[IlCallo]

Hey there, seems like we hit exactly the same need to programmatically update/merge JSON5 (and similar) files.
Our case is about updating VSCode settings.json (and potentially other JSON with Comments stuff): quasarframework/quasar#6582

I took a look to TS and Angular custom parsers implementation. The first one is only meant to be used to read stuff, the second one is really too much complex.

[jordanbtucker]

Thanks for the suggestions. While I like your ideas, I don't plan on adding any more features to the JSON5 parser at this time. In an effort to keep JSON5 simple, this library will be a reference parser rather than a full-featured, production-ready implementation of a JSON5 parser.

[msikma]

That's a shame, but understandable.

I wouldn't mind helping out with a JSON5 AST parser as a separate project, if anybody else here is interested, but I've personally never worked on such a project before. I assume a good way to write one would be to use a library that allows a formal grammar to be specified for parsing and stringifying. Maybe someone else who was interested in this has a better idea of what the options are for such a project.

[jordanbtucker]

@msikma See #194

[giacomorebonato]

Golden Fleece provides the patch function for editing JSON5 while preserving comments.

Would it be appreciated to transfer that implementation to JSON5?
It would be useful and maybe receive more maintenance, what do you think?

[jordanbtucker]

@giacomorebonato Thanks. I've added golden-fleece to the Wiki.

I plan on releasing JSON5 v3 after April, when Node.js v12 is no longer supported, however it will mostly include performance improvements and refactors.