-
Notifications
You must be signed in to change notification settings - Fork 585
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New HCL and JSON parser in pure Go, printer support #45
Conversation
Convert JSON parser to pure Go
The README hasn't changed much, and there aren't too many adjustments to the test fixtures... so I assume that the actual definition of HCL hasn't changed much. I haven't looked too deeply at the changes yet.. It would be great if you could implement an hcl2json (and back) tool on top of these changes -- which, presumably is easier with the new AST et al. It would also be nice to see more 'round trip' tests if you haven't added them already -- eg, going from JSON to data to HCL to data to JSON and ensuring that the output at the end is the same. |
@virtuald That's correct, the syntax of HCL is completely the same. The API has changed if you were using this library, though. |
And regarding |
It seems like those types of ambiguities should be in the README file. As the maintainer of the python implementation of HCL, I suppose that's directly in conflict from what I want from hcl then -- namely, I like to think about HCL as a replacement (ish) for JSON that isn't YAML that has really convenient constructs for creating nested structures and supports comments. This of course is important for me because I just take HCL and turn it into a python dictionary -- which is basically JSON already. Having an unambiguous (canonical?) way to translate HCL -> JSON in a one-to-one way would be convenient for that use case. |
Unfortunately due to the syntax of HCL it is simply impossible. A basic example: HCL:
Can be represented in JSON as:
The latter representation is because in HCL that block can be repeated multiple times to make a list of them. The former is allowed in JSON because it feels right as a JSON user, you expect it to work. It is a shortcut for the 2nd form. I agree we should put this in the README. The goal of HCL isn't to have a 1:1 mapping between HCL/JSON: it is so that your configuration can be represented semantically equivalent in either format, to allow humans and machines to interoperate with software. Or, to just allow JSON where it is convenient. The idea is that you should be able to parse HCL and also magically get JSON support. It isn't a goal that the JSON has to be a strict representation in one structure, hence multiple are allowed above. |
Doing it! |
New HCL and JSON parser in pure Go, printer support
👍 Awesome step. I'm at DotGo, didn't had the time to look. |
This is an API backwards incompatible change. - We've actually never done this with any of our libraries, but given HCL is a fairly HashiCorp-specific library, we're going to do this. Plus, the AST is vastly superior to what we had before, and this is an important change going forward. Sorry, downstreams. Really, really sorry.
This brings in @fatih's awesome work rewriting the HCL parser into pure Go in the style of the Go parser package (
go
in the stdlib). This also addshcl/printer
which lets us output HCL!I built on top of @fatih's work to make the decoding work with the new AST as well as rewrite the JSON parser into pure Go as well in the same style, so there aren't two differing styles within the repository. The JSON parser still decodes directly into the HCL AST.
This also vastly improves the HCL AST generated by the JSON parser. In every case we have (all our projects), we no longer need to special case certain AST weirdness caused by JSON. Everything just works with the HCL/JSON interoperability. This is a huge win, and thanks a lot to the AST generated by @fatih's work.