Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Initial stab at JSON plan output #3170
Guidelines said to submit even if it's early work, so here goes.
I'm pretty new to go, but have found it pretty easy to onboard into a build environment and navigating the codebase, good work everyone!
This relates to #2460 - specifically supporting JSON as plan output.
The particular scenario that led me to implement this went like this:
At this point, one option is to manually fill in the state file with a name and id for the resource, and terraform is then able to refresh and figure out where it got to.
My thinking was that it'd be nice to be able to machine read the plan (or the whole config perhaps?), and then consume this with a different tool that knows which fields should be unique, and can go fill out my state file based on what the plan might have acheived.
What do you think?
Hmm interesting... I've found myself wanting to parse the plans before, just to render them in a different way. So this seems useful to me.
I wonder though if it's necessary to complicate the UI with this option or whether Terraform could just switch to using a JSON plan in all cases. I believe that historically Terraform had a pretty opaque state file format too, and that was changed to JSON. As long as all the necessary info can be packed in there I'd assume it doesn't matter too much what the serialization is.
I must admit I've never looked too closely at the details of what gets saved in a plan file, though. Is there something in there that is difficult to represent in JSON?
My team recently discussed wanting to parse the plan files to present something differently to our users. We have a team dedicated to cloud infrastructure that understands terraform, but we want to roll something out to application developers so that they can easily deploy their own infrastructure. In doing so, we want to hide certain details about what terraform may be doing, or make it more understandable to an application developer. Also, for me personally, I oftentimes only care about why a resource has to be destroyed/created, which means I'd filter out the "(forces new resource)" lines. Seeing
With regards to this specific PR, I'd personally either a) use the extension on
@thegedge (and others here), I don't know if this is at all useful for you: https://github.com/jzohrab/terraform_tfstate_report
It's a rough tool that should let you create a report in ERB. Right now the ERB template is hardcoded, but it could easily be extended. (Should be easily extensible, anyway)
First of all, sorry for letting this PR languish here for well over a year.
We've gone back and forth on the idea of creating more machine-readable output for many Terraform commands. Since Terraform is still pre-1.0 software we're concerned about creating too many compatibility constraints; already since this issue was opened there have been changes to Terraform that would've required us to break a JSON format for plans, and this is likely to continue for the foreseeable future as plans are a very fundamental part of Terraform's architecture.
So it is with a heavy heart that we're going to say no to this idea for the moment, though that doesn't mean we won't revisit it as Terraform evolves and its model becomes more stable.
In the mean time, there is a third-party tool terraform-landscape that scrapes the human-oriented plan output. Of course a tool like this is very likely to be broken by future Terraform releases, since we don't consider the human-oriented plan output to be a compatibility constraint, but in a pinch this approach could be helpful for certain specific use-cases where machine-analysis of plans is required.
Thank you very much for working on this, and again sorry for the long silence without any feedback.
(I'd also like to clarify, for anyone looking back on this discussion in future, that my earlier contributions to this discussion and similar PRs were as a community contributor, before I joined the Terraform team full-time.)
To rant a little bit:
Terraform has been pre-1.0 for years now! When, if ever, will it hit 1.0? Clearly Sentinel has some capacity to introspect Terraform. If there are no plans to ever produce a stable API or tooling for parsing Terraform data in the open source tools, just say so instead of doing this dance for years on end about waiting for a 1.0 release.
This is not really a great place to talk about Terraform's overall roadmap, but I'll try to summarize the situation here: Terraform will be marked as 1.0 once we believe there will be no more breaking changes for some time, at which point we will make a promise similar to the Go 1.0 compatibility promise about what will and will not change moving forward. It is not 1.0 yet because we know there are several aspects of it that need to be reworked, based on the feedback collected in GitHub issues, and that several breaking changes will be required to get that done. We're intentionally front-loading the breaking-change work to get it done in as few breaking steps as possible, which is the main reason why the Terraform 0.12 release has been the biggest major release to date. Following that, we have some work to do on the CLI workflow that are also expected to include some carefully-considered breaking changes.
At present, Terraform Enterprise Sentinel is parsing plan files directly using its own parser that copies the behavior of the one inside Terraform, which is of course something that any other program could do too using @philips's
With that said, we are actually planning to introduce a JSON-formatted plan in a near-future release which the Sentinel integration will then transition to. We will not be able to commit to 100% compatibility with it at first, but we believe the 0.12 work has got the plan model into a good enough shape that future changes should only be minor. Then once the various new formats -- including this one -- are proven by real-world use, then we will consider them stable, document them comprehensively, and promise not to break external software that parses them per that documentation.