Validate against schema used by eclipse plugin

[stevenjack]

After speaking to a number of colleagues who use the AWS eclipse plugin it seemed clear that it was using some sort of schema to validate the CF in the plugin as people were able to get feedback about incorrect params straight away.

After digging around, I found the following:

TemplateSchemaRules.java

This links off to a json schema:

CloudFormationV1.schema

that has the complete list of resources and valid properties for them.

I think it would be extremely valuable to incorporate this in so we could use this to build up the resource types and validate the resulting templates against.

I need to check with the developers of the plugin first about how this schema is maintained and if it's something we can depend on.

[stevenjack]

Issue opened in against aws-toolkit-eclipse: aws/aws-toolkit-eclipse#14

[Pramod-Pankajakshan]

[If this helps]
I had a tough time making this plugin work [cloudscripting].
Problem occurs when we have proxy setup or when there is an html response by proxy(or internal servers) asking for business justification to connect to schema.
The fix I did was to open the schema in browser first [provide justification ]and then reopen eclipse.
Thanks.

[johnf]

I've been having a go at trying to use this to generate the DSL.

Some initial work at https://github.com/johnf/cfndsl/tree/new_types

It's sort of getting there but I'm not really happy with it.

I'm trying to be really general to try and support everything but I think that approach makes the code to hard.

Will try and find a balance next time I get to it.

[johnf]

@stevenjack see my previous comment. I've made some good progress but there is still a fair amount of work to make this work properly.

Is this something others see value in? I don't want to spend more time on it unless people think it's a good idea.

Also raise questions around Heat support as it will still have to work the old way. Is anyone using cfndsl to manage Heat or should it be deprecated?

[johnf]

@stevenjack any thoughts here?

[johnf]

bump

[gergnz]

@stevenjack bump
I'd be keen to see this happen.

[gergnz]

@kornypoet can you please weigh in on this? I'd be keen to see this happen so we don't have to maintain the types.yaml file.

[kornypoet]

I like the idea of this. Things we gain:

• No longer have to support code changes to update types.yaml
• Offload support for keeping compatibility up to date
• Potentially large support base for maintaining the schema

Things we lose:

• The schema is now an outside dependency. We would have to version against it somehow to be able to mitigate any bugs introduced.
• We are now bound to another repo's update and release schedule. Right now our turnaround on type updates is pretty quick. It might be much longer when the code is not under our control.

The best solution would be to have AWS publish their schema in a machine ingestible format as described here. That thread has dried up without a real answer though. I can go either way on this one. As long as we, and the cfndsl community know the tradeoffs.

[johnf]

My thinking along being able to add our own functionality where AWS is slow is to so say have a subdirectory of files in the same format, that we add for new features. Simply deep merge them in before processing, then blow them away once upstream includes them.

[gergnz]

We really need a true source of truth. Reading this: fungusakafungus/cloudformation-jsonschema#10 (comment) is a bit of a worry :(

[cmaxwellau]

The eclipse SDK is out of date. See issue #75.The service team is definitely aware of the need for a schema. I'll post updates there if/when I am able.

[johnf]

So based on #75 it looks like AWS are working on something.

In a couple of other threads I noticed some talk about introducing a major new version with some breaking changes. If we do want to go down this path we may want to delay or not (since timing is unknown). Or alternatively introduce some changes to the DSL in preparation.

The reason I bring it up is that when I tried to create the DSL dynamically based on the existing schema, one of the major headaches was some of the non-standard bits we have in the DSL. They add a lot of extra use cases to the code and some of them were very difficult if not impossible to support.

I'd need to dive back in to remember what the issues were but I do remember them being really painful :)

Based on others thoughts on what approach we want to take I could dive in a bit further and drag out the key points.

[gergnz]

@johnf It would be really interesting to hear about which points they are. If you don't mind taking the time to list them, that would be really great.

[kornypoet]

Moving to #257

[ebekker]

Everyone interested in this should check out and provide feedback at #264

[gergnz]

Closing in favour of AWS Cloudformation Schema