Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Property pattern validation #88

Closed
ewbankkit opened this issue Aug 10, 2021 · 3 comments · Fixed by #394
Closed

Property pattern validation #88

ewbankkit opened this issue Aug 10, 2021 · 3 comments · Fixed by #394
Labels
code-generation Relates to the conversion of CloudFormation schema to Terraform schema at buildtime. schema-handling Relates to retrieval and handling of CloudFormation schema at buildtime.

Comments

@ewbankkit
Copy link
Contributor

Relates #45.
Relates hashicorp/aws-cloudformation-resource-schema-sdk-go#7.

Due to regex syntax mismatches (e.g. negative lookahead) between the JSON Schema/ECMA-262 specification and the gojsonschema implementation which uses Go's re2, when downloading CloudFormation resource schemas we are currently rewriting all pattern (and propertyPattern) values to the empty string, which in effect means no validation takes place.

Longer term we should investigate less brute force workarounds.

@ewbankkit ewbankkit added code-generation Relates to the conversion of CloudFormation schema to Terraform schema at buildtime. schema-handling Relates to retrieval and handling of CloudFormation schema at buildtime. labels Aug 10, 2021
@PatMyron
Copy link

@PatMyron
Copy link

PatMyron commented Dec 23, 2021

will likely run into golang/go#7252 too

grep -E '\d{4}}' *
aws-appintegrations-eventintegration.json:          "pattern" : "^arn:aws:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$",
aws-appintegrations-eventintegration.json:      "pattern" : "^arn:aws:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$",
aws-apprunner-service.json:          "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-apprunner-service.json:      "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-apprunner-service.json:      "pattern" : "arn:aws(-[\\w]+)*:[a-z0-9-\\\\.]{0,63}:[a-z0-9-\\\\.]{0,63}:[0-9]{12}:(\\w|\\/|-){1,1011}"
aws-chatbot-slackchannelconfiguration.json:      "pattern" : "^arn:(aws[a-zA-Z-]*)?:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:        "pattern" : "^arn:(aws[a-zA-Z-]*)?:[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:      "pattern" : "^arn:(aws[a-zA-Z-]*)?:chatbot:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9_/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-chatbot-slackchannelconfiguration.json:        "pattern" : "^(^$|arn:aws:iam:[A-Za-z0-9_\\/.-]{0,63}:[A-Za-z0-9_\\/.-]{0,63}:[A-Za-z0-9][A-Za-z0-9:_\\/+=,@.-]{0,1023})$"
aws-finspace-environment.json:          "pattern" : "^https?://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:          "pattern" : "^https?://[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z0-9. ]{1,1000}$"
aws-finspace-environment.json:      "pattern" : "^[-a-zA-Z0-9+&@#/%?=~_|!:,.;]*[-a-zA-Z0-9+&@#/%=~_|]{1,1000}"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z-0-9-:\\/.]*{1,1000}$"
aws-finspace-environment.json:      "pattern" : "^[a-zA-Z-0-9-:\\/]*{1,1000}$"
aws-groundstation-config.json:      "pattern" : "^[{}\\[\\]:.,\"0-9A-z\\-_\\s]{1,8192}$"
aws-kendra-datasource.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-datasource.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-faq.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-kendra-index.json:      "pattern" : "arn:[a-z0-9-\\.]{1,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[a-z0-9-\\.]{0,63}:[^/].{0,1023}"
aws-lookoutequipment-inferencescheduler.json:          "pattern" : "^[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,2048}$",
aws-lookoutequipment-inferencescheduler.json:      "pattern" : "^[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,2048}$",
aws-resiliencehub-app.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-resiliencehub-app.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-resiliencehub-resiliencypolicy.json:      "pattern" : "^arn:(aws|aws-cn|aws-iso|aws-iso-[a-z]{1}|aws-us-gov):[A-Za-z0-9][A-Za-z0-9_/.-]{0,62}:([a-z]{2}-((iso[a-z]{0,1}-)|(gov-)){0,1}[a-z]+-[0-9]):[0-9]{12}:[A-Za-z0-9][A-Za-z0-9:_/+=,@.-]{0,1023}$"
aws-wafv2-rulegroup.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-rulegroup.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-webacl.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"
aws-wafv2-webacl.json:      "pattern" : "^[0-9A-Za-z_:-]{1,1024}$"

@ewbankkit
Copy link
Contributor Author

Until either

  1. Go's regexp package supports ECMA-262 pattern
  2. All CloudFormation resource schemas use a common subset of supported patterns

we could make some progress by checking during schema download whether a pattern is a valid Go regexp pattern.
This would involve changes to https://github.com/hashicorp/aws-cloudformation-resource-schema-sdk-go/blob/main/sanitize.go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-generation Relates to the conversion of CloudFormation schema to Terraform schema at buildtime. schema-handling Relates to retrieval and handling of CloudFormation schema at buildtime.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants