Fixed APPolicy CRD

[fabriziofiorucci]

Proposed changes

• Added the missing maskValueInLogs to cookies for app-protect
• Added the missing useXmlResponsePage to xml-profiles for app-protect
• Changed several types from string to integer in spec.policy.xml-profiles[0].defenseAttributes
• Added missing enums in blocking-settings.properties.evasions.items.properties.description.enum

Checklist

Before creating a PR, run through this checklist and mark each as complete.

• I have read the CONTRIBUTING doc
• I have added tests that prove my fix is effective or that my feature works
• I have checked that all unit tests pass after adding my changes
• I have updated necessary documentation
• I have rebased my branch onto main
• I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

[schweits]

Not sure if you can change anything, but according to the NAP schema both ints and strings are valid values for those fields. Eg.

                                 "maximumAttributeValueLength" : {
                                    "default" : "1024",
                                    "oneOf" : [
                                       {
                                          "maximum" : 2147483647,
                                          "minimum" : 0,
                                          "type" : "integer"
                                       },
                                       {
                                          "enum" : [
                                             "any"
                                          ],
                                          "type" : "string"
                                       },
                                       {
                                          "pattern" : "\\d+$",
                                          "type" : "string"
                                       }
                                    ]
                                 },

[fabriziofiorucci]

@schweits for some reason several fields apparently need to be set to integer. Applying an APPolicy manifest containing:

  defenseAttributes:
    allowCDATA: true
    allowDTDs: true
    allowExternalReferences: true
    allowProcessingInstructions: true
    maximumAttributeValueLength: any
    maximumAttributesPerElement: 32
    maximumChildrenPerElement: 4096
    maximumDocumentDepth: 32
    maximumDocumentSize: 1024000
    maximumElements: 512000
    maximumNSDeclarations: 64
    maximumNameLength: any
    maximumNamespaceLength: any
    tolerateCloseTagShorthand: true
    tolerateLeadingWhiteSpace: true
    tolerateNumericNames: true

Triggers:

The APPolicy "nginx-policy" is invalid: 
* spec.policy.xml-profiles[0].defenseAttributes.maximumAttributesPerElement: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumAttributesPerElement in body must be of type string: "integer"
* spec.policy.xml-profiles[0].defenseAttributes.maximumElements: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumElements in body must be of type string: "integer"
* spec.policy.xml-profiles[0].defenseAttributes.maximumDocumentSize: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumDocumentSize in body must be of type string: "integer"
* spec.policy.xml-profiles[0].defenseAttributes.maximumNSDeclarations: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumNSDeclarations in body must be of type string: "integer"
* spec.policy.xml-profiles[0].defenseAttributes.maximumDocumentDepth: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumDocumentDepth in body must be of type string: "integer"
* spec.policy.xml-profiles[0].defenseAttributes.maximumChildrenPerElement: Invalid value: "integer": spec.policy.xml-profiles[0].defenseAttributes.maximumChildrenPerElement in body must be of type string: "integer"

Unless I'm missing something, the manifest snippet should be correct according to https://docs.nginx.com/nginx-app-protect-waf/declarative-policy/policy/#policy/graphql-profiles/defenseAttributes

[schweits]

@fabriziofiorucci Right, integers should be valid, but according to the schema (and the doc you linked too), passing in a value like "1024000" should be valid too. But with the change here, wouldn't it reject strings no matter what?

Edit: You should be able to do something like:

maximumBatchedQueries:
  pattern: any|\d+
  anyOf:
     - type: integer
     - type: string
  x-kubernetes-int-or-string: true

[fabriziofiorucci]

@schweits as discussed, I'll check, test and add the definitive version of the CRD yaml shortly.

[galitskiy]

agree with @schweits on changing type to int; but not sure if "x-kubernetes-int-or-string: true" is a way to go either (I think there was a discussion long time ago with kic team, dont remember what we agreed on). @fabriziofiorucci also from your example of the manifest there are fields with the same type (int or string) set to "any".

[schweits]

@galitskiy I provided @fabriziofiorucci with the CRD we're using in our internal testing repo, so that's where many of those changes came from.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7fcee3d) 51.93% compared to head (c55d000) 51.92%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4640      +/-   ##
==========================================
- Coverage   51.93%   51.92%   -0.01%     
==========================================
  Files          59       59              
  Lines       16972    16973       +1     
==========================================
- Hits         8814     8813       -1     
- Misses       7861     7862       +1     
- Partials      297      298       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[vepatel]

@fabriziofiorucci Also the definitions in deploy/crds-nap-waf.yaml are generated automatically by running make update-crds from project root, so you only need to:

1. change CRDs in config/crds/bases,
2. run make update-crds

[vepatel]

approved pending comment resolution

[fabriziofiorucci]

@vepatel for some reason make update-crds throws an error:

$ make update-crds
go run sigs.k8s.io/controller-tools/cmd/controller-gen crd paths=./pkg/apis/... output:crd:artifacts:config=config/crd/bases
go: downloading sigs.k8s.io/controller-tools v0.13.0
[...]
go: finding golang.org/x/text v0.13.0
build sigs.k8s.io/controller-tools/cmd/controller-gen: cannot load io/fs: malformed module path "io/fs": missing dot in first path element
make: *** [Makefile:66: update-crds] Error 1

this happens on a Ubuntu 20.04 VM. Any clue?

[vepatel]

@fabriziofiorucci its a local golang issue it seems, make sure you've installed go version i.e. 1.21.3

>> go version                              
go version go1.21.3 darwin/arm64

Now if it fails even after update, edit Makefile job update-crds like:

update-crds: ## Update CRDs
	@kustomize version || (code=$$?; printf "\033[0;31mError\033[0m: there was a problem with kustomize, use 'brew install kustomize' to install it\n"; exit $$code)
	kustomize build config/crd >deploy/crds.yaml
	kustomize build config/crd/app-protect-dos --load-restrictor='LoadRestrictionsNone' >deploy/crds-nap-dos.yaml
	kustomize build config/crd/app-protect-waf --load-restrictor='LoadRestrictionsNone' >deploy/crds-nap-waf.yaml

i.e. basically remove https://github.com/nginxinc/kubernetes-ingress/blob/main/Makefile#L66

[shaun-nx]

Hi @fabriziofiorucci
The PR is good to go now. If you rebase your branch I'll merge the PR as soon as the pipeline passes.

[fabriziofiorucci]

reviewed