Transformation of Mustache parse results into parameter schema does not condense required properties from independent sections

[sectoreleven]

We are trying to use the FAST CLI as part of an internal toolchain for producing AS3 declarations absent the FAST GUI (we do not allow users direct access to our BIG-IPs). As we have begun writing FAST templates, I have found that if a template attempts to loop over a collection multiple times independently, any elements referenced in both sections get incorrectly duplicated in the relevant 'required' section of the parameter schema. This causes the template validation/load/render process to fail.

Sample failing template (not a full/valid AS3 declaration, but enough to get structure across):

title: basic-https
description: Basic HTTPS
parameters:
    appName: myApp
    poolMembers:
        memberRatio: 1
definitions:
    poolMonitors:
        type: array
        items: 
            type: object
            properties:
                monitorName:
                    type: string
                monitorType:
                    type: string
                monitorSend:
                    type: string
                monitorReceive:
                    type: integer
                monitorInterval:
                    type: integer
                monitorTimeout:
                    type: integer
template: |
    {
        "class": "AS3",
        "persist": false,
        "declaration": {
            "class": "ADC",
            "schemaVersion": "3.21.0",
            "constants": {
                "class": "Constants",
                "timestamp": "{{buildTimeStamp}}",
                "version": "{{buildNumber}}"
            },
            "id": "{{appName}}",
            "remark": "templateVersion:##templateVersion##",
            "{{appName}}": {
                "class": "Tenant",
                "app": {
                    "class": "Application",
                    "template": "generic",
                    "{{appName}}-vs": {
                        "class": "Service_HTTP",
                        "virtualAddresses": [
                            {{virtualServerIP}}
                        ],
                        "pool": "{{appName}}-pool"
                    },
                    {{#poolMonitors}}
                    "{{monitorName}}-monitor" : {
                        "class": "Monitor",
                        "monitorType": "{{monitorType}}",
                        "send": "{{monitorSend}}",
                        "receive": "{{monitorReceive}}",
                        "interval": {{monitorInterval}},
                        "timeout": {{monitorTimeout}}
                    },
                    {{/poolMonitors}}
                    "{{appName}}-pool": {
                        "class": "Pool",
                        "monitors": [
                            {{#poolMonitors}}
                            {
                                "use": "{{monitorName}}-monitor",
                            }
                            {{/poolMonitors}}
                        ],
                        "members": [
                            {{#poolMembers}}
                            {
                                "servicePort": {{memberPort}},
                                "serverAddresses": [
                                    "{{memberIpAddress}}"
                                ]
                            },
                            {{/poolMembers}}
                        ],
                    }
                }
            }
        }
    }

Sample Data:

{
	"poolMonitors": [
		{
			"monitorName": "monitor1",
			"monitorType": "https",
			"monitorSend": "GET /",
			"monitorReceive": "200.OK",
			"monitorInterval": 30,
			"monitorTimeout": 91
		},
		{
			"monitorName": "monitor2",
			"monitorType": "https",
			"monitorSend": "GET /heartbeat",
			"monitorReceive": "App.is.alive",
			"monitorInterval": 30,
			"monitorTimeout": 91
		},
	],
	"buildTimeStamp": "2021-11-17 0800H",
	"buildNumber":  "21.1.0.2525",
	"appName": "mySampleApp",
	"virtualServerIP": "10.0.0.10",
	"poolMembers": [
		{
			"memberIpAddress": "10.0.1.11",
			"memberPort": "443"
		},
		{
			"memberIpAddress": "10.0.1.12",
			"memberPort": "443"
		},
		{
			"memberIpAddress": "10.0.1.15",
			"memberPort": "443"
		}
	]
}

Results:

❯ fast validate .\basic-https.yml
failed to load template
Error: Failed to compile parameter validator
schema:
{
  "type": "object",
  "properties": {
    "poolMonitors": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "monitorName": {
            "type": "string"
          },
          "monitorType": {
            "type": "string"
          },
          "monitorSend": {
            "type": "string"
          },
          "monitorReceive": {
            "type": "integer"
          },
          "monitorInterval": {
            "type": "integer"
          },
          "monitorTimeout": {
            "type": "integer"
          }
        },
        "required": [
          "monitorName",
          "monitorType",
          "monitorSend",
          "monitorReceive",
          "monitorInterval",
          "monitorTimeout",
          "monitorName"
        ]
      },
      "skip_xform": true
    },
    "buildTimeStamp": {
      "type": "string"
    },
    "buildNumber": {
      "type": "string"
    },
    "appName": {
      "type": "string"
    },
    "virtualServerIP": {
      "type": "string"
    },
    "poolMembers": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "memberPort": {
            "type": "string"
          },
          "memberIpAddress": {
            "type": "string"
          }
        },
        "required": [
          "memberPort",
          "memberIpAddress"
        ]
      },
      "skip_xform": true
    }
  },
  "required": [
    "buildTimeStamp",
    "buildNumber",
    "appName",
    "virtualServerIP",
    "poolMonitors",
    "poolMembers"
  ],
  "dependencies": {
    "monitorName": [
      "poolMonitors"
    ],
    "monitorType": [
      "poolMonitors"
    ],
    "monitorSend": [
      "poolMonitors"
    ],
    "monitorReceive": [
      "poolMonitors"
    ],
    "monitorInterval": [
      "poolMonitors"
    ],
    "monitorTimeout": [
      "poolMonitors"
    ],
    "memberPort": [
      "poolMembers"
    ],
    "memberIpAddress": [
      "poolMembers"
    ]
  },
  "title": "basic-https",
  "description": "Basic HTTPS",
  "definitions": {}
}
compile error:
schema is invalid: data.properties['poolMonitors'].items.required should NOT have duplicate items (items ## 6 and 0 are identical), data.properties['poolMonitors'].items should be array, data.properties['poolMonitors'].items should match some schema in anyOf
    at C:\Users\myuser\AppData\Roaming\npm\node_modules\@f5devcentral\f5-fast-core\lib\template.js:682:40

The expectation here is that the items.required array would include 'monitorName' only once in this case.

[sectoreleven]

Mustache can and does appropriately handle multiple sections that loop over the same item in the view:

Working template concept

Hello {{name}}
You have just won {{value}} dollars!
{{#in_ca}}
Well, {{taxed_value}} dollars, after taxes.
{{/in_ca}}
Here's how it breaks down:
{{#breakdown}}
 - {{amount_spent}} dollars for {{on_what}}.
{{/breakdown}}

All told, you bought:
{{#breakdown}}{{on_what}},{{/breakdown}}

Working data:

{
  "name": "Chris",
  "value": 10000,
  "taxed_value": 7000,
  "in_ca": true,
  "breakdown": [
    {
      "amount_spent": 300,
      "on_what": "apples"
    },
    {
      "amount_spent": 400,
      "on_what": "peaches"
    },
    {
      "amount_spent": 200,
      "on_what": "pears"
    }
  ]
}

Produces the following:

Hello Chris
You have just won 10000 dollars!
Well, 7000 dollars, after taxes.
Here's how it breaks down:
 - 300 dollars for apples.
 - 400 dollars for peaches.
 - 200 dollars for pears.

All told, you bought:
apples,peaches,pears,

Since Mustache can handle this, it would follow that FAST should be able to do so as well, if it can produce the properly-formatted JSON schema for the FAST template.

[shyawnkarim]

Thanks for reporting this issue. We are now tracking this internally with ID MYSTIQUE-476.

[shyawnkarim]

Closing.

This issue was resolved with Release 0.17.0.