Missing job validation as per latest changes

[TimothyAsir]

SDS job validation is missing right now due to lot of changes in the schema.
Also there are unit test failures in sds job validation module to be fixed.

Currently sds-validation does not support any parameter to be specified without the object type.
For example if the parameter is specified as Volume.brickdetails or Node.Node it will validate; where as, sds-validation skips the parameter check if the particular parameter is specified without the parent object like Node or Node[] which may have a single element or a list of elements

[brainfunked]

Is this something that needs a specification or is it a bug fix on the existing codebase?

[TimothyAsir]

Currently sds-job validate module validates only one level. The functionality for sub objects or regressive (nested) object validation is not found. So I thought that, this may be little more then a bug fix. We can consider this as an enhancement.

[brainfunked]

Well, in that case, I would suggest that we have a specification regarding how the validation should be done properly. Essentially, the complete strategy for validating flows, including how to test the validations.

[TimothyAsir]

Required changes:

1. Update validation based on schema change
   As per the latest change in the schema, the following changes are required in the validator module
   tag changes:
   a) name space is introduced in the recent schema. The validation should use
   the name space during validation
   b) object_details is renamed to objects
   c) value attribute is newly added for an object. This should be validated

2. Currently the sds-job-validator can identify an object only when it has the full name. It does not support any parameter to be specified without the object type. Some object can be defined directly without specifying its parent name. Means, any mandatory or optional parameter can be specified with out parent name for the whole object. i.e:- parent_object_name = parameter_name
   For example if the parameter is specified as Volume.brickdetails or Node.Node it will validate; where as, sds-validation skips the parameter check if the particular parameter is specified without the parent object like Node or Node[] which may have a single element or a list of elements

3. Currently sds-job-validator validated the integrity of a flow and atoms involved for a particular job. If a flow or atom is used multiple time or if an atom is used by many flows then the same peace of code will be executed again and again. This should be enhanced properly to check the sds schema only once and only the input values should be validated every time.

4. Currently it does not support the input parameter to be case sensitive. Because it converts the custom data type (class names / object names etc) into lower case during validation. This may cause "custom object not defined" error if the object is defined with case sensitive (snake case / camel case / upper case)

5. Currently the sds-job-validator does not support the input parameter to have a nested input parameter (dict of dict of … / object of objects). Does support only one level. Which will not help in case of input parameters have multiple sub parameters. For example in case of importNode the parameter can have nested input in addition to its regular inputs like Node: [ { fqdn, ...} ], Disk [ of devices [of type ...] ] and so on. Sds-job-validator should travel into the leaf element in the chain and checks its parameter type recursively for every items and checks the whole object.

6. Currently few sds-job-validation test cases test cases could not succeeded due to the sds-schema changes to introduce name-space details. The test cases should be updated based on the namespace introduction.

7. Currently validation is not enabled in some components (bridge). This should be enabled.

Test the validations:
i) provide different schema
Should return appropriate message for the given schema
ii) pass different input (type) / missing input values
Should return appropriate message or exception for the given input thru ui / thru api
ex:- create volume with invalid / missing inputs