Flexible format validations

[czosel]

Form validations typically include format validations for fields things like emails, social security numbers, credit card and phone numbers, etc.

After discussing with @kaldras, we propose adding a new feature which functions similarly to #320 for this:

• Add new extension point, e.g. format-validations.py (maybe find better name to avoid confusion with validations.py)
• Add some pre-defined validations to caluma core for common stuff (e.g. email)
• Add a new query to the API allFormatValidations which will be used in the form builder UI
• Add a new field to formatValidation to TextQuestion, IntegerQuestion, FloatQuestion
• Integrate new validations in form/validators.py
• A format validation is defined by
  • a slug (e.g. email)
  • a name (translated for display in the form builder, e.g. E-Mail)
  • a definition of the actual validation rule. Open question: Would this be regex? Python and JS have different engines - I'm not sure how different they are in practice. A possible escape hatch would be (if the two engines are too different) to use node.js for the backend side.

[czosel]

/cc @winged Can you share some insights on the regex question?
/cc @open-dynaMIX

[winged]

I like the idea, however I wouldn't use plain regex, but JEXL instead.

The data structure and API/query is a good idea. We could have some built-in validators (hmm regarding filename: validators.py, whaddya think?) and let the users extend it via extension points.

Then we'd have a bunch of JEXL validators available to the users via API. Those could be executed in frontend and in backend as well. Why would I chose JEXL instead of plain regex? Maybe you have a validation of a question in the context of another question, for example your chimney height must be in relation to the overall height of the roof etc.

What's also needed is a (localized) error message to be displayed in case the validation fails, along with the possibility to add multiple validators to a field.

So, we'd have, as a summary:

• A global validations list, extendable via extension point
• Questions can have multiple validators attached (combined via AND operator)
• Validators consist of:
  • Slug / identifier
  • Name / description (localized)
  • JEXL implementation
  • Error message (localized, possibly with placeholders for the question's name)
• The JEXL code in the validator should be able to access:
  • A regex match function
  • the answer's data
  • access to the other answers in the document (possibly even parent/neighbor documents in the tree)

[open-dynaMIX]

Great idea 💡!

format_validations seems to me to be a good name for this feature.

When discussing this with @czosel, we decided to go with plain regex as a start and will switch to JEXL as the need arises.

EDIT: Or better call it format_validators.