Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External validation with HTTP/database services #1304

Open
avernet opened this issue Oct 2, 2013 · 14 comments
Open

External validation with HTTP/database services #1304

avernet opened this issue Oct 2, 2013 · 14 comments

Comments

@avernet
Copy link
Collaborator

@avernet avernet commented Oct 2, 2013

Right now this is possible, but only indirectly:

  1. Define the service to call in Form Builder.
  2. Store the result of the service in a hidden field, i.e. a field which visibility has been set to false().
  3. Use the value of that field in the constraint expression of the field you really want to validate.
  4. Define an action that calls the service when there a value change on the field to validate.

This approach has some limitations:

  1. It is convoluted – If a number of fields need to be validated this way, as many services, actions, and hidden fields need to be defined and maintained.
  2. It isn't well-suited to validating multiple fields – Often, customers would like to use the same service to validate several fields.
  3. It doesn't allow for the validation to run on save or submit – As calling the service can be an expensive operation, it might make sense to just do it on save or submit, not on value change.
@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented May 6, 2014

This comment contains ideas moved from the wiki.

Rationale

Currently, Form Runner only supports "internal" validation, that is validation expressed with types and constraints (like in plain XForms).

Often, you need external validation, which might involve complex calculations or lookups that are hard or inconvenient to implement with XForms. E.g.:

  • credit card number/expiration is validated against an external service
  • a value must belong to a large, changing set of data stored into a database
    etc.

A possible way

A possible way of implementing this is documented in this how-to: Perform external validation.

This works in limited cases but has limitations, including:

  • attributes are not supported
  • annotations pollute the document to validate

These limitations are likely not acceptable for Form Runner as we want this feature to be future-proof.

New thoughts

Requirements

  • supports attributes
  • does not pollute the XML data
  • future-proof in Form Runner

Lifecycle

With internal validation, data is frequently revalidated, typically whenever a field changes.

With external validation, this by default might be too costly. In that case, external validation:

  • takes place upon explicit user action, such as "Save", "Submit" or "Refresh"
  • upon the user changing the value of an externally-invalid field, the field is marked as valid again, until the next external validation takes place
    Internal validation can still be applied incrementally.

Service

Validation information is provided by a service:

  • in: instance data as XML
  • out: invalidity information if any
    • invalid nodes
    • custom error messages

The exact format of the response is TBD, but could be:

  • annotated XML instance data
  • ⇒ drawback: attributes not supported in an elegant way
  • list of invalid paths using canonical XPath, e.g. /foo[1]/bar[3]/@gaga
    • ⇒ drawback: need to produce and evaluate such expressions

Persistent/imperative MIPs

In XForms, Model Item Properties (MIPs) are declarative, so don't need to be "persisted" in an instance DOM as they can be recomputed as needed.

With external validation, we might need to introduce a type of persistent, imperative MIPs, settable with an action:

<xxf:setmip ref="foo/bar" name="valid" value="false()"/>

and readable as an XPath function:

xxf:getmip($name as xs:string) as item()

An action to clear all MIPs, or all MIPs of a given type, might be needed:

 <xxf:clearmip ref="instance()" name="valid"/>

NOTE: Persistent MIPs do not replace declarative MIPs: they combine with them.

When nodes are removed from an instance, these MIPs are automatically removed.

Impact on the XForms engine

This has the following impact on the implementation:

  • implement action
  • implement function
  • add support, on data nodes, for holding such imperative MIPs
  • add support for this to XPath analysis (parallel with setvalue)
  • serialize/deserialize this in dynamic state

Impact on Form Runner

  • property enables this feature with a service URL
  • Form Runner modified to call service when needed, parse result, and set
    custom MIPs

Impact on Form Builder

None for now, this has to be configured through properties.

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented May 6, 2014

At this point, I am not 100% sure that the "persistent MIP idea" is the best one. But the idea that external validation could be triggered by default only upon save/submit is good, and maybe configurable via independent properties or directly in the save/send processes properties.

There should probably also be ways to trigger external validation separately, maybe by adding an action once we have separated Form Builder actions from services.

As a first step, it might be ok to say the following:

  • "external validation" is a built-in feature
  • there is a new process action to trigger it
  • save/send processes can call this action
  • the action
    • sends the entire instance document
    • expects a format TBD in return
    • uses the result of the action to mark fields as valid/invalid (exact way TBD, with persistent MIPs or not)
@avernet
Copy link
Collaborator Author

@avernet avernet commented May 16, 2014

+1 from customer

@ebruchez ebruchez added this to the Consider for 4.8 milestone Jun 26, 2014
@avernet avernet modified the milestone: Consider for 4.8 Oct 17, 2014
@avernet
Copy link
Collaborator Author

@avernet avernet commented Nov 13, 2014

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Aug 31, 2016

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Aug 31, 2016

Comment on the proposal above: we could send the whole XForms instance, or possibly pass a list of control names to send. Then only those values would be sent, as URL parameters or in a application/x-www-form-urlencoded POST.

Clearly an important step is to determine the format(s) of the request/response.

@ebruchez ebruchez changed the title Allow HTTP/DB services to be used for validation Allow HTTP/DB services to be used for validation ("external validation") May 30, 2017
@ebruchez ebruchez changed the title Allow HTTP/DB services to be used for validation ("external validation") External validation with HTTP/database services May 30, 2017
@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented May 30, 2017

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented May 30, 2017

  • Re-reading the above, I am not sure we need to support validating attributes. Some of those are used by Form Runner, such as filename, size, and mediatype for attachment, but it is not a very general requirement besides that.
  • Another requirement the ability for external validation to provide error messages. Ideally, those could be in multiple languages.
  • We might want to support global error messages, although this can be implemented separately.
  • Should Form Builder allow specifying custom error messages? Scenario: external validation only marks the element as invalid, without a custom error message.
@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Jun 5, 2017

With datasets, the workaround can look like:

  • call service and store annotated XML data in a dataset
  • each control must have a constraint using data in that dataset

This is still heavy but doesn't require hidden controls. A single service call can return all validity information.

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Jun 5, 2017

Two scenarios to consider:

  • single service call to validate one or more or all controls
  • multiple service calls to validate individual controls

Both are probably reasonsable depending on the situation.

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Jun 5, 2017

With datasets:

  • add "Validate" button to form (see also #3236)
  • send the form's XML data and set a dataset via a service activated by the button
    • NOTE: This is not possible yet! What about from a process?
  • create a custom bind which validates all controls based on information in the dataset

Example:

<xf:bind
    ref="instance('fr-form-instance')//*[empty(*)]"
    constraint="
        for $name in name(.), $ds in fr:dataset('validation-dataset') return
            empty($ds/*[name() = $name][1]/[@valid = 'false'])
    "
/>

Above assuming a validation document like:

<validation>
    <name valid="false"/>
    <foo valid="false"/>
    <bar valid="false"/>
</validation

You can vary the format of the validation document.

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Jun 5, 2017

The above might work if we add the ability to set a dataset with the result of a process send() action.

@avernet
Copy link
Collaborator Author

@avernet avernet commented Mar 6, 2019

@ebruchez
Copy link
Collaborator

@ebruchez ebruchez commented Mar 8, 2019

@ebruchez ebruchez added the Top RFE label Mar 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.