Research spike: Drools & business rules interface

[brainwane]

We need to get a better handle on the Drools interface for the PSM. Right now the only way to edit the validation and screening rules is for a programmer to edit cms.validation.drl and cms.screening.drl in psm-app/cms-business-process/src/main/resources/. We need to make it so service admins can change enrollment assessment rules/workflow without having to be programmers. James & Jason have looked a little bit at this, but we need a better sense of what it is we're aiming for when we make it configurable. What should the interface be, allow, do?

This issue is for me to document the current state of affairs and propose aspirations for the business rules interface for the MVP.

[brainwane]

We're using jBPM 5.4, which is old and which maps to Drools 5.5. They're pretty coupled; we get Drools by way of jBPM. In a later release, Drools, Guvnor (the visual rules creation/management interface), and jBPM get combined into "Knowledge is Everything". Thus I'm looking at this guidance for KIE 7 and related documentation to understand what we need to do in order to get Guvnor, or its successor the "Drools Workbench.... a full featured web application for the visual composition of custom business rules and processes" which can be embedded into another web app.

[brainwane]

We currently use jBPM embedded rather than standalone and I'm not clear what the advantages of either approach are, and whether we should switch (database considerations?).

Per the jBPM 6.0.1 release notes and this Stack Overflow answer, if we move up a version to jBPM 6.x then we will get into the world of KIE. "With the Drools and jBPM (KIE) 6 series came a new workbench" (I'm not clear on how this relates to this console). jBPM 7.0 includes an upgrade to WildFly 10, which is the version of WildFly we use.

[brainwane]

Looking a little deeper at our existing Drools files:

1. The validation ruleset cms.validation.drl is the one Jason's looked at most. It has 647 rules at the moment, and validates input as the provider goes through enrollment process. There is some duplication here between our XML files and Drools rules. To set up a set of rules for a new provider type, for example, we need to add the provider type name, the license(s) required, and whether any subspecialties are required. Adding/changing provider type name, licenses, and subspecialties all currently need both XML, database, and validation rule changes, which is pretty silly; why not consolidate this into just database changes?

2. The screening ruleset cms.screening.drl has 116 rules in it and we think it gets activated after an admin clicks approve for an enrollment. I'm not sure how many of these rules we actually need to preserve; some rules don't have a then clause (like line 812, rule 'MN Accepts LPCC License from Virginia') which confuses me; I can't figure out what a rule does when the then clause is empty. Is it used elsewhere as a kind of testing assertion? There are three rules that say in their names that they assert something, but they don't have empty then clauses....

3. Our external sources ruleset, cms.externalsources.drl, has 30 rules and it's unclear to me how many of those we will preserve, since we're revamping everything about how we're dealing with external sources.

So the main Drools ruleset that we need to preserve or port is the screening ruleset. And since the structure of our rules is not going to change as often as the content, the boxes-and-arrows flowchart maker that Workbench provides isn't going to be very user friendly. A UI that optimizes for flexibility doesn't help the user here. Instead we should provide an interface that has more in common with the filter rules interfaces of an email application or a shopping site. We should reduce the number of choices administrators need to make by giving them dropdowns and checkboxes and so on, to help them make contextually constrained filtering rules, as GMail does in its filter settings.

Even though Workbench promises us an already-built UI, UberFire (another component of KIE) gives us a framework to build our own. So we can use that.

[cecilia-donnelly]

@brainwane thank you for this! Can you leave us a link to UberFire / an example of it in use?

[jasonaowen]

As @brainwane mentioned, we have hundreds of rules checked in to the repository as text files. These rules files are included by the build in the artifact that is deployed to the application server. We have a few options around how we can suggest states change these rules:

1. Directly edit them in the repository, perhaps using the Eclipse plugin. This is what Minnesota did. It is a very simple approach, and requires no changes on our part, but does not scale to multiple states.
2. Alter our build to pull in state-specific rules files at build time. We could have a build.properties file or similar which points to the specific rules file(s) to include, and the build could copy only those rules into the application and give them a specific filename. The state-specific rules could live in a particular directory in this repository, or they could be in separate repositories, or they could be .gitignored. Some advantages to this include: it would require very few code changes, it would scale to multiple states, and it would allow states to keep their rules private; disadvantages include: states would have to redeploy to alter the rules, and we are putting the burden of managing rules files and building the application onto states.
3. Alter our application to use the Guvnor web UI to manage rules. This is the approach @jvasile and I have talked about. Using the web UI would allow for runtime rules changes, would free states from needing to manage any files or even build the application, and would allow rules editing from a browser, instead of needing a special application. However, this will require a lot of work on our part to alter the application. While it was originally written to use Guvnor (see the WebSphere deployment guide (.doc file download link)), the version of Guvnor that corresponds to the version of Drools we are using is incompatible with WildFly 10. We will need to upgrade jBPM.

I started looking at upgrading jBPM earlier this week (see #267 for some of the supporting work). The short version is that it's non-trivial. There have been significant API changes; some of the APIs we're using now are deprecated, and seem to have been removed. For now, I will continue to work on upgrading jBPM.

[brainwane]

@cecilia-donnelly This is UberFire -- I spoke with Jason yesterday and the Guvnor approach is probably more sensible.

Jason and I spent a little time yesterday thinking about whether we need to be using a rules engine; towards that end, I noted some categories that the existing rules fall into.

Screening:

• validation reporting
• assertions (provider agreements, licenses, etc. exist) - perhaps should be in the validation rules?
• "this provider/code/documentation issuer etc. has to be one of these codes/has to not be in this list" - perhaps should be in the validation rules?
• Provider Type (x) requires (y) documentation
• Practice Type (x) requirement/prohibition
• Working on a reservation -> exception to a documentation requirement
• Provider Type (x) -> risk assessment level
• Risk assessment level -> verification requirement
• field -> needs verification
• License issued by [state] -> is or is not ok based on practice/residence location/education level/provider type
• Provider type (x) with existing registration as provider type (y) -> should register or renew as (y)

The validation rules include stuff that should probably be validated at some other layer, like "birthdate should not be in future", but also:

• Provider Type (x) -> requires background study
• Rules about "beneficial owners" who do or don't have "interests"
• Provider Type (x) requires [license type]

We concluded that yeah, probably we do want to do these as business rules rather than in some other layer.

[brainwane]

This issue is for me to document the current state of affairs and propose aspirations for the business rules interface for the MVP.

We've now done that, so I'm going to close this issue. If anyone would like to open new issues more specifically focused on the jBPM upgrade, Guvnor usage, or something else, please go ahead and feel free to link to/copy from this issue history as is useful.