[Rules Engine Release 1] Migrate the frontend to support the new rules engine for Merchant Rules

[JS00001]

We're in the process of migrating all of our rules into one place, with one format. Lets start by migrating merchant rules to use these new data formats and APIs

Data storage

• We have added a new onyx collection: rules_<ruleID>
• Each rule has the following structure:

{
  // For merchant rules, lets hard code this to 'policy'
  "scope": "policy" | "account",
  "scopeID": "<relevant ID based on the scope, so either the policyID, accountID, or domainAccountID>"
   // The triggers for a rule are when the relevant rules run. There must always be 1 trigger, but there can be more. For now, lets just worry
   // about CreateTransaction
  "triggers": {
    "0": "CreateTransaction",
  },
  // Filters follow the exact same AST format as search. They are infinitely extendable, however, for merchant rules, all we care about right now 
  // is the 'merchant' filter
  "filters": {
    "left": {
      "left": "merchant",
      "operator": "eq" | "neq" | "contains" | "notContains" | "gt" | "gte" | "lt" | "lte",
      "right": ["Hello world"],
    }
    "operator": "and"
    "right": {
      "left": "category",
      "operator": "eq" | "neq" | "contains" | "notContains" | "gt" | "gte" | "lt" | "lte",
      "right": ["Hello world"],
    },
  // The actions that are taken on any expenses that match the filters object above
  "actions": {
    "0": { "name": "Set", "field": "billable", "value": true},
    "1": { "name": "Set", "field": "merchant", "value": "modified merchant"},
    "2": { "name": "Set", "field": "category", "value": "category"},
    "3": { "name": "Set", "field": "tag", "value": "tag"},
    "4": { "name": "Set", "field": "reimbursable", "value": true},
    "5": { "name": "Set", "field": "vendorID", "value": "vendorid"},
    "6": { "name": "Set", "field": "comment", "value": "mycomment"},
    "7": { "name": "Set", "field": "tax", "value": { "field_id_TAX": {"externalID": "taxCode"}}},
  }
}

When a user creates a merchant rule, we should modify the filters to meet the request of the user, and we should modify the actions to include the fields the user wants to set. The triggers should always be just CreateTransaction for now

API Commands

GetRules - Gets all of the rules that a user has access to. This command takes no params. We return the rules_ collection, which SETs the value.

Note: Any rule that has a CreateTransaction trigger, and at least one Set action is an Expense Default Rule. Lets use that to determine where the rule shows up

SetRule - Sets a new rule in the database
Params

• scope - The scope of the rule. For merchant rules, this should be policy
• scopeID - The ID for the scope of the rule. For merchant rules, this should be the policyID
• ruleID - A random rand64() value for the ID of the rule
• priority - This determines which order the rules should apply in. For expense defaults, lets hard code this to 10000
• value - The value of the rule. This will be an object that looks like:

{
  "filters": // ...
  "triggers": //...
  "actions": // ...
}

DeleteRule - Deletes a rule in the database
Params

• ruleID - The ID of the rule

Issue Owner

Current Issue Owner: @Krishna2323

[Krishna2323]

@JS00001 a few things aren't clear to me before I can start:

1. Do category tax defaults migrate too? They're in policy.rules.expenseRules, but they have a CreateTransaction trigger and a Set action, so they match your definition of an Expense Default Rule. This also affects #99420.

2. Will the backend keep writing policy.rules.codingRules during the switch? If old rules get backfilled while codingRules stays populated, every rule will show up twice. Is this a hard cutover, or will both paths coexist during the beta? Also do we even have beta for this? 😅

3. Leaf nodes use op, while the branch uses operator. Our ASTNode uses operator everywhere. Is this a typo, or is this format different from search's AST?

4. notContains isn't in CONST.SEARCH.SYNTAX_OPERATORS. Is it being added, or should this be neq?

5. There's no UpdateRule. Is editing a rule just SetRule with the same ruleID?

6. right is an array, but our filter has right as a string. Does an array mean multiple values OR'd together, and can a merchant have more than one?

7. The actions list skips comment (Description) and vendorID. Both have live UI pages today. Are they being dropped, or just left out of the example?

[JS00001]

1. Do category tax defaults migrate too? They're in policy.rules.expenseRules, but they have a CreateTransaction trigger and a Set action, so they match your definition of an Expense Default Rule. This also affects Add category tax defaults to Expense defaults #99420.

Lets keep this out of scope to start with, while we get the FE working. But then yes, eventually they will be migrated to the new format

2. Will the backend keep writing policy.rules.codingRules during the switch? If old rules get backfilled while codingRules stays populated, every rule will show up twice. Is this a hard cutover, or will both paths coexist during the beta? Also do we even have beta for this? 😅

No, we will not be writing to codingRules anymore. We are doing the inverse. Currently, every call to SetPolicyCodingRule is writing to the new rules format as well as the legacy one.

3. Leaf nodes use op, while the branch uses operator. Our ASTNode uses operator everywhere. Is this a typo, or is this format different from search's AST?

Sorry, I'm used to using op haha, its operator, updated

4. notContains isn't in CONST.SEARCH.SYNTAX_OPERATORS. Is it being added, or should this be neq?

This is a new operator. We don't use it in the FE, but we support it now. We support neq and notContains

5. There's no UpdateRule. Is editing a rule just SetRule with the same ruleID?

Yes, use SetRule for this

6. right is an array, but our filter has right as a string. Does an array mean multiple values OR'd together, and can a merchant have more than one?

Yes, an array is an OR, the value of filters can be an array or a string (or a number for other filters), either of them works

7. The actions list skips comment (Description) and vendorID. Both have live UI pages today. Are they being dropped, or just left out of the example?

Sorry, comment is also a valid property. We don't support vendorID yet because thats still in progress, we will support that before this feature goes live, so lets add vendorID here as well