feat: add support for variables in filter values

[ainouzgali]

What change does this PR introduce?

When filtering a step, allow using payload/subscriber/actor.tenant values
for example:

Payload :
Key: data.customProp
Value: {{subscriber.data.someOtherValue}}

Subscriber:
Key: data.subscriberValue
Value: {{someProp.from.payload}}

Why was this change needed?

Other information (Screenshots)

[linear]

NV-3049 Allow using variables as values in the step filters

What?

When filtering a step, allow using payload/subscriber values for example:

• Payload >
  • Key: data.customProp
  • Value: {{subscriber.data.someOtherValue}}
• Subscriber >
  • Key: data.subscriberValue
  • Value: {{someProp.from.payload}}

Why? (Context)

In some cases, the filters would love to be applied based of some other stored dynamic variables instead of hardcoded ones in the system.

Customer Notes:

We discussed the ability to filter based on passed variables, both the value and the threshold, and to allow those to be passed both on the subscriber level as well as the workflow.

To explain, look at this example:

Send a notification based on a user-configured threshold, or filters, so with that will be sent on the subscriber level a property called 

```jsx
{
	"notification_threshold_amount": 50
}

And then on the filter config in our workflow to allow the use of the above property. Similar to:

### Definition of Done

* Users are able to specify variables similar to usage in the templates using the `{{payloadData}}` syntax
* Topic, subscriber and actor data should be allowed to used aswell
* User can specify a combined value: "Text and {{variable}}", the variable will just be replaced and then the filter evaluation be performed.
</p>
</details>

[ainouzgali]

So it presented with the filters ui

[ainouzgali]

Unrelated to this PR. It was not filtering correctly a group filter. This fixes it.

[ainouzgali]

This is used in all send message usecases and now in the filters as well. Extracted this object so we fetch and build it once.

[ainouzgali]

Should this be in the value of webhook? I am not 100% certain

[LetItRock]

yes, the webhook URL could be also a dynamic value or have dynamic parameters :)

[LetItRock]

shouldn't we have for this case 0 messages?

[LetItRock]

yes, the webhook URL could be also a dynamic value or have dynamic parameters :)

[LetItRock]

From what I see the payload is a type of IFilterVariables and we are passing it also as the compileContext. But under the hood, the actual "compile context" is a destructured payload + partial type of IFilterVariables.

So I have a tiny suggestion to improve the code readability and types (not much different from what you have):

• creating a function for the common parts like subscriber, tenant, step, and actor
• then pass appropriate objects to the filter and compile context
• type the SendMessageCommand.compileContext prop

Code:

// common object that has subscriber, tenant, step, actor
const sendMessageContext = await this.buildSendMessageContext(command);

const [shouldRun, preferred] = await Promise.all([
      this.filter(command, { payload: command.payload, ...sendMessageContext }),
      this.filterPreferredChannels(command.job),
    ]);

...

const { payload, ...rest } = filterVariables;
const sendMessageCommand = SendMessageCommand.create({
   ...command,
   compileContext:  { payload: command.payload, ...sendMessageContext },
});

[LetItRock]

And I'm wondering wouldn't it be better to create this sendMessageContext even outside of the SendMessage use-case, like in the RunJob use-case, and then just pass it down? 🤔 then the compileContext will become sendMessageContext and will be always defined (now it's optional).

Types:

export class SendMessageCommand extends EnvironmentWithUserCommand {
...

@IsDefined()
sendMessageContext: {
  subscriber?: SubscriberEntity;
  actor?: SubscriberEntity;
  webhook?: Record<string, unknown>;
  tenant?: TenantEntity;
  step?: {
    digest: boolean;
    events: any[] | undefined;
    total_count: number | undefined;
  };
}

}

[LetItRock]

no need in ?? undefined part ;)