[Proposal] Workflow layer for custom alerts across jobs

[electricalgorithm]

Hello all,

I want to use Fredy to look for apartments to buy as investments. Fredy finds new listings very well, but I want alerts based on what a listing means in the context of other listings, not only on whether it matches a search. I would like to contribute this and want to check first if it fits the direction of the project.

Problem Statement

Today every job works on its own. A notification fires when a single listing matches a single search. There is no way to combine data from several jobs or compare a listing with the ones already stored. Users who want smarter alerts have to read Fredy's database from outside and build their own notification logic, which breaks when the internals change.

Possible Solution

Add a workflow component as a middleware layer between jobs and notification adapters. It listens to events Fredy already produces, can look at stored listings across jobs, and decides whether to push a notification through the existing adapters. This lets users define their own "power alerts" without touching the scraping or notification code.

Possible Use-Cases

1. Rental yield alert. One job collects apartments for sale, another collects rental listings in the same area. When the buy job finishes, the workflow estimates the rent per m² from the rental listings and alerts only if the expected yield is above a threshold.
2. Price drop alert. Notify when a listing that was seen before comes back with a lower price.
3. Below market alert. Notify when a listing's price per m² is clearly below the median of comparable listings in the same area.

Proposed Changes

• Emit an internal event when a job run finishes, including the new listings of that run
• Load workflows as plain JS modules from a workflows folder, each exporting a trigger and a run function
• Give workflows read access to stored listings across jobs
• Send workflow results through the existing notification adapters
• Allow enabling and disabling workflows in the config

sequenceDiagram
    participant Job as Fredy job
    participant Store as Listing store
    participant WF as Workflow engine
    participant N as Notification adapter
    Job->>Store: save new listings
    Job->>WF: job run completed event
    WF->>Store: query listings across jobs
    Store-->>WF: listings
    WF->>WF: run user workflow
    WF->>N: send power alert

Loading

Discussions

What do you think? Would that be a benefit for the project?

Thanks,
Gökhan

[orangecoding]

Hey,

the general idea is pretty good (and I had it as well in the past) however a majority of Fredy's users are no developers. They are capable of downloading/installing Fredy, but I do not expect them to be able to build a workflow file (as you call it).

That makes this feature somewhat special to a small subset of users... That plus the fact, that Fredy is already packed with tons of Features that only serves a small(er) group of (power) users makes me hesitant.

That being said, I still like the idea, however IF we go that route, it would need to be something, that can be done by a regular user. In fact I'm thinking about some sort of query builder... Not sure if this makes sense.

[electricalgorithm]

That's reasonable, I wasn't aware the main audience of the project. If they're non programmers, I believe they'd still have problems with text-based query builders. Perhaps we can find a NodeRed kinda block based builder library and use it, what do you think? That'd definitely increase the complexity of the proposal, however, it's possible to do it in milestones where the first one is making sure there's a script based workflow, and later on preparing a graph-based builder and it's intertranslater to JavaScript.

[orangecoding]

Check this release. I've decided to only include a couple of new filters.
https://github.com/orangecoding/fredy/releases/tag/31.1.0