[feat] Dynamic jobs

[corzani]

Hi,
is there a way to create a dynamic set of jobs? How can I pass custom parameters to a job?
I was wondering if a similar approach to the following one can be used...

const currencies = ['USD', 'EUR','GBP'];
const currencyJob = currency => () => {
// job
console.log(currency)
};

const jobs = currencies.map(currency => ({task: currencyJob(currency), interval: '2m'}));
const bree = new Bree({jobs});

// start...

Thanks

[ggb88]

Just from my little investigation, would using bree.add(..) work?

[corzani]

Just from my little investigation, would using bree.add(..) work?

Hi @ggb88 , first of all, thanks...
As far as I understood, add doesn't let you specify different parameters either. In my example, every job is started with different currencies because they are treated as a function and not as an external file dynamically loaded/required.
By adding a lazy approach you can execute your task only when you need them (like the current implementation) and at the same time having the best from a functional approach (parametrized jobs, lazy loading...). Let's say you have 1000 jobs, every job differs only by its parameters and not its implementation. How would you implement it with this current implementation?
Consider my example, if you have 1000 currencies that code would work, it's not optimal regarding the interval bit... but it's working.

[corzani]

I did find a possible solution, workerData to pass parameters to the dynamically created job list. It would have been more "elegant" having functions but that would work well anyway. Thanks @ggb88

[fernandolguevara]

@niftylettuce this could be a very useful/great feature for the library, there is a better way to do this?

[tamaker]

I did find a possible solution, workerData to pass parameters to the dynamically created job list. It would have been more "elegant" having functions but that would work well anyway. Thanks @ggb88

@corzani Can you share a rough example of how you're achieving this? I'm exploring options along the same lines as your approach. Thanks!

[corzani]

@tamaker
When you create a task you need to specify also the workerData property.

    {
      name: 'task1',
      worker: {
        workerData: {
          foo: 'something',
          fie: 'something else'
        }
      }

Considering the case above you can retrieve those properties from your job (task1.js) using worker threads.

const { workerData } = require('worker_threads');
console.dir(workerData); // This is what you need

See https://nodejs.org/api/worker_threads.html#worker_threads_worker_workerdata

I did notice the following method when I was looking for an example (2 mins ago)...
Check => https://github.com/breejs/bree#using-functions-for-jobs
So, it seems you can specify a function... I didn't even noticed that when I was reading the docs... my bad :)

So currently you might have 2 ways to do it...