Research item: Long running batch-jobs / workflow

[alexellis]

Should we support long-running batch-jobs?

Are these in the scope of OpenFaaS functions which are typically several seconds in duration?

Are there primitives in Kubernetes such as jobs which we can leverage?

Kubernetes jobs example

Edit 29 Sept 2019

Since long-running jobs and workflows are related, I've added that to the title, so if you're looking for workflow, please feel free to comment with your use-case and whether it's for business purposes or for fun.

[alexellis]

I spoke with @iyovcheva about this yesterday. My initial thoughts are that jobs are orthogonal to functions, but they are a common ask from the community around longer-running batch processing and CI jobs. Is there an opportunity to add value here?

[cpitkin]

Nomad and Kubernetes both have these primitives but I don't believe Swarm does. It looks like Swarm may get the primitive in the near future from the issue but not sure how long that will be.

I can see this being a larger ask from the community going forward. I have been thinking a lot about this myself for personal and work-related tasks. It would be nice to have something that can be scheduled to run at a specific time but not be locked into time constraints. I also feel that it is a slippery slope to not turn the project into something like Rundeck. It may be worth waiting to see what Swarm does before building anything. Leveraging the primitives of something already build is probably going to be a smaller lift.

[alexellis]

It looks like Swarm may get the primitive in the near future from the issue but not sure how long that will be.

Having read the thread and following the thread, I think this is unlikely to happen in the near future.

Kubernetes can support this use-case natively, Swarm may need a separate controller writing to make this possible - I made a start with a CLI tool which has a few users called JaaS

The Rundeck project looks interesting btw.

[CuZn13]

There are many such scenarios in the actual business. Containers can do this kind of work well. Pull up the container when using it,Complete work to destroy the container. We are doing this based on openfaas.
When invoked, put the request in a queue and deploy function. Record the number of call requests and completions. If there is no new request in a certain period of time and all operations are completed, delete function. Wait for the next invoked

[lukasheinrich]

Hello -- just commenting here as well to share our science use-case. "Black box" functions that are expensive to evaluate are a common setup in optimization problems. Often a machine-learning based strategy defines at what parameters a function is evaluated in order to reduce the number of evaluations: see e.g. https://scikit-optimize.github.io/ or https://github.com/diana-hep/excursion function evaluations can easily be multiple hours. My ideal user-interface would be

> faas submit --parameters '{"a": 1, "b": "Hello World"}'
http://some.url/to/future
> faas ready http://some.url/to/future
false
> faas ready http://some.url/to/future
true
> faas retrieve http://some.url/to/future
{"value": 1.23}

[alexellis]

Thanks for adding your use-case.

What if instead of checking for the result in a stateful way, you specified a callback URL in an event-driven way?

Is "job checking" a hard requirement too?

I think you could do this basic flow with the asynchronous processing using a long enough max timeout. Each async function call returns a call ID which you get back on the callback.

Alex

[lukasheinrich]

Hi
-- I think a callback would work equally well. definitely not a hard requirement. Are you thinking one call back for all function calls (where the callback specifies some invocation id) or a unique callback url per call?

[sheryever]

Is there any update?

I am using csharp functions, If a function is receiving the request on single thread then I can start a new thread with some unique key and return that in response. Then on the other request i can send the key and check the status of the thread.

I didn't try to implement this until now because I am new and testing the OpenFaas for our need and don't know how it will behave to threading and searching for the solution which is already available.

I was planing to replace the windows services which with OpenFaas. These service are running the scheduled task which usually takes 3 to 10 minutes but now we also need that to run those tasks on demand.

[alexellis]

Hi @sheryever you can run for 3-5 mins, no problems. Threads are also fair game. 👍

I don't think you need what I'm calling long-running jobs for that.

Alex

[alexellis]

Some of the requirements/constraints I'm hearing from users:

• run for up to several hours
• status can be checked
• can be cancelled
• new version can be scheduled without cancelling the existing versions executing

Assumed, but need users to confirm:

• uses or looks like a function
• invoked or scheduled in a similar way to a function

A thin wrapper around a Kubernetes Job, or clear documentation on this use-case using Kubernetes jobs may be enough for a large % of the people asking for the above, but this is unclear. I think it would be worth looking into with 1-2 of the people who need this.