Add scheduler support for Local server

[aritzbastida]

Currently, there is only support for PCF Scheduler in Spring Cloud Data Flow. As a result, using any other runtime will raise exceptions when trying to use the scheduler "feature toggle".

Are there any plans to add Local Server scheduler support to the mix? It would be fine either to integrate it with Spring Scheduler or Quartz. Currently, the only options for scheduling with the Local server are to do it externally (maybe a cron process launching through the shell) or using triggertask. But both options are quite convoluted, comparing to the "Schedule" button in the Dashboard...

[sabbyanandan]

Hi, @aritzbastida. We are currently working on Kubernetes implementation. There's plan to add a Local implementation in the future, though.

In the meantime, it'd be great if you can contribute it to the project. You can use the CF and K8s as reference implementations of the SPI. We would love to work with you by reviewing the PR and guide you in the process.

There's an empty repo for it - we welcome PRs!

[aritzbastida]

Thanks for your feedback, @sabbyanandan! I'm on holidays for the next three weeks (honey moon!), but I'll try to contribute to the Local implementation when I come back.

Just to be sure, Quartz integration is preferred over Spring's internal TaskScheduler, right? Quartz allows to persist the scheduling information, so that would make sense.

[Manokethan]

Hi @aritzbastida and @sabbyanandan.

i'am interessed by the project quartz integration.. I will try this weekend to contribute to the project. i got some ideas in mind, do you advise to launch the scheduled task by the LocalTaskLauncher from spring-cloud-deployer or via rest api ?
.
Advance thanks

[sabbyanandan]

@aritzbastida / @Manokethan: Great! Thanks for the continued interest in SCDF.

I'd suggest looking at the K8s-scheduler implementation for reference. It uses the Fabric8 client to perform the REST operations [see example]. Likewise, the CF-scheduler implementation uses PCF-java-client.

[Manokethan]

Hi @sabbyanandan,

As promised, I worked on quartz integration in SCDF. You can see the result in my repo: https://github.com/Manokethan/spring-cloud-scheduler-quartz

I am a beginner in the world of spring. It is also the first time I contribute on github. Could you review the code and guide me in the process?

Hoping that the implementation follows your ideas.

Thank you beforehand.

[sabbyanandan]

Great. Thanks for the update! Before we start to review the work, please make sure you follow the contributor guide, so the subtle things are taken care of right away.

Once when you're ready with it, please submit a pull-request against spring-cloud-scheduler-quartz repo.

[Manokethan]

Thanks for your quick answer. I tried to make a pull request but he redirects me to the main page of the repo showing that it is empty...

[sabbyanandan]

Fixed that. Please retry.

[Manokethan]

thanks, it worked. I submitted a pull request spring-attic/spring-cloud-scheduler-quartz#1

[rtrompier]

Hi guys,
Any news about this feature ?
Thx ;)

[cppwfs]

Our team has discussed this addition to the Spring Cloud Data Flow project. At this point we will not be supporting a local scheduler..

[deshanshehntha]

Hi, Are there any updates for this issues ? Still no scheduler for SCDF local ?

[f3l1xss]

I find it quite baffling for this seemingly basic feature to be missing

[sabbyanandan]

Hi, folks. As you might have read in the docs or StackOverflow threads, the local impementation of the SCDF is recommended for dev/sandbox experimentation, and there are no resiliency patterns backed into the local deployment experience. Scheduling happens to be one of the critical components that you'd expect to have non-functional requirements baked-in, and we didn't want to add support for it and not meet the expectations fully.

The Local experience is intentionally not built with HA/FT in mind. Instead, using a platform like Kubernetes (and its native cronjob spec) would help orchestrate the scheduled launches more efficiently and predictably — the platform here takes the responsibility of resilient operations. I hope this clarifies our position.

All that said, there are few options to build and wrap a scheduler implementation that matches your desired deployment model, and we have them listed in the Microsite for reference (see under Local tab).