Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch PAUSE/RESUME flow option [BATCH-1962] #1633

Closed
spring-issuemaster opened this issue Feb 12, 2013 · 10 comments
Closed

Batch PAUSE/RESUME flow option [BATCH-1962] #1633

spring-issuemaster opened this issue Feb 12, 2013 · 10 comments

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Will Schipp opened BATCH-1962 and commented

Allow a job to be preconfigured (xml) to PAUSE at a given position in an execution and RESUMED later.

This can be achieved by modifying the FlowExecutor to have a function of pauseJob() and a FlowState of PAUSED. Once the state is triggered, the BatchStatus and ExitStatus are set to PAUSED/PAUSING and persisted. To RESUME the job, a the PausedJobManager.resumeJob(Long jobExecutionId) is called which resumes the same job execution in the task executor, skipping already completed steps and starting at the next in the flow.

Key changes to support this are in the new JobManager that would allow for the resetting of the JobExecution state (from PAUSED to STARTING).

NamespaceHandler changes would need to support a "step-level" <batch:pause id="someJob.pause"/> element that is referenced by a "next-step".


Affects: 2.2.0 Backlog

1 votes, 3 watchers

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Michael Minella commented

How is this different from the STOPPED state we already support?

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Will Schipp commented

the STOPPED process "restarts" with a new JobExecution whereas this proposal resumes with the same execution. the xml structure and usage is very similar.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Michael Minella commented

A couple things here:

  1. What is a JobManager (and specifically, the PausedJobManager implementation)?
  2. What is the event or trigger used to kick off the job resuming (who calls PausedJobManager.resumeJob for example)?
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Will Schipp commented

JobManager (working title) is akin to the JobOperator and is responsible for the resumption of the JobExecution and managing updates to it's state and starting it again inside a taskExecutor. This additional function could (should?) be wrapped into the JobOperator interface and SimpleJobOperator implementation.

The event may be manual (user clicks "resume" through a web interface).

User Story;

  • user has a 10 step job and puts a "pause" between step 8 and 9 to allow for 'manual' verification of data before "resuming" the job

Possible Solution;

  • listener on step 8 raises an event saying that step 8 is completed against x data
  • user receives event and 'manually' verifies information
  • user "resumes" the job to continue with step 9
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Michael Minella commented

  1. Is the JobManager really the JobOperator? This is what i would expect it to be but I want to be clear.
  2. The JobLauncher is what actually handles the passing of the job execution to the task executor. I wouldn't expect this behavior to change with this feature (i.e. job launcher would still be responsible for that work).
  3. Are you viewing this as just another transition like stop/next? Something like the following:
<step id="step1">
    <next on="exitStatus1" to="step2"/>
    <stop on="exitStatus2" restart="step4"/>
    <pause on="exitStatus3" resume="step2"/>
</step>
  1. I just found this page in the documentation: http://static.springsource.org/spring-batch/cases/pause.html We don't currently support this use case but I want to find out if we did and it was removed or if it was just planned and never implemented.
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 12, 2013

Will Schipp commented

  1. was only thinking of the JobOperator as a candidate as it has the restart() function
  2. the joblauncher can be updated to 'resume' as well, based on the logic of lastExecution == PAUSED
  3. that model would work
  4. that use case is pretty much what i'm thinking
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Jun 5, 2017

Ketan Prajapati commented

Michael Minella, Will Schipp

I can see PAUSE/RESUME case was described as a one of use case under Spring Batch 2.0.X docs http://docs.spring.io/spring-batch/2.0.x/cases/pause.html but might not be considered for out of box development.

IMO, this seems to be a basic requirement for long running background jobs. I am also looking for pause/restart support without involving STOPPED/STOPPING because it will demand to issue a new job execution and I want to avoid it.

Any plan to have this feature implemented in near by future?

My requirement

  • I want to schedule a job to start at <effectiveStartDateTime> and PAUSE job after execution goes beyond <runDurationInHours>.
  • Job should be automatically resumed next day at same time specified by <effectiveStartDateTime> and again job has to be paused once execution goes beyond <runDurationInHours>.
  • This has to be processed until job execution gets COMPLETED or job stopped externally from either Spring Batch Admin UI or command prompt.

As Will Schipp mentioned, I can make use of any scheduling task executor/timers along with JobOperator to meet this requirement at minimal cost but it will be always appreciated if we get PAUSE/RESUME support out of box from Spring Batch framework.

Looking for your input :)

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Jun 5, 2017

Michael Minella commented

Ketan Prajapati, Your use case is slightly different in that you are looking for orchestration (your bullet two with regards to restarting automatically). That is not a use case Spring Batch itself will support. The concept of the long running job as you describe has been a long running topic. A pull request on this would definitely be welcome assuming it didn't break backwards compatibility.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Jun 5, 2017

Ketan Prajapati commented

Michael Minella, appreciate your prompt response.
I gave that use case for walk through.

I can consider bullet 2 as customized orchestration as that is only difference from what http://docs.spring.io/spring-batch/2.0.x/cases/pause.html says.

Description

User launches a new job execution.

  1. Framework begins processing, and successfully executes one or more steps.
  2. At the end of a step Framework encounters condition that signals it should pause (e.g. a status flag).In my case, it will be a timer which is going to send PAUSE signal
  3. Framework gracefully exits the job execution, marking it as paused so that it can be identifed as such when asked to resume. Often the framework will also be configured to notify a user that the pause has occurred, so that some business condition can be verified manually.
  4. -User requests the job execution be resumed.- Here instead of manual resume, I am looking for automated resume which I can implement/customize on top of basic PAUSE/RESUME support.
  5. Framework picks up where it left off, ignoring steps that have already successfully executed and starting with the one after the pause.
  6. Job finishes processing and Framework marks it as sucessfully completed, just as if it hadn't paused in the first place.

Rest of other implementation is looking for BatchStatus.PAUSED and RESUME support and relevant business operations support mentioned on above link.

Under Long running job subjects, this can be one of common requirement where users can leverage PAUSE/RESUME support with provided hook points and perform orchestration to meet their custom requirements.

I know upfront, my job is going to take several days and I do not want to impact my application users in Peak time and hence every day, I wanted to run job during <off pick time> only.
Example: Everyday starting from 12 to 4 PM only.

@benas
Copy link
Contributor

@benas benas commented Jan 17, 2020

Closing this issue as the requested feature was removed in #2626 (see 5d62e4d).

@benas benas closed this Jan 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.