Graceful shutdown for @Scheduled tasks quits instantly

[membersound]

The following configuration shuts down instantly when the application is stopped.

If I'm not missing any configuration, this might be a bug?

@Configuration
public class ScheduledGracefulShutdownConfig {
        //this shouldn't be necessary at all with ' spring.task.scheduling.shutdown.await-termination'. but neither works
	@Bean
	TaskSchedulerCustomizer taskSchedulerCustomizer() {
		return taskScheduler -> {
			taskScheduler.setWaitForTasksToCompleteOnShutdown(true);
			taskScheduler.setAwaitTerminationSeconds(60);
		};
	}
}

@SpringBootApplication
@EnableScheduling
@EnableAsync
public class AppConfiguration {
	public static void main(String[] args) {
		SpringApplication.run(AppConfiguration.class, args);
	}
}

@Service
public class ScheduledService {
	private final Logger LOGGER = LoggerFactory.getLogger(this.getClass());

	@Scheduled(fixedRate = 100000000L)
	public void scheduled() throws InterruptedException {
		LOGGER.info("Starting scheduled job...");
		TimeUnit.MINUTES.sleep(5);
		LOGGER.info("Scheduled job finished.");
	}
}

application.properties:

server.shutdown=graceful
server.shutdown.grace-period=30s
spring.task.execution.shutdown.await-termination=true
spring.task.execution.shutdown.await-termination-period=30s
spring.task.scheduling.shutdown.await-termination=true
spring.task.scheduling.shutdown.await-termination-period=30s

spring-boot-2.6.6 with spring-boot-starter-web dependency.

2022-04-06 12:36:46.237  INFO 256621 --- [   scheduling-1] ScheduledService     : Starting scheduled job...
2022-04-06 12:36:49.496  INFO 256621 --- [ionShutdownHook] o.s.b.w.e.tomcat.GracefulShutdown        : Commencing graceful shutdown. Waiting for active requests to complete
2022-04-06 12:36:49.499  INFO 256621 --- [tomcat-shutdown] o.s.b.w.e.tomcat.GracefulShutdown        : Graceful shutdown complete
2022-04-06 12:36:49.506 ERROR 256621 --- [   scheduling-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task

java.lang.InterruptedException: sleep interrupted
	at java.base/java.lang.Thread.sleep(Native Method) ~[na:na]
	at java.base/java.lang.Thread.sleep(Thread.java:334) ~[na:na]
	at java.base/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446) ~[na:na]
	at ScheduledService.scheduled(ScheduledService.java:17) ~[classes/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.3.17.jar:5.3.17]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.3.17.jar:5.3.17]
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:305) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java) ~[na:na]
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Process finished with exit code 130 (interrupted by signal 2: SIGINT)

[wilkinsona]

This is the standard behaviour of Framework's scheduling support. When destroy is called on ScheduledAnnotationBeanPostProcessor, it cancels all of its scheduled tasks and interrupts them if they're running. There's no guarantee that interruption will do anything as it depends on the task implementation. When it does not do anything and the task continues to run, awaiting termination then comes into play.

[membersound]

But what's the sense of spring.task.scheduling.shutdown.await-termination if any scheduled task is directly destroyed?

So how could I then delay the shutdown if a @Scheduled method has not yet finished, for example a batch-updating database flow in a long-running for-each loop inside the scheduled method?

[wilkinsona]

if any scheduled task is directly destroyed?

They aren't destroyed. As I said above, an attempt is made to interrupt the task. That's all.

So how could I then delay the shutdown if a @Scheduled method has not yet finished, for example a batch-updating database flow in a long-running for-each loop inside the scheduled method?

If the task ignores the thread being interrupted, it will continue running. Here's an example:

@Scheduled(fixedRate = 100000000L)
public void scheduled() {
	LOGGER.info("Starting scheduled job...");
	long end = System.currentTimeMillis() + (5 * 60 * 60 * 1000);
	while (System.currentTimeMillis() < end) {
		try {
			Thread.sleep(100);
		} catch (InterruptedException ex) {
			// Reset interrupt flag and keep going
			Thread.currentThread().interrupt();
		}
	}
	LOGGER.info("Scheduled job finished.");
}

The app will now wait for 30 seconds for the task to complete:

2022-04-06 12:46:24.319  INFO 60547 --- [           main] com.example.demo.Gh30556Application      : Started Gh30556Application in 1.685 seconds (JVM running for 2.031)
2022-04-06 12:46:33.151  INFO 60547 --- [ionShutdownHook] o.s.b.w.e.tomcat.GracefulShutdown        : Commencing graceful shutdown. Waiting for active requests to complete
2022-04-06 12:46:33.161  INFO 60547 --- [tomcat-shutdown] o.s.b.w.e.tomcat.GracefulShutdown        : Graceful shutdown complete
2022-04-06 12:47:03.170  WARN 60547 --- [ionShutdownHook] o.s.s.c.ThreadPoolTaskScheduler          : Timed out while waiting for executor 'taskScheduler' to terminate

If you would like ScheduledAnnotationBeanPostProcessor to behave differently so that running scheduled tasks are not interrupted, please open a Spring Framework issue.

[membersound]

Thanks for the insight.
So it's basically a problem of my test scenario, not the functionality itself?

I changed the test method as follows, then the shutdown is delayed correctly:

public void scheduled() {
	LOGGER.info("Starting scheduled job...");
	do { } while (true);
}

[wilkinsona]

So it's basically a problem of my test scenario, not the functionality itself?

Yes.

If you have any further questions, please follow up on Stack Overflow or Gitter. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.

[isaac-nygaard-iteris]

The solution given, where we can add the following...

try {
	...
} catch (InterruptedException ex) {
	// Reset interrupt flag and keep going
	Thread.currentThread().interrupt();
}

... to code that throws those exceptions does not work. Sending SIGTERM still causes the application to close before thead pool tasks have finished running.

[jhkim-grip]

any changes? it seem spring project define it bug and fix it though
spring-projects/spring-framework#31019