Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: There is a not daemon thread exists when spark job finished #2336

Closed
1 of 2 tasks
Tracked by #2448
baiyangtx opened this issue Nov 20, 2023 · 0 comments · Fixed by #2337
Closed
1 of 2 tasks
Tracked by #2448

[Bug]: There is a not daemon thread exists when spark job finished #2336

baiyangtx opened this issue Nov 20, 2023 · 0 comments · Fixed by #2337
Labels
type:bug Something isn't working

Comments

@baiyangtx
Copy link
Contributor

What happened?

The spark driver will not exist when job is finished.

Affects Versions

master

What engines are you seeing the problem on?

Core

How to reproduce

Submit a spark job, and run insert overwrite.

The driver process will not exist when job finished.

Relevant log output

"pool-31-thread-1" #215 prio=5 os_prio=0 tid=0x00007f6ea5392000 nid=0x202fe waiting on condition [0x00007f6e9dda4000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000ad53afb8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Anything else

The RenewableBlocker create a thread pool which creating not daemon thead.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

  • I agree to follow this project's Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant