-
Notifications
You must be signed in to change notification settings - Fork 757
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dnsjava NIO selector thread stuck at 100% after terminating job #425
Comments
Update: this bug was introduced with version 3.4.0-20210617, and is present in all versions after that |
Confirming I can reproduce this in 20210803. Hitting shift-H in top shows it's the
|
Poking this with a debugger a bit it appears select returns immediately because the thread was interrupted. dnsjava's runSelector() code never clears the interrupted flag so it just busy loops calling select. Looks like dnsjava NIO selector ends up in in ToePool.getToes() which presumably means ToePool.shutdown() is interrupting it. One workaround might be to have ToePool check the thread name and exclude it from interrupting. As the dnsjava selector thread is global per process it seems wrong that it ends up in the ToePool thread group at all. So perhaps it'd be better to prevent it from being assigned to the group in the first place. I guess one way to do this would be to do a dummy lookup on startup from a thread that's not in a group. |
Using the latest version (20210803) and a lot of versions before that, when the job is terminated, one CPU thread seems to be stuck at 100% doing nothing. This never goes away until I restart Heritrix.
For reference, this doesn't happen with version 20200304, for example. I haven't tried all versions, so I don't know when this problem started. There's also nothing in the logs that would indicate something is wrong.
Using default crawler-beans with a set operator URL and any seed you like.
The text was updated successfully, but these errors were encountered: