FetchDNS: Keep dnsjava selector thread out of ToePool #444
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #425, the dnsjava selector thread using 100% CPU because it was being created in the toe thread pool and then interrupted when the job was terminated. By doing a dummy lookup in a static block we ensure the selector thread is launched from a normal thread rather than from a toe thread.
There was an existing workaround in CrawlController for a similar problem with the dnsjava cache cleaner thread. This cleaner thread no longer exists in the current version of dnsjava but I assume we still want to set the cache size anyway. I moved this code to FetchDNS as well to keep the code that interfaces with dnsjava centralized. Since the cache is global it only needs to be configured once, not on every job launch.