Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileResourceManagers inadvertently leave directories open #594

Closed
petebankhead opened this issue Aug 15, 2020 · 0 comments
Closed

FileResourceManagers inadvertently leave directories open #594

petebankhead opened this issue Aug 15, 2020 · 0 comments
Assignees
Labels
Milestone

Comments

@petebankhead
Copy link
Member

Bug report

Describe the bug

The following code in FileResourceManager leaves an open connection to the directory:

return Files.list(path)
	.filter(p -> Files.isRegularFile(p) && p.toString().endsWith(ext))
	.collect(Collectors.toMap(p -> nameWithoutExtension(p, ext), p -> p));

It was expected that the collect terminal operation would close the connection... it it seems it does not.

This causes real-world problems when running scripts for many images, e.g. see https://forum.image.sc/t/error-too-many-open-files/41628

To Reproduce
Steps to reproduce the behavior:

  1. Create a pixel or object classifier
  2. Run a script using loadPixelClassifier('Anything')
  3. Check for open files... on macOS this can be through Activity monitor or lsof

Each time the script is called, the directory containing the pixel classifiers is left open.

Expected behavior
Directories aren't left open unnecessarily.

Desktop (please complete the following information):

  • QuPath v0.2 -- most problematic on Linux, where the number of open files is limited.

Additional context
The error can be easily fixed by using the following code instead:

try (var stream = Files.list(path)) {
	return stream.filter(p -> Files.isRegularFile(p) && p.toString().endsWith(ext))
		.collect(Collectors.toMap(p -> nameWithoutExtension(p, ext), p -> p));
}

However, the code should be checked for other instances of this pattern. Also, DefaultProject should perhaps cache resource managers rather than creating them anew on each request.

@petebankhead petebankhead self-assigned this Aug 15, 2020
@petebankhead petebankhead added this to the v0.2.3 milestone Aug 16, 2020
petebankhead added a commit to petebankhead/qupath that referenced this issue Aug 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant