Skip to content

Commit

Permalink
Add news fragment
Browse files Browse the repository at this point in the history
  • Loading branch information
timj committed Oct 6, 2022
1 parent bb1e139 commit 29db3e9
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions doc/changes/DM-36412.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
If ``pipetask`` is run with multiple processes and if a butler datastore cache is configured, all subprocesses will now share the same cache.
For large numbers of simultaneous processes it may be necessary to significantly increase the number of datasets in the cache to make the cache usable.
This can be done by using the ``$DAF_BUTLER_CACHE_EXPIRATION_MODE`` environment variable.

Previously each subprocess would get its own cache and if ``fork`` start method was used these cache directories would not be cleaned up.

0 comments on commit 29db3e9

Please sign in to comment.