When I run this drake benchmark on Windows, file.rename() is a bottleneck (second flame graph here). Would you be open to a global cache option to bypass scratch/ and write directly to the final destination? I know scratch/ prevents race conditions in the general case, but for RDS storrs, drake never tries to write to the same file from two concurrent processes.
Hmm... there is a chance this will conflict with #80 in some cases. But drake has changed a lot since then, including a couple changes that should avoid the need for atomic writes. I will investigate.
When I run this
drakebenchmark on Windows,file.rename()is a bottleneck (second flame graph here). Would you be open to a global cache option to bypassscratch/and write directly to the final destination? I knowscratch/prevents race conditions in the general case, but for RDSstorrs,drakenever tries to write to the same file from two concurrent processes.Related: #114.
The text was updated successfully, but these errors were encountered: