Fixes race conditions around task logging #5069
Conversation
Interesting. I'm probably guilty of reusing streams across tasks without thinking about this. |
|
I noticed that both in actions/cross-multiproject and the stacktrace reported by @pshirshov in #5067 the failures happened in ivy related tasks. I think a number of these are dynamic tasks so I wonder if that has anything to do with it. |
|
CI has been failing due to |
|
I guess this started failing yesterday: https://travis-ci.org/sbt/sbt/builds/582813802 |
|
@eatkins Let's disable dependency-management/cached-resolution-classifier for now. |
We have seen failures in scripted to create the output file for streams and it has also been reported in #5067. I believe this may caused by the same stream output being initialized by multiple tasks. To fix this, I add locking on a per-file basis. There was (and is) additional synchronization on the Streams _instance_, but the per-file locks are stored in the Streams companion object so the locking should be honored no matter which Streams instance calls make. I also skip the call to IO.touch if the file already exists. I believe that this is the common case and since IO.touch was being called with setModified = false, it should be fine to skip the touch when the file exists. Prior to this change, I was able to induce the issue in #5067 in roughly 1/50 of calls to `scripted actions/cross-multiproject`. I wasn't able to reproduce after this change.
eed3si9n
changed the title
We have seen failures in scripted to create the output file for streams
and it has also been reported in #5067.
I believe this may caused by the same stream output being initialized by
multiple tasks. To fix this, I add locking on a per-file basis. There
was (and is) additional synchronization on the Streams instance, but
the per-file locks are stored in the Streams companion object so the
locking should be honored no matter which Streams instance calls make.
I also skip the call to IO.touch if the file already exists. I believe
that this is the common case and since IO.touch was being called with
setModified = false, it should be fine to skip the touch when the file
exists.
Prior to this change, I was able to induce the issue in #5067 in roughly
1/50 of calls to
scripted actions/cross-multiproject. I wasn't able toreproduce after this change.