Sort may get stuck for some tasks. #393

Closed
pooya opened this Issue Jan 13, 2014 · 1 comment

Comments

Projects
None yet
1 participant
@pooya
Owner

pooya commented Jan 13, 2014

I have seen a scenario where for a very small input file, the sort never finished. We could see that sort started and never finished. There are some odd things about this failure:

  1. In the task directory, we could see both sort.dl file and .partial file. Since we are using the atomic os.rename this should not happen.
  2. Both sort.dl and the partial file are empty. This is odd because we saw the size of the downloaded input and it was not zero.
  3. The sort started and never finished. Running strace on this process reported that it was stuck on a futex. This issue might be a race condition in python subprocess module. (e.g. http://bugs.python.org/issue19809)

Once the sort process was killed manually, the task was restarted and finished successfully.

@pooya

This comment has been minimized.

Show comment Hide comment
@pooya

pooya Apr 9, 2014

Owner
Owner

pooya commented Apr 9, 2014

@pooya pooya closed this Apr 9, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment