Skip to content

Loading…

Raise if read of object bytes returns 0 bytes. #70

Merged
merged 1 commit into from

4 participants

@spikegronim

Hello s3cmd maintainers! At wavii we have been seeing s3cmd hang while reading large numbers of medium sized (~4MB) files from S3 within EC2. The symptom is that strace reports an infinite loop of poll, recvfrom. Poll indicates that the socket is closed (POLLHUP) and recvfrom returns 0 each time. The output of netstat indicates that the socket is gone. The S3 class was not respecting a zero byte return from the httplib connection. Without this fix s3cmd hangs once every 2-3 hours under our workload. With the fix it hasn't hanged in over 12 hours.

  • indication of EOF on the socket

PS Thanks for the great tool!

Spike Gronim Raise if read of object bytes returns 0 bytes.
- indication of EOF on the socket
c2c53af
@dustinboswell

I am having the same trouble. s3 is busy-looping on a broken socket, and doing so for hours. I straced the process and got an infinite stream of:

poll([{fd=4, events=POLLIN}], 1, 300000) = 1 ([{fd=4, revents=POLLIN}])
poll([{fd=4, events=POLLIN}], 1, 300000) = 1 ([{fd=4, revents=POLLIN}])
poll([{fd=4, events=POLLIN}], 1, 300000) = 1 ([{fd=4, revents=POLLIN}])
...

Was your change incorporated?

@spikegronim
@mdomsch
s3tools member

Pulled into my merge tree, thanks!

@mludvig mludvig merged commit c2c53af into s3tools:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 13, 2012
  1. Raise if read of object bytes returns 0 bytes.

    Spike Gronim committed
    - indication of EOF on the socket
Showing with 3 additions and 0 deletions.
  1. +3 −0 S3/S3.py
View
3 S3/S3.py
@@ -826,6 +826,9 @@ def recv_file(self, request, stream, labels, start_position = 0, retries = _max_
while (current_position < size_total):
this_chunk = size_left > self.config.recv_chunk and self.config.recv_chunk or size_left
data = http_response.read(this_chunk)
+ if len(data) == 0:
+ raise S3Error("EOF from S3!")
+
stream.write(data)
if start_position == 0:
md5_hash.update(data)
Something went wrong with that request. Please try again.