Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
switching from a curl-shell-out to a streaming net-http for Action#do…
…wnload. Slower, but more portable.
  • Loading branch information
jashkenas committed Jun 22, 2010
1 parent 7c2b91f commit a7bd56d
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 14 deletions.
1 change: 1 addition & 0 deletions lib/cloud-crowd.rb
Expand Up @@ -26,6 +26,7 @@

# Common code which should really be required in every circumstance.
require 'socket'
require 'net/http'
require 'cloud_crowd/exceptions'

module CloudCrowd
Expand Down
25 changes: 12 additions & 13 deletions lib/cloud_crowd/action.rb
Expand Up @@ -42,19 +42,18 @@ def process

# Download a file to the specified path.
def download(url, path)
`curl -s "#{url}" > "#{path}"`
return path
# The previous implementation is below, and, although it would be
# wonderful not to shell out, RestClient wasn't handling URLs with encoded
# entities (%20, for example), and doesn't let you download to a given
# location. Getting a RestClient patch in would be ideal.
#
# if url.match(FILE_URL)
# FileUtils.cp(url.sub(FILE_URL, ''), path)
# else
# resp = RestClient::Request.execute(:url => url, :method => :get, :raw_response => true)
# FileUtils.mv resp.file.path, path
# end
if url.match(FILE_URL)
FileUtils.cp(url.sub(FILE_URL, ''), path)
else
File.open(path, 'w+') do |file|
Net::HTTP.get_response(URI(url)) do |response|
response.read_body do |chunk|
file.write chunk
end
end
end
end
path
end

# Takes a local filesystem path, saves the file to S3, and returns the
Expand Down
9 changes: 8 additions & 1 deletion test/unit/test_action.rb
Expand Up @@ -50,14 +50,21 @@ class ActionTest < Test::Unit::TestCase
end

should "be able to count the number of words in this file" do
assert @action.process == 247
assert @action.process == 274
end

should "raise an exception when backticks fail" do
def @action.process; `utter failure 2>&1`; end
assert_raise(CloudCrowd::Error::CommandFailed) { @action.process }
end

should "be able to download a remote file" do
path = "temp.txt"
@action.download('http://example.com', path)
assert File.read(path).match(/These domain names are reserved for use in documentation/)
FileUtils.rm path
end

end


Expand Down

0 comments on commit a7bd56d

Please sign in to comment.