Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

shake the API a bit, now Curly can be used more like an agent

  • Loading branch information...
commit 4cbf83783a679c8dc807fbc534b44ae1908e239f 1 parent 899bf37
Mislav Marohnić authored

Showing 1 changed file with 51 additions and 4 deletions. Show diff stats Hide diff stats

  1. +51 4 curly.rb
55 curly.rb
... ... @@ -1,16 +1,40 @@
1 1 require 'rubygems'
  2 +require 'uri'
2 3 require 'curb'
3 4 gem 'activesupport', '>= 2.1'
4 5 require 'active_support/basic_object'
5 6 require 'hpricot'
  7 +require 'iconv'
6 8
7 9 class Curly < ActiveSupport::BasicObject
8   - def initialize(url = nil)
9   - @curl = Curl::Easy.new(url)
  10 + attr_reader :uri
  11 +
  12 + def initialize(uri = nil)
  13 + @curl = Curl::Easy.new
  14 + self.uri = uri
10 15 self.follow_location = true
11 16 yield self if block_given?
12 17 end
13 18
  19 + def uri=(obj)
  20 + case obj
  21 + when String
  22 + unless @uri
  23 + @uri = URI.parse(obj)
  24 + else
  25 + @uri += obj
  26 + end
  27 + when URI::HTTP
  28 + @uri = obj
  29 + when nil
  30 + return
  31 + else
  32 + raise "unsupported URI type (#{obj.class.name} given)"
  33 + end
  34 +
  35 + self.url = @uri.to_s
  36 + end
  37 +
14 38 def method_missing(method, *args, &block)
15 39 @curl.send(method, *args, &block)
16 40 end
@@ -20,8 +44,27 @@ def cookiejar=(filename)
20 44 @curl.cookiejar = filename
21 45 end
22 46
23   - def get
  47 + def get(uri = nil)
  48 + self.uri = uri
24 49 http_get
  50 + raise "expected 2xx, got #{response_code} (GET #{url})" unless success?
  51 + self
  52 + end
  53 +
  54 + def success?
  55 + response_code >= 200 and response_code < 300
  56 + end
  57 +
  58 + def doc
  59 + Hpricot body_unicode
  60 + end
  61 +
  62 + def body_unicode
  63 + body = body_str
  64 + if body =~ /;\s*charset=([\w-]+)\s*['"]/ and $1.downcase != 'utf-8'
  65 + body = Iconv.conv('UTF-8', $1, body)
  66 + end
  67 + body
25 68 end
26 69
27 70 def self.get_document(url)
@@ -31,7 +74,11 @@ def self.get_document(url)
31 74 end
32 75
33 76 def self.parse_curl(object)
34   - Hpricot(object.body_str)
  77 + body = object.body_str
  78 + if body =~ /;\s*charset=([\w-]+)\s*['"]/ and $1.downcase != 'utf-8'
  79 + body = Iconv.conv('UTF-8', $1, body)
  80 + end
  81 + Hpricot(body)
35 82 end
36 83
37 84 def post(params)

0 comments on commit 4cbf837

Please sign in to comment.
Something went wrong with that request. Please try again.