Skip to content

warning: Possibly lost track of Curl::Easy VALUE, it may not be reclaimed by GC #63

Open
ghazel opened this Issue Dec 15, 2010 · 3 comments

2 participants

@ghazel
ghazel commented Dec 15, 2010

Seen again in the log with 0.7.8.
Here's some code I use to call multi.perform (the retrying is a little aggressive, that has to do with the particular service I am communicating with):

class Curl::Easy
  attr_accessor :retry_timeout
  attr_accessor :hard_error_retry
  attr_accessor :retries

  attr_accessor :multi

  def retry
    self.retries += 1
    if not @multi.nil?
      @multi.add(self)
    else
      perform()
    end
  end

end


def make_request(url)
  ncurl = Curl::Easy.new(url) do |curl|
    curl.useragent = request.user_agent
    # If a zero-length string is set, then an Accept-Encoding header
    # containing all supported encodings is sent.
    curl.encoding = ""
    #curl.verbose = true
    curl.max_redirects = MAX_REDIRECTS
    curl.connect_timeout = 10
    curl.timeout = 300
    # our options
    curl.retry_timeout = 100.0
    curl.retries = 0
    curl.hard_error_retry = 0

    curl.on_failure do |c,e|
      t = Time.now.to_f.to_s.ljust(14)
      if [404, 400, 500].include?(c.response_code)
        puts "#{t} #{c.object_id}: HTTP response code #{c.response_code}. Retrying in #{curl.retry_timeout}ms (#{curl.retries} retries)..."
        if c.response_code != 503
          if curl.hard_error_retry >= 100
            raise HTTPError, "HTTP response code #{c.response_code} hard error retry: #{curl.hard_error_retry}"
          end
          curl.hard_error_retry += 1
        end
        # sleep a little..
        sleep curl.retry_timeout/1000.0
        curl.retry_timeout += 10
        c.retry
      elsif [Curl::Err::TimeoutError, Curl::Err::GotNothingError].include?(e[0])
        puts "#{t} #{c.object_id}: #{e[1]} (#{e[0]}). Retrying (#{curl.retries} retries)..."
        c.retry
      else
        if not [0, 200].include?(c.response_code)
          if [410, 503].include?(c.response_code)
            success_func.call(c.body_str)
          else
            raise HTTPError, "HTTP response code #{c.response_code}"
          end
        else
          raise *e
        end
      end
    end

    curl.on_success do |c|
      success_func.call(c.body_str)
    end

  end
end

@@curl_multi = Curl::Multi.new
urls.each do |url|
  curl = make_request(url)
  curl.multi = @@curl_multi
  @@curl_multi.add(curl)
end

@@curl_multi.perform    
@taf2
Owner
taf2 commented Dec 22, 2010

I modified you script slightly to get it to run https://gist.github.com/751155 i'm not seeing the warning you were getting... it is a fairly harmless warning - it just means when we get to the point of GC, the internal Hash that manages the easy structures was unable to locate the easy handle - this is probably because it was all ready released.

@ghazel
ghazel commented Dec 22, 2010

Yeah it doesn't happen every time or anything. In fact it happens very very rarely, and I'm not sure which codepath triggers it. It is fairly harmless, I just thought I'd report it.

@taf2
Owner
taf2 commented Dec 22, 2010

yeah, it's definitely valid - i've also spotted it a few times... i added that report because I figured it could be possible... so i 'll keep this ticket open...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.