warning: Possibly lost track of Curl::Easy VALUE, it may not be reclaimed by GC #63

Open
ghazel opened this Issue Dec 15, 2010 · 3 comments

Comments

Projects
None yet
2 participants
@ghazel
Contributor

ghazel commented Dec 15, 2010

Seen again in the log with 0.7.8.
Here's some code I use to call multi.perform (the retrying is a little aggressive, that has to do with the particular service I am communicating with):

class Curl::Easy
  attr_accessor :retry_timeout
  attr_accessor :hard_error_retry
  attr_accessor :retries

  attr_accessor :multi

  def retry
    self.retries += 1
    if not @multi.nil?
      @multi.add(self)
    else
      perform()
    end
  end

end


def make_request(url)
  ncurl = Curl::Easy.new(url) do |curl|
    curl.useragent = request.user_agent
    # If a zero-length string is set, then an Accept-Encoding header
    # containing all supported encodings is sent.
    curl.encoding = ""
    #curl.verbose = true
    curl.max_redirects = MAX_REDIRECTS
    curl.connect_timeout = 10
    curl.timeout = 300
    # our options
    curl.retry_timeout = 100.0
    curl.retries = 0
    curl.hard_error_retry = 0

    curl.on_failure do |c,e|
      t = Time.now.to_f.to_s.ljust(14)
      if [404, 400, 500].include?(c.response_code)
        puts "#{t} #{c.object_id}: HTTP response code #{c.response_code}. Retrying in #{curl.retry_timeout}ms (#{curl.retries} retries)..."
        if c.response_code != 503
          if curl.hard_error_retry >= 100
            raise HTTPError, "HTTP response code #{c.response_code} hard error retry: #{curl.hard_error_retry}"
          end
          curl.hard_error_retry += 1
        end
        # sleep a little..
        sleep curl.retry_timeout/1000.0
        curl.retry_timeout += 10
        c.retry
      elsif [Curl::Err::TimeoutError, Curl::Err::GotNothingError].include?(e[0])
        puts "#{t} #{c.object_id}: #{e[1]} (#{e[0]}). Retrying (#{curl.retries} retries)..."
        c.retry
      else
        if not [0, 200].include?(c.response_code)
          if [410, 503].include?(c.response_code)
            success_func.call(c.body_str)
          else
            raise HTTPError, "HTTP response code #{c.response_code}"
          end
        else
          raise *e
        end
      end
    end

    curl.on_success do |c|
      success_func.call(c.body_str)
    end

  end
end

@@curl_multi = Curl::Multi.new
urls.each do |url|
  curl = make_request(url)
  curl.multi = @@curl_multi
  @@curl_multi.add(curl)
end

@@curl_multi.perform    
@taf2

This comment has been minimized.

Show comment
Hide comment
@taf2

taf2 Dec 22, 2010

Owner

I modified you script slightly to get it to run https://gist.github.com/751155 i'm not seeing the warning you were getting... it is a fairly harmless warning - it just means when we get to the point of GC, the internal Hash that manages the easy structures was unable to locate the easy handle - this is probably because it was all ready released.

Owner

taf2 commented Dec 22, 2010

I modified you script slightly to get it to run https://gist.github.com/751155 i'm not seeing the warning you were getting... it is a fairly harmless warning - it just means when we get to the point of GC, the internal Hash that manages the easy structures was unable to locate the easy handle - this is probably because it was all ready released.

@ghazel

This comment has been minimized.

Show comment
Hide comment
@ghazel

ghazel Dec 22, 2010

Contributor

Yeah it doesn't happen every time or anything. In fact it happens very very rarely, and I'm not sure which codepath triggers it. It is fairly harmless, I just thought I'd report it.

Contributor

ghazel commented Dec 22, 2010

Yeah it doesn't happen every time or anything. In fact it happens very very rarely, and I'm not sure which codepath triggers it. It is fairly harmless, I just thought I'd report it.

@taf2

This comment has been minimized.

Show comment
Hide comment
@taf2

taf2 Dec 22, 2010

Owner

yeah, it's definitely valid - i've also spotted it a few times... i added that report because I figured it could be possible... so i 'll keep this ticket open...

Owner

taf2 commented Dec 22, 2010

yeah, it's definitely valid - i've also spotted it a few times... i added that report because I figured it could be possible... so i 'll keep this ticket open...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment