Cannot rescue MultiJson::DecodeError raised by HTTParty #170

Closed
pstare opened this Issue Oct 25, 2012 · 7 comments

Projects

None yet

5 participants

@pstare
pstare commented Oct 25, 2012

Under some circumstances HTTParty seems to raise a MultiJson::DecodeError which is not rescue-able. The stack trace shows the exception is initially raised by MultiJson and it definitely bubbles up to the rails console, so I have no idea why I can't wrap the HTTParty.get call in a begin/rescue/end block to rescue this exception and take appropriate measures to handle it.

The issue is that the URL occasionally responds with a 500 Internal Server error, which is not properly formatted as JSON. Since HTTParty uses MultiJson to parse the response, a MultiJson::DecodeError is raised.

We have no problem with the exception being raised, as we anticipate that happening from time to time. We want to rescue it, but this doesn't seem possible, even though MultiJson::DecodeError is a subclass of StandardError.

There's mention of this issue at #101, but that issue thread seems mainly focused on what MultiJson should and shouldn't do when encountering things that aren't JSON. We don't care about that, we simply want to be able to rescue this exception, but it doesn't seem possible.

We are using httparty (0.8.3) and multi_json (1.3.6).

Here are two relevant examples with completely different results:

# This exception is rescued.
begin
  MultiJson.decode Net::HTTP.get('rubygems.org', '/api/v1/versions/doesnotexist.json')
rescue StandardError => e
  puts "Rescued!: #{e.inspect}"
end

Rescued!: #<MultiJson::DecodeError: 757: unexpected token at 'This rubygem could not be found.'>

# This exception is NOT rescued.
begin
  HTTParty.get "https://rubygems.org/api/v1/versions/doesnotexist.json"
rescue StandardError => e
  puts "Rescued!: #{e.inspect}"
end

MultiJson::DecodeError: 757: unexpected token at 'This rubygem could not be found.'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/json-1.7.3/lib/json/common.rb:155:in `parse'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/json-1.7.3/lib/json/common.rb:155:in `parse'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/multi_json-1.3.6/lib/multi_json/adapters/json_common.rb:7:in `load'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/multi_json-1.3.6/lib/multi_json.rb:93:in `load'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/httparty-0.8.3/lib/httparty/parser.rb:117:in `json'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/httparty-0.8.3/lib/httparty/parser.rb:140:in `parse_supported_format'
        from /Users/kiyan/.rvm/gems/ruby-1.9.2-p290/gems/httparty-0.8.3/lib/httparty/parser.rb:102:in `parse'
@nickhoffman

Hey guys. Any word on how to fix this?

@kenmazaika
Contributor

Weird. It looks like the HTTParty::Response#to_s is coming up as what an exception would look like, however no exception is being raised.

1.9.3p0 :018 > a = HTTParty.get("https://rubygems.org/api/v1/versions/doesnotexist.json")
MultiJson::DecodeError: no object read at line 1, column 1 [load.c:1042]
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/multi_json-1.5.0/lib/multi_json/adapters/oj.rb:17:in `load'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/multi_json-1.5.0/lib/multi_json/adapters/oj.rb:17:in `load'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/multi_json-1.5.0/lib/multi_json.rb:96:in `load'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/parser.rb:117:in `json'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/parser.rb:140:in `parse_supported_format'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/parser.rb:102:in `parse'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/parser.rb:66:in `call'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/request.rb:206:in `parse_response'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/request.rb:176:in `block in handle_response'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/response.rb:18:in `call'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/response.rb:18:in `parsed_response'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/response.rb:51:in `method_missing'
    from /Users/kmazaika/.rvm/gems/ruby-1.9.3-p0@campaign_manager/gems/httparty-0.9.0.where/lib/httparty/response.rb:30:in `inspect'
    from /Users/kmazaika/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
1.9.3p0 :019 > a.class
 => HTTParty::Response 
1.9.3p0 :020 > a.body
 => "This rubygem could not be found." 
@kenmazaika
Contributor

Ahh, calling #to_s is raising, which is implicitly being done in irb.

1.9.3p0 :025 > a = HTTParty.get("https://rubygems.org/api/v1/versions/doesnotexist.json").to_s rescue nil
 => nil 

Which is failing because inspect tries to use the parsed response: https://github.com/jnunemaker/httparty/blob/master/lib/httparty/response.rb#L31

presumably .inspect should gracefully deal with this case.

@jnunemaker
Owner

Ah, interesting. This is because we do lazy parsing of responses. Responses are only parsed when you actually use them. Hmm... Seems like we could rescue the exception then and wrap it with a new exception (HTTParty::ParseError) or something. Thoughts?

@nickhoffman

@jnunemaker That would make it easy to rescue a single type of exception. However, it doesn't really address the problem, because a rescue will still be needed every time the parsed response is used.

Since the response is evaluated lazily, and if that should continue to happen, the only solution that I can think of is to add documentation that explains what's going on. E.g.


Do not expect a problem with parsing the response to raise an exception when the HTTP request is made. This is due to HTTParty evaluating the response lazily. Thus, to check if an exception will be raised due to parsing the response, you must use the response, or force it to be parsed. E.g.

# Take note of the "; 1" at the end of the following line. It's required only if
# running this in IRB, because IRB will try to inspect the variable named
# "request", triggering the exception.
request = HTTParty.get 'https://rubygems.org/api/v1/versions/doesnotexist.json' ; 1

begin
  request.inspect
  # This would also suffice by forcing the request to be parsed:
  # request.parsed_response
rescue StandardError => e
  puts "Rescued #{e.inspect}"
end
@jnunemaker
Owner

ten four.

@rusikf
Contributor
rusikf commented Dec 12, 2014

@jnunemaker , I think this issue must be closed, no?

@jnunemaker jnunemaker closed this Dec 12, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment