Some bad sites gives in location header URL without proper encoding #189

Closed
wants to merge 1 commit into
from

Projects

None yet

2 participants

@marcocarvalho
require 'open-uri'

utl = "http://www.grupogonzaga.com.br/tabid/2076/ref/-Resid%C3%AAncia-Curitiba-Tatuquara-1-quarto/fic/875756.aspx" 
o = open(url)

#OpenURI::HTTPError: 301 Moved Permanently (Invalid Location URI)
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:342:in `rescue in open_http'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:339:in `open_http'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:201:in `catch'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:677:in `open'
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/open-uri.rb:33:in `open'
#   from (irb):68
#   from /usr/local/rvm/rubies/ruby-1.9.3-p194/bin/irb:16:in `<main>'

becouse this url returns a redirect with improper encoding:

resp = Net::HTTP.get_response(URI.encode(url))
resp.header['location']
# "http://www.grupogonzaga.com.br/Alugar/Ficha-do-im\xC3\xB3vel/fic/875756.aspx?ref=-Resid\xC3\xAAncia-Curitiba-Tatuquara-1-quarto"

Note the \xC3\xB3 bad encoded location return.

Added URI.encode to header['location'] on open-uri module.

Member
zzak commented Nov 19, 2012

Closing this, please refer to Feature #7396 in redmine

@zzak zzak closed this Nov 19, 2012
@tenderlove tenderlove added a commit that referenced this pull request Aug 29, 2014
@tenderlove tenderlove * ext/psych/lib/psych/scalar_scanner.rb: fix loading strings that
  look like integers but have a newline. Fixes GH #189
* test/psych/test_string.rb: test for fix

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47318 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
8662841
@mmasaki mmasaki pushed a commit to mmasaki/ruby that referenced this pull request Sep 8, 2014
@tenderlove tenderlove * ext/psych/lib/psych/scalar_scanner.rb: fix loading strings that
  look like integers but have a newline. Fixes GH #189
* test/psych/test_string.rb: test for fix

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47318 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
bdae441
@toddmazierski toddmazierski added a commit to lolibrarian/nypl-tweetwall that referenced this pull request Jul 25, 2016
@toddmazierski toddmazierski Make URL redirect following more reliable
Use Curb to work around ruby/ruby#189
5bbe5c0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment