Skip to content
This repository has been archived by the owner on Mar 4, 2022. It is now read-only.

Commit

Permalink
Try curl fallback.
Browse files Browse the repository at this point in the history
  • Loading branch information
reitermarkus committed Feb 19, 2019
1 parent 23ee787 commit dbd181c
Showing 1 changed file with 15 additions and 5 deletions.
20 changes: 15 additions & 5 deletions scraper.rb
Original file line number Diff line number Diff line change
@@ -1,9 +1,19 @@
require 'scraperwiki'
require 'mechanize'

agent = Mechanize.new

page = agent.get('https://www.fastenergy.at/heizoelpreis-tendenz.htm')
require 'scraperwiki'
require 'open3'

url = 'https://www.fastenergy.at/heizoelpreis-tendenz.htm'

page = begin
agent = Mechanize.new
agent.get(url)
rescue
out, err, status = Open3.capture3('curl', '-kL', url)
puts out
puts
raise err unless status.success?
Mechanize::Page::new(URI(url), nil, out, nil, agent)
end

rows = page.search('#tabelle tbody tr')

Expand Down

0 comments on commit dbd181c

Please sign in to comment.