SteamDB.py no longer works due to CloudFlare protection

[MikeRixWolfe]

Could not get user info: HTTP Error 503: Service Temporarily Unavailable

CloudFlare protection is preventing steamdb.py from retrieving information with requests.

Suggestion:

• Implement https://github.com/Anorov/cloudflare-scrape

import cfscrape

scraper = cfscrape.create_scraper()
request = scraper.get(CALC_URL, params=params)
soup = bs4.BeautifulSoup(request.content)

Advantages:

• Easy
• Confirmed to fix the issue (by myself)

Disadvantages:

• Large dependencies (PyExecJS and Node.js)
• May not work forever (see author's notes on repo)

[dmptrluke]

Ah shit.

[foxlet]

Alternatively, adding a browser user-agent header to the requests operation could reduce the chance of CloudFlare pushing a JS challenge to Cloudbot.

[dmptrluke]

Yeah, I'll add that for now and see how it goes

On Wed, Feb 18, 2015 at 12:32 PM, Foxlet notifications@github.com wrote:

Alternatively, adding a browser user-agent header to the requests
operation could reduce the chance of CloudFlare pushing a JS challenge to
Cloudbot.

—
Reply to this email directly or view it on GitHub
#95 (comment).

[dmptrluke]

I've done what I can for now. Closing issue!