Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
tree: f1c96f12ae
Fetching contributors…

Cannot retrieve contributors at this time

160 lines (130 sloc) 6.513 kB

BnetScraper Build Status

BnetScraper is a Nokogiri-based scraper of Battle.net profile information. Currently this only includes Starcraft2.

Installation

Run gem install bnet_scraper or add gem 'bnet_scraper' to your Gemfile.

Usage

All of the scrapers take an options hash, and can be created by either passing a URL string for the profile URL or passing the account information in the options hash. Thus, either of these two approaches work:

BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/12345/1/TestAccount/')
BnetScraper::Starcraft2::ProfileScraper.new(bnet_id: '12345', account: 'TestAccount', region: 'na')

There are several scrapers that pull various information. They are:

  • BnetScraper::Starcraft2::ProfileScraper - collects basic profile information and an array of league URLs
  • BnetScraper::Starcraft2::LeagueScraper - collects data on a particular league for a particular Battle.net account
  • BnetScraper::Starcraft2::AchievementScraper - collects achievement data for the account.
  • BnetScraper::Starcraft2::MatchHistoryScraper - collects the 25 most recent matches played on the account

All scrapers have a #scrape method that triggers the scraping and storage. By default they will return the result, but an additional #output method exists to retrieve the results subsequent times without re-scraping.

BnetScraper::Starcraft2::ProfileScraper

This pulls basic profile information for an account, as well as an array of league URLs. This is a good starting point for league scraping as it provides the league URLs necessary to do supplemental scraping.

scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
scraper.scrape
# => {
  bnet_id: '2377239',
  account: 'Demon',
  bnet_index: 1,
  race: 'Protoss',
  wins: '684',
  achievement_points: '3630',
  current_solo_league: 'Not Yet Ranked',
  highest_solo_league: 'Platinum',
  current_team_league: 'Not Yet Ranked',
  highest_team_league: 'Diamond',
  career_games: '1568',
  games_this_season: '0',
  most_played: '4v4',
  leagues: [
    {
      name: "1v1 Platinum Rank 95",
      id:   "96905",
      href: "http://us.battle.net/sc2/en/profile/2377239/1/Demon/ladder/96905#current-rank"
    }
  ]
}

BnetScraper::Starcraft2::LeagueScraper

This pulls information on a specific league for a specific account. It is best used either in conjunction with a profile scrape that profiles a URL, or if you happen to know the specific league_id and can pass it as an option.

scraper = BnetScraper::Starcraft2::LeagueScraper.new(league_id: '12345', account: 'Demon', bnet_id: '2377239')
scraper.scrape
# => {
  season: '6',
  name: 'Aleksander Pepper',
  division: 'Diamond',
  size: '4v4',
  random: false,
  bnet_id: '2377239',
  account: 'Demon'
}

BnetScraper::Starcraft2::AchievementScraper

This pulls achievement information for an account. Note that currently only returns the overall achievements, not the in-depth, by-category achievement information.

scraper = BnetScraper::Starcraft2::AchievementScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
scraper.scrape
# => {
  recent: [
    { title: 'Blink of an Eye', description: 'Complete round 24 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
    { title: 'Whack-a-Roach', description: 'Complete round 9 in "Starcraft Master" in under 45 seconds', earned: '3/5/2012' },
    { title: 'Safe Zone', description: 'Complete round 8 in "Starcraft Master" without losing any stalkers', earned: '3/5/2012' },
    { title: 'Starcraft Master', description: 'Complete all 30 rounds in "Starcraft Master"', earned: '3/5/2012' },
    { title: 'Starcraft Expert', description: 'Complete any 25 rounds in "Starcraft Master"', earned: '3/5/2012' },
    { title: 'Starcraft Apprentice', description: 'Complete any 20 rounds in "Starcraft Master"', earned: '3/5/2012' }
  ],
  showcase: [
    { title: 'Hot Shot', description: 'Finish a Qualification Round with an undefeated record.' },
    { title: 'Starcraft Master', description: 'Complete all rounds in "Starcraft Master"' },
    { title: 'Team Protoss 500', description: 'Win 500 team league matches as Protoss' },
    { title: 'Night of the Living III', description: 'Survive 15 Infested Horde Attacks in the "Night 2 Die" mode of the "Left 2 Die" scenario.' },
    { title: 'Team Top 100 Diamond', description: 'Finish a Season in Team Diamond Division' }

  ],
  progress: {
    liberty_campaign: '1580',
    exploration: '480',
    custom_game: '330',
    cooperative: '660',
    quick_match: '170'
  }
}

BnetScraper::Starcraft2::MatchHistoryScraper

This pulls the 25 most recent matches played for an account. Note that this is only as up-to-date as battle.net is, and will likely not be as fast as in-game.

scraper = BnetScraper::Starcraft2::MatchHistoryScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
scraper.scrape
# => {
  wins: '15',
  losses: '10',
  matches: [
    { map_name: 'Bx Monobattle - Sand Canyon (Fix)', outcome: :win, type: 'Custom', date: '3/12/2012' },
    { map_name: 'Deadlock Ridge', outcome: :loss, type: '4v4', date: '3/12/2012' },
    { map_name: 'District 10', outcome: :win, type: '4v4', date: '3/12/2012' },
    # ...
  ]
}

BnetScraper::Starcraft2::Status

Scraping is only possible if the site is up. Use this if you want to verify the failed scrape is because the site is down:

BnetScraper::Starcraft2::Status.na # => 'Online'
BnetScraper::Starcraft2::Status.fea # => 'Offline'
BnetScraper::Starcraft2::Status.cn #  => nil (China is unsupported)
BnetScraper::Starcraft2::Status.fetch # => [
  {:region=>"North America", :status=>"Online"},
  {:region=>"Europe", :status=>"Online"},
  {:region=>"Korea", :status=>"Online"},
  {:region=>"South-East Asia", :status=>"Online"}
]

Contribute!

I would love to see contributions! Please send a pull request with a feature branch containing specs (Chances are excellent I will break it if you do not) and I will take a look. Please do not change the version as I tend to bundle multiple fixes together before releasing a new version anyway.

Author

Written by Andrew Nordman, see LICENSE for details.

Jump to Line
Something went wrong with that request. Please try again.