Two scripts for gathering betting data from site: CybBet.com. First script - 'cybbet.py' - using scrapy to acquire data, when second - 'cybbet_scraper.py' - using combination of BeautifulSoup + requests.
Spider for scraping esport data from betting site cybbet.com using scrapy with css selectors(for educational purposes).
To run this script, you'll need to install scrapy packet first. On a Windows machine, you'll also need to install appropriate C++ SDK to run 'Twisted' (check a documentation).
Create scrapy project and put this script into 'spiders' folder.
Use command 'scrapy crawl cybbet' from the command line to start the crawler(you need to be in the scrapy project directory).
To save scraped data into .csv file - uncomment section of code with 'custom_settings' definition.
This script will scrape game's data from betting site: cybbet.com/
Mind, that scraper was built for educational purposes.
'cybbet_scraper' script using combination of 'BeautifulSoup' + 'requests' for scraping data. So, before running it - make sure that you have 'bs4' and 'requests' packets installed.
You can run this script from command line with parameter 'show' to list scraped data directly into terminal window (also, command 'help' to check all available options).
After importing 'cybbet_scraper' into your own script - run 'cybbet_spider()' function to collect data from the site. Returning data type - dictionary of lists of dictionaries:
{
'name of the 1st esport': [{'time': event time, 'player1': name of the first player,
'player2': name of the second player, 'odds1': bet odds on the first player,
'odds2': bet odds on the second player},
{another game information},],
'name of the 2nd esport': [{}, {}, ],
..............................................
}
Script scrapes only event's data on which we can still bet on(but you can comment/uncomment necessary sections to scrape all available data).