This is a simple Python3 script to scrape Fandom site and get much information on various items available in the game.
List of defined numeric ID
s is from here
with minor editing and corrections (BEWARE: I might have goofed badly!).
Installation under Linux is straightforward (I don't use Windows for programming):
# clone reository
git clone <repository>
#enter project root directory
cd ScrapeFandom/
# create basic Vitual ENVironment
python3 -m venv venv
# install required non-standard-lib packages
# i.e.: lxml and requests
venv/bin/pip install -r requirements.txt
# run the script
venv/bin/python ParseFandom.py
After execution (it takes a bit of time) you should have several new files: the most important are:
-
Fandom.py
: the resulting script containing the whole scraped information inlist
format and a couple of helperdict
s:all_by_id[int]
: access info byID
valueall_by_name[int]
: access info by in-game name
-
resources/images/*
: collection of images pertaining to objects.Names are computed from in-game item name.
Note: These are images and not the icons used in-game as I didn't find a reliable way to getthem fromFandom. I am thinking about scraping them from Fextralife but that's for another day.
In the code there are several workarounds needed to account for differences in Wiki pages edited over a quite sizable time span. I wouldn't be surprised if "further enhancements" will break the scraping code; please notify me if you find inconsistencies.
Suggestions and comments (and patches) are always welcome.
Enjoy!