Skip to content

RyuaNerin/lodestone-recipe-db-scraper

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

56 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lodestone Recipe DB Scraper

This project provides a tool which scrapes recipe information from the Lodestone website's Eorzea Database. It extracts recipes for all classes, levels 1-60, including the English, German, French and Japanese names. Each class' recipes are written out to a separate JSON file in the order defined by the Lodestone's recipe listings.

Pre-requisites

This project uses Python 3.4 (or newer) and describes its dependencies in a pip requirements.txt file. You can install all the packages required by running:

pip install -r requirements.txt

If you are using Windows, you can first download the lxml extension (e.g. lxml‑3.6.4‑cp35‑cp35m‑win_amd64.whl for CPython 3.5 64-bit) from here and manually install using:

pip install C:\path\to\downloaded\package.whl

And then install the remainder of the dependencies using the first pip install command.

If you would rather have pip compile the package, you will need to ensure that you have the correct version of Visual Studio installed (the Express edition will suffice), and run the first pip install command inside a "Visual Studio Command Prompt". Please see the lxml installation instructions for further information.

Another alternative for Windows 10 is to use the Windows Subsystem for Linux. Once installed, in a Bash window you can use:

sudo apt install python3 && sudo pip install -r requirements.txt

to install Python and all the dependencies, and then run Python as described below.

Usage

Run main.py with the following arguments:

python main.py -l cn=items_cn.json

It will output its progress to stdout and generate json files in the out subdirectory.

The -l option includes names in languages not supported by the Lodestone, e.g. Chinese. The option must be provided as LANG=FILE, where LANG is the two-letter code identifying the language and FILE is the path to the file containing the translations. The file must contain a JSON object with the English recipe names as the keys and the translated names as the values. You can specify the -l option multiple times to add multiple languages.

About

Scrapes recipe data from the FFXIV Lodestone website.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%