Loa is a program that parses the information from http://gatherer.wizards.com and produces XML formatted data.
- Get leiningen
- Setup dependencies (see below)
lein run --full
The output can be found in the data/zip directory.
- Get ants
- lein install
lein run --full lein run --meta --language --package
--meta --language --package are equivalent. Produces
zip-file with current date.
lein run --full --threads 4
The number of threads default to the number of cores available on your
machine but can be overridden with
lein run --debug --set mirrodin alara --card forest demon
--debug prints the internal data to the console.
--card narrows the set of data to sets and cards whose names contain
any of the given strings.
src/ '-- loa |-- cleanup |-- format |-- gatherer |-- indata |-- program |-- util '-- validation
cleanup has functions and data for processing "raw" cards (from gatherer). Also contains information about cards not in gatherer.
format is for transforming the internal data represenation to xml.
gatherer functions for parsing gatherer html pages.
indata processing of data that is not fetched from gatherer.
program functions that combine the other parts into a proper application.
util non-application specific functionality.
validation functions for data validation.
data/ |-- indata/ |-- tmp-download/ |-- xml/ '-- zip/
indata contains data that is not currently available in other form.
tmp-download is where downloaded data is cached. This directory needs to be cleared if new data should be fetched from gatherer.
xml is where the canonical data is stored.
zip contains zip-files containing data from indata, text and xml directories. This is the deliverable.