This repository has scripts for obtaining and processing some data from
The main script is
proc.py, which does tags fetching, series ID fetching, and
the actual data gathering.
modify_sql.py modifies the output of
proc.py by parsing each line and
transforming it. This allows us to download the data from the API once, then
transform it later. (It might have been better to download the data in a more
"neutral" format like a TSV so that the parsing would have been simpler.)
To generate the SQL files, the scripts in this repository require the
Python package. To install, run:
git clone https://github.com/riceissa/devec_sql_common cd devec_sql_common pip3 install -e .