Getting Data

Andrew Ekstedt edited this page Nov 30, 2017 · 18 revisions

If you're like most people, you just want to get a hold of the data for your own nefarious purposes. Lucky for you, I have made this as simple as I can imagine. As long as you know your way around a terminal, anyway. You do, right?

For more exhaustive documentation on using the pokedex program, see CLI.

0. Install the prerequisites

Install Git, Python, and the Distribute and Virtualenv libraries for Python. MAKE SURE YOU USE PYTHON 2, NOT 3. In some distributions you may find that Distribute comes under the name of python-Distutils2.

On Debian or Ubuntu, that would be sudo apt-get install git python python-distribute python-virtualenv.

On Windows, get Git and Python.

If you're using Python older than 2.7.9 on Windows, then get the get-pip.py and run it using Python, by typing C:\python27\python C:\wherever\get-pip.py in the command line (substitute whatever paths you have). You can then use pip in the instructions below.

1. Get the code.

You have two options here.

You can use Git, like I do, if you're familiar with source control:

git clone git://github.com/veekun/pokedex.git

Or you can just go to Releases on Github and download an archive, or download the latest development version.

2. "Install" the code.

You have to do this for two reasons: (a) to make sure you have all the right libraries installed, and (b) to create the pokedex program, which actually loads the data into a database for you.

You can do this...

2A. The quick way

 python setup.py develop

(Windows users will want to use the full path, e.g. C:\python27\python instead of just python.)

You might have to invoke sudo if you're on a good platform, because this will install necessary libraries to your system Python directory.

If you'd rather not the quick way (good for you!), you should instead use:

2B. The good way

  1. Get virtualenv. It's in most repositories as python-virtualenv, or you can use pip install virtualenv, or get it manually if you must.
  2. Create a virtualenv for the pokedex project by running virtualenv ./ inside the topmost pokedex directory. This will set up a little Python environment inside it. If your system defaults to Python 3 or you run into other version-related problems, make sure you use the --python=python2 argument (or similar).
  3. Run bin/python setup.py develop. (Or, on Windows, use Scripts/python setup.py develop.) Make sure you run the bin/python inside your virtualenv. This will install libraries and the pokedex program only inside the virtualenv and will not touch your system Python. Note that the pokedex program will then end up inside that same bin/ (or Scripts/) directory, so you'll have to use bin/pokedex (or Scripts/pokedex) in step 4.

3. Prep your database.

A lot of people come ask for help, only to realize they sort of forgot this bit. Here's a quick checklist:

  • Be sure you know which database engine you're using!
  • Be sure you've actually created the database!
  • Be sure you have Python support for it installed! Out of the box, Python ONLY supports SQLite! For PostgreSQL, you'll need psycopg2; for MySQL, you'll need mysqldb.

If you don't care, you can use the default SQLite, but this won't work if you used sudo before.

4. Load the data.

For the default SQLite, just do:

pokedex load

(Windows users will use C:\Python27\Scripts\pokedex.exe or similar instead of python.)

For different databases, do this:

pokedex load -e postgresql://username:password@localhost/database
pokedex load -e mssql+pyodbc://.\SQLEXPRESS/database?driver=SQL+Server+Native+Client+11.0
pokedex load -e mysql+pymysql://username:password@localhost/database

Or, you know, something like that. This will churn away for a while (a pretty long while, actually). Then, do:

pokedex reindex

When it's done, you should have a database full of data. Congrats!

For more stuff you can do with the pokedex program, and better instructions on how to use load, see the pokedex documentation (pokedex -h).

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.