Skip to content

Commit

Permalink
Added SQLite support contributed from Justin Jones.
Browse files Browse the repository at this point in the history
Included extra columns in gtf_stops.
  • Loading branch information
Colin Bick committed Feb 9, 2011
1 parent 764a7f6 commit fd9373e
Show file tree
Hide file tree
Showing 7 changed files with 425 additions and 22 deletions.
40 changes: 38 additions & 2 deletions README
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Released under the MIT (X11) license. See LICENSE in this directory.

== How To ==
This is how to import GTFS data into SQL:
( see below for SQLite)

1. Initial import

Expand Down Expand Up @@ -47,10 +48,9 @@ psql -f gtfs_tables_dropindexes.sql
psql -f gtfs_tables_makeindexes.sql



== Test/Demonstration ==

The (CORRECTED -- yes even google's example data has errors) demo feed from the
The corrected (even google's example data has errors) demo feed from the
GTFS website is included in this distribution. You should play around with that
first to get everything to work and to see how the data gets put into tables.

Expand All @@ -63,3 +63,39 @@ cat gtfs_tables.sql \
vacuumer.sql \
| psql testgtfs
psql testgtfs -c "\dt"



== Importing into an SQLite Database ==
Contributed by Justin Jones, Feb 07, 2011 bjustjones@netscape.net

1. Initial import

Note:
From http://www.sqlite.org/omitted.html as of Feb, 2011:
"Only the RENAME TABLE and ADD COLUMN variants of the ALTER TABLE command
are supported. Other kinds of ALTER TABLE operations such as DROP COLUMN,
ALTER COLUMN, ADD CONSTRAINT, and so forth are omitted."

gtfs_tables.sqlite includes the constraints on creation.


Use: (note use of NOCOPY option)

cat gtfs_tables.sqlite \
<(python import_gtfs_to_sql.py sample_feed nocopy) \
| sqlite3 ANewDatabase.db


SQLite doesn't enforce constraints by default. See first section of
gtfs_tables.sqlite for line to change. Move it to the end?
If you need to makeindices or dropindices as above you'll have to experiment
with doing it yourself.


2. Modification within SQL db
- included for completeness. Tested.

sqlite3 -init gtfs_tables_dropindexes.sqlite myDatabase.db
# do your stuff
sqlite3 -init gtfs_tables_makeindexes.sqlite myDatabase.db
63 changes: 45 additions & 18 deletions html/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -25,45 +25,72 @@ <h3>A Small Python Tool for Importing GTFS Data into SQL</h3>


<h3>Using GTFS SQL Importer</h3>
The code below shows how to import some data from a bash prompt.
<p>The code below shows how to import some data from a bash prompt.</p>

<hr />

<h4>PostgreSQL</h4>

<code><pre>
cat gtfs_tables.sql \
&lt;(python import_gtfs_to_sql.py path/to/gtfs/data/directory) \
gtfs_tables_makeindexes.sql \
vacuumer.sql \
| psql mydbname myusername
</pre></code>

cat gtfs_tables.sql \
&lt;(python import_gtfs_to_sql.py path/to/gtfs/data/directory) \
gtfs_tables_makeindexes.sql \
vacuumer.sql \
| psql mydbname myusername
<hr />

<h4>SQLite</h4>

<code><pre>
cat gtfs_tables.sqlite \
&lt;(python import_gtfs_to_sql.py path/to/gtfs/data/directory nocopy) \
| sqlite3 ANewDatabase.db
</pre></code>

SQLite compatibility contributed by Justin Jones. See README for details.
<hr />

<h4>Other Non-PostgreSQL</h4>
<p>
Most GTFS data has errors in it, so you will likely encounter an error when
running the step above. After fixing the error by manually correcting the GTFS
files, you can simply repeat the command (which will likely break again, and
so on).
If you don't have PostgreSQL, then the python line should be as follows:<br>
<code><pre>
&lt;(python import_gtfs_to_sql.py path/to/gtfs/data/directory nocopy)
</pre></code>
This will use "INSERT" statements instead of "COPY" statements.
The vacuumer.sql file is also postgres specific and should be omitted.
</p>

<hr />

<p>
If you don't have PostgreSQL, then the python line should be as follows:<br>
<code><pre>
&lt;(python import_gtfs_to_sql.py path/to/gtfs/data/directory nocopy)
</pre></code>
This will use "INSERT" statements instead of "COPY" statements.
Also, I believe the vacuumer.sql file is also postgres specific, so omit it if
it gives errors.
Most GTFS data has errors in it, so you will likely encounter an error when
running the above steps. After fixing the error by manually correcting the GTFS
files, you can simply repeat the command (which will likely break again, and
so on).
</p>


<p>
The source download includes the (CORRECTED -- yes even google's example data has errors) demo feed from the GTFS website, in the src/sample_feed directory. You should play around with that first to get everything to work and to see how the data gets put into tables.
The source download includes a corrected version of the demo feed from the
GTFS website, in the src/sample_feed directory. You should play around with
that first to get everything to work and to see how the data gets put into
tables.
</p>




<h3>Links</h3>

<a href="http://code.google.com/transit/spec/transit_feed_specification.html">GTFS (General Transit Feed Specification)</a><br>

<a href="http://www.gtfs-data-exchange.com/">GTFS Data Exchange</a><br>

<a href="http://code.google.com/p/googletransitdatafeed/wiki/PublicFeeds">List of Public GTFS feeds</a>


</tr></td></table>
</body>
</html>
12 changes: 10 additions & 2 deletions src/gtfs_tables.sql
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,17 @@ create table gtf_stops (
stop_lon double precision,
zone_id int,
stop_url text,
stop_code text
stop_code text,

-- new
stop_street text,
stop_city text,
stop_region text,
stop_postcode text,
stop_country text,

-- unofficial features
,

location_type int, --FOREIGN KEY REFERENCES gtfs_location_types(location_type)
parent_station text, --FOREIGN KEY REFERENCES gtf_stops(stop_id)
stop_timezone text
Expand Down
Loading

0 comments on commit fd9373e

Please sign in to comment.