New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amend importgtfs.py to ignore extra columns in feed files #10

Closed
araichev opened this Issue Sep 3, 2013 · 6 comments

Comments

Projects
None yet
4 participants
@araichev

araichev commented Sep 3, 2013

Some feeds e.g. Adelaide, Australia (http://www.gtfs-data-exchange.com/agency/adelaide-metro/) have extra columns in their feed files e.g. 'wheelchair_accessible' in trips.txt. Currently importgtfs.py fails on such feeds. It would be more useful, i reckon, for importgtfs.py to simply ignore these extra columns and import the rest of the columns as usual.

@jwhitlock

This comment has been minimized.

Show comment
Hide comment
@jwhitlock

jwhitlock Sep 6, 2013

Member

I've been following OpenTripPlanner development, and extra columns are a common mechanism for extending GTFS (I'd prefer different files, but no one asked me). So, I'd like multigtfs to be able to round trip these extra columns as well. I'm thinking we could store extra data in a JSON-encoded text column, or in an extra key-value table.

But, I agree, a quick first step would be to ignore them.

Member

jwhitlock commented Sep 6, 2013

I've been following OpenTripPlanner development, and extra columns are a common mechanism for extending GTFS (I'd prefer different files, but no one asked me). So, I'd like multigtfs to be able to round trip these extra columns as well. I'm thinking we could store extra data in a JSON-encoded text column, or in an extra key-value table.

But, I agree, a quick first step would be to ignore them.

@slai

This comment has been minimized.

Show comment
Hide comment

slai commented Mar 23, 2014

For what it's worth, 'wheelchair_accessible' was added to trips.txt in the GTFS spec (https://developers.google.com/transit/gtfs/reference#trips_fields) in 2012 (https://groups.google.com/forum/#!msg/gtfs-changes/xyPh5stQ8o4/ATa1nQZLcb8J).

@jwhitlock

This comment has been minimized.

Show comment
Hide comment
@jwhitlock

jwhitlock Mar 23, 2014

Member

Sadly, they don't bump the date at the top of the file when they add columns. I've been looking at that and assuming I'm up to date. I'm not sure what the Feb 17th additions are. Anyway, opened issue #23 .

Member

jwhitlock commented Mar 23, 2014

Sadly, they don't bump the date at the top of the file when they add columns. I've been looking at that and assuming I'm up to date. I'm not sure what the Feb 17th additions are. Anyway, opened issue #23 .

@jwhitlock

This comment has been minimized.

Show comment
Hide comment
@jwhitlock

jwhitlock Mar 28, 2014

Member

They document these new optional fields on a different page:
https://developers.google.com/transit/gtfs/changes#RevisionHistory .

The 'wheelchair_accessible' field has been added in v0.3.3. However, Adelaide's latest feed still fails to import - next crash is a RouteGroup column in the route. So, leaving the ticket open until the 'ignore' feature ships/

Member

jwhitlock commented Mar 28, 2014

They document these new optional fields on a different page:
https://developers.google.com/transit/gtfs/changes#RevisionHistory .

The 'wheelchair_accessible' field has been added in v0.3.3. However, Adelaide's latest feed still fails to import - next crash is a RouteGroup column in the route. So, leaving the ticket open until the 'ignore' feature ships/

@jwhitlock jwhitlock added this to the v0.4.0 milestone Jun 11, 2014

@Permafacture

This comment has been minimized.

Show comment
Hide comment
@Permafacture

Permafacture Jun 11, 2014

A Json encoded field makes sense to me. Keep it in the database and let any users use the extra fields they import. If a field ever becomes officially recognized, users could migrate the data rather than dumping/reimporting.

A Json encoded field makes sense to me. Keep it in the database and let any users use the extra fields they import. If a field ever becomes officially recognized, users could migrate the data rather than dumping/reimporting.

@jwhitlock

This comment has been minimized.

Show comment
Hide comment
@jwhitlock

jwhitlock Jun 18, 2014

Member

Extra data is stored as strings in the new extra_data JSON field as a key-value dictionary. The feed has a new meta JSON field, which says which models contain additional columns. Extra columns and data appear in the exported GTFS feed. Scheduled for the 0.4.0 release.

Member

jwhitlock commented Jun 18, 2014

Extra data is stored as strings in the new extra_data JSON field as a key-value dictionary. The feed has a new meta JSON field, which says which models contain additional columns. Extra columns and data appear in the exported GTFS feed. Scheduled for the 0.4.0 release.

@jwhitlock jwhitlock closed this Jun 18, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment