Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

README.md

City SDK API Crawler and Visualizer for Istanbul Bus Network

This project aims to crawl bus route and stop informations from CitySDK and visualize the bus routes of Istanbul on a spatial map. Work has been presented at Akademik Bilişim 2016 conference at Aydın, Turkey. Paper and visuals can be seen in paper folder.

Installing Dependencies

cd crawler
virtualenv --distribute venv
. venv/bin/activate
pip install -r requirements.txt

Crawlera settings (Optional)

Crawlera allows using a proxy for crawling operation so that you are not banned from the system. Uncomment corresponding lines and enter your API key in settings.py for Crawlera. See more details on Crawlera manual.

Running the crawler

First run LineListSpider to get the list of routes.

scrapy crawl linelist -o iett/resources/linelist.csv

This will create a list of lines: iett/resources/linelist.csv. Some of the fields are not really needed. We conly need code and cdk. Run the following command:

Rscript simplify_lines.r

and this will create a simplified bus line list: iett/resources/bus_lines.csv


Now we will scrape bus line details:

scrapy crawl linedetail

This will produce linedetails.csv (bus stop sequence) and stops.csv (coordinates of the stops) in output folder.


Now we will visualize this.

Visualization

Open visualizer/istanbul-transportation-network.Rmd in RStudio and press Knit HTML button to see the results.

About

Istanbul Transportation Network Scraping & Analysis

Resources

Releases

No releases published

Packages

No packages published

Languages