Update reproducible example

.gitignore

@@ -40,3 +40,4 @@ vignettes/*.pdf
 *.html
 *.geojson
 *.csv
+*.Rds

README.Rmd

@@ -61,17 +61,23 @@ Please take a read of the information below and take a look at (and ideal test)
 
 If you plan to use R to develop solutions, please ensure you can reproduce the results of the code shown here: [reproducible-example.R](https://github.com/U-Shift/cyclingpotential-hack/blob/master/code/reproducible-example.R). If your output looks like [this](https://github.com/U-Shift/cyclingpotential-hack/blob/master/reproducible-example.md) congratualtions :tada:🎉 you have the necessary packages installed.
 
-## Agenda (draft)
+## Agenda
 
-- Introduction to the hackathon: 10:00 - 10:45
-- Talks: methods for modelling cycling networks: 10:45 - 11:15 
-- Live, interactive demo of data: 11:15 - 12:00
-- Topic pitches 12:00 - 12:10
+- Introduction to the hackathon: 10:00 - 10:40 (Rosa Felix and Robin Lovelace)
+<!-- How we started the collaboration, why Lisbon, aims (importance of free, open and participatory solutions) -->
+<!-- Robin on the methods and software underlying the PCT. -->
+- Participant introductions: 10:40 - 10:45 (participants)
+- Talks: methods for modelling cycling networks: 10:45 - 11:15 (Crispin Cooper and Lorena Abad)
+<!-- Crispin to talk about approaches for modelling active transport -->
+<!-- Lorena to talk about cycle network analysis in R -->
+- Live, interactive demo of data and hack rules: 11:15 - 11:45 (Rosa and Robin)
+- Topic pitches 11:45 - 12:00 (participants proposing topics)
 
-Lunch break and finalising teams 12:10 to 13:00
+Lunch break and finalising teams 12:00 to 13:00
 
 - Hackathon 13:00 - 15:30
 - Presentation of results 15:30 - 16:00
+<!-- ? someone to assess / 'most creative', best technicallay? -->
 
 ## Input datasets
 

README.md

@@ -1,102 +1,152 @@
 
 <!-- README.md is generated from README.Rmd. Please edit that file -->
-Cycling Potential Hackathon: Lisbon
-===================================
 
-<!-- badges: start -->
-<!-- badges: end -->
-Welcome to the Cycling Potential Hackathon, Lisbon (CPHL), on reproducible methods for estimating cycling potential, based on a case study of Lisbon, Portugal.
-![](https://user-images.githubusercontent.com/1825120/87960376-21e40a00-caac-11ea-8103-5948dfcd3a25.jpg)
-This [GitHub repo](https://github.com/U-Shift/cyclingpotential-hack) contains everything needed to get started with modelling cycling potential, including example data and reproducible code.
-
-Interested in getting involved? Here's some [key information](https://en.wikipedia.org/wiki/Five_Ws) that will help decide if this is for you and register to participate if so!
-
-Who?
-----
-
-**The hackathon is organised** by Rosa Felix of the [University of Lisbon](http://ushift.tecnico.ulisboa.pt/team-rosa-felix/) and Robin Lovelace of the [University of Leeds](https://environment.leeds.ac.uk/transport/staff/953/dr-robin-lovelace), who were awarded funding from the Portuguese Association of Researchers and Students in the United Kingdom's Bilateral Research Fund ([PARSUK BRF](https://en.parsuk.pt/brf)) to develop a '[PCT Portugal](http://ushift.tecnico.ulisboa.pt/pct-portugal/)' project building on methods used to develop the Propensity to Cycle Tool ([PCT](https://www.pct.bike/)), which is based on open source software.
+# Cycling Potential Hackathon: Lisbon
 
-**Attendees** are expected to have knowledge of transport planning interventions to enable cycling uptake and/or technical skills needed to analyse, interactively visualise and develop scenarios of cycling uptake. Experience with the statistical programming language R will be particularly useful, although anyone with experience of front-end or back-end development with open source software will be very welcome. **Working knowledge of GitHub is highly recommended for collaboration during the hackathon.** [Here is a tutorial example](https://www.youtube.com/watch?v=iv8rSLsi1xo) to get familiarized with GitHub.
-
-What?
------
+<!-- badges: start -->
 
-A one day participatory code-focussed hackathon. The morning will be dedicated to getting up-to-speed with the input data. The afternoon will be dedicated to the hackathon!
+<!-- badges: end -->
 
-When?
------
+Welcome to the Cycling Potential Hackathon, Lisbon (CPHL), on
+reproducible methods for estimating cycling potential, based on a case
+study of Lisbon, Portugal.  
+![](https://user-images.githubusercontent.com/1825120/87960376-21e40a00-caac-11ea-8103-5948dfcd3a25.jpg)  
+This [GitHub repo](https://github.com/U-Shift/cyclingpotential-hack)
+contains everything needed to get started with modelling cycling
+potential, including example data and reproducible code.
+
+Interested in getting involved? Here’s some [key
+information](https://en.wikipedia.org/wiki/Five_Ws) that will help
+decide if this is for you and register to participate if so\!
+
+## Who?
+
+**The hackathon is organised** by Rosa Felix of the [University of
+Lisbon](http://ushift.tecnico.ulisboa.pt/team-rosa-felix/) and Robin
+Lovelace of the [University of
+Leeds](https://environment.leeds.ac.uk/transport/staff/953/dr-robin-lovelace),
+who were awarded funding from the Portuguese Association of Researchers
+and Students in the United Kingdom’s Bilateral Research Fund ([PARSUK
+BRF](https://en.parsuk.pt/brf)) to develop a ‘[PCT
+Portugal](http://ushift.tecnico.ulisboa.pt/pct-portugal/)’ project
+building on methods used to develop the Propensity to Cycle Tool
+([PCT](https://www.pct.bike/)), which is based on open source software.
+
+**Attendees** are expected to have knowledge of transport planning
+interventions to enable cycling uptake and/or technical skills needed to
+analyse, interactively visualise and develop scenarios of cycling
+uptake. Experience with the statistical programming language R will be
+particularly useful, although anyone with experience of front-end or
+back-end development with open source software will be very welcome.
+**Working knowledge of GitHub is highly recommended for collaboration
+during the hackathon.** [Here is a tutorial
+example](https://www.youtube.com/watch?v=iv8rSLsi1xo) to get
+familiarized with GitHub.
+
+## What?
+
+A one day participatory code-focussed hackathon. The morning will be
+dedicated to getting up-to-speed with the input data. The afternoon will
+be dedicated to the hackathon\!
+
+## When?
 
 Friday 25<sup>th</sup> September, 2020, 10:00 to 16:00 (UTC±0)
 
-Where?
-------
+## Where?
 
 Online, we will send a conference link to participants.
 
-Why?
-----
-
-There is a great need for transparent and actionable evidence to support investment in sustainable transport futures, and cycling uptake in particular.
-
-Before signing-up
------------------
-
-Please take a read of the information below and take a look at (and ideal test) the following resources:
-
--   exercises in the [PCT Training course](https://itsleeds.github.io/pct/articles/pct_training.html)
--   Geocomputation with R online [book](https://geocompr.robinlovelace.net/), especially Chapter [12](https://geocompr.robinlovelace.net/transport.html) on transport data
--   Papers on the PCT: Lovelace et al. ([2017](https://doi.org/10.5198/jtlu.2016.862)) and Goodman et al. ([2019](https://doi.org/10.1016/j.jth.2019.01.008))
--   Lisbon cycling potential and bike network analysis: Abad and Van der Meer ([2018](https://doi.org/10.3390/info9110287) - paper); Abad and Van der Meer ([2018](https://github.com/GeoTecINIT/OpenData4OpenCities/blob/master/Presentations/AGILE_2018_Presentation_Abad-vdMeer.pdf) - presentation)
--   Test out the PCT at <https://www.pct.bike/>
-
-If you plan to use R to develop solutions, please ensure you can reproduce the results of the code shown here: [reproducible-example.R](https://github.com/U-Shift/cyclingpotential-hack/blob/master/code/reproducible-example.R). If your output looks like [this](https://github.com/U-Shift/cyclingpotential-hack/blob/master/reproducible-example.md) congratualtions :tada:🎉 you have the necessary packages installed.
-
-Agenda (draft)
---------------
-
--   Introduction to the hackathon: 10:00 - 10:45
--   Talks: methods for modelling cycling networks: 10:45 - 11:15
--   Live, interactive demo of data: 11:15 - 12:00
--   Topic pitches 12:00 - 12:10
-
-Lunch break and finalising teams 12:10 to 13:00
-
--   Hackathon 13:00 - 15:30
--   Presentation of results 15:30 - 16:00
-
-Input datasets
---------------
+## Why?
+
+There is a great need for transparent and actionable evidence to support
+investment in sustainable transport futures, and cycling uptake in
+particular.
+
+## Before signing-up
+
+Please take a read of the information below and take a look at (and
+ideal test) the following resources:
+
+  - exercises in the [PCT Training
+    course](https://itsleeds.github.io/pct/articles/pct_training.html)
+  - Geocomputation with R online
+    [book](https://geocompr.robinlovelace.net/), especially Chapter
+    [12](https://geocompr.robinlovelace.net/transport.html) on transport
+    data
+  - Papers on the PCT: Lovelace et
+    al. ([2017](https://doi.org/10.5198/jtlu.2016.862)) and Goodman et
+    al. ([2019](https://doi.org/10.1016/j.jth.2019.01.008))
+  - Lisbon cycling potential and bike network analysis: Abad and Van der
+    Meer ([2018](https://doi.org/10.3390/info9110287) - paper); Abad and
+    Van der Meer
+    ([2018](https://github.com/GeoTecINIT/OpenData4OpenCities/blob/master/Presentations/AGILE_2018_Presentation_Abad-vdMeer.pdf)
+    - presentation)  
+  - Test out the PCT at <https://www.pct.bike/>
+
+If you plan to use R to develop solutions, please ensure you can
+reproduce the results of the code shown here:
+[reproducible-example.R](https://github.com/U-Shift/cyclingpotential-hack/blob/master/code/reproducible-example.R).
+If your output looks like
+[this](https://github.com/U-Shift/cyclingpotential-hack/blob/master/reproducible-example.md)
+congratualtions :tada:🎉 you have the necessary packages installed.
+
+## Agenda
+
+  - Introduction to the hackathon: 10:00 - 10:40 (Rosa Felix and Robin
+    Lovelace)
+    <!-- How we started the collaboration, why Lisbon, aims (importance of free, open and participatory solutions) -->
+    <!-- Robin on the methods and software underlying the PCT. -->
+  - Participant introductions: 10:40 - 10:45 (participants)
+  - Talks: methods for modelling cycling networks: 10:45 - 11:15
+    (Crispin Cooper and Lorena Abad)
+    <!-- Crispin to talk about approaches for modelling active transport -->
+    <!-- Lorena to talk about cycle network analysis in R -->
+  - Live, interactive demo of data and hack rules: 11:15 - 11:45 (Rosa
+    and Robin)
+  - Topic pitches 11:45 - 12:00 (participants proposing topics)
+
+Lunch break and finalising teams 12:00 to 13:00
+
+  - Hackathon 13:00 - 15:30
+  - Presentation of results 15:30 - 16:00
+    <!-- ? someone to assess / 'most creative', best technicallay? -->
+
+## Input datasets
 
 Initial datasets, `.geojson`
 
-1.  Desire lines between cities
+1.  Desire lines between cities  
 2.  Cycling routes from [cyclestreets.net](cyclestreets.net)
-    -   Fastest
-    -   Balanced
-    -   Quietest
+      - Fastest  
+      - Balanced  
+      - Quietest
 
-[Go to Releases](https://github.com/U-Shift/cyclingpotential-hack/releases/)
+[Go to
+Releases](https://github.com/U-Shift/cyclingpotential-hack/releases/)
 
-Topics
-------
+## Topics
 
 We have developed ideas for a few hack topics:
 
--   Cycling uptake functions
--   Estimating flow between Origins and Destinations using spatial interaction models
--   Routing algorithms and routing services - hilliness integration?
--   Use of counting points data
--   Minimum requirements for a PCT at any place
--   How cycling responds locally to different intervention types, e.g. based on cycle counter data
+  - Cycling uptake functions  
+  - Estimating flow between Origins and Destinations using spatial
+    interaction models  
+  - Routing algorithms and routing services - hilliness integration?  
+  - Use of counting points data  
+  - Minimum requirements for a PCT at any place
+  - How cycling responds locally to different intervention types,
+    e.g. based on cycle counter data
 
-Any further ideas very welcome, feel free to bring your own!
+Any further ideas very welcome, feel free to bring your own\!
 
-How to sign up
---------------
+## How to sign up
 
-To sign-up complete [this](https://ushift.tecnico.ulisboa.pt/~ushift.daemon/limesurvey/629464) application form.
+To sign-up complete
+[this](https://ushift.tecnico.ulisboa.pt/~ushift.daemon/limesurvey/629464)
+application form.
 
-Any questions?
---------------
+## Any questions?
 
-Feel free to ask any questions related to this hackathon on the [issue tracker](https://github.com/U-Shift/cyclingpotential-hack/issues).
+Feel free to ask any questions related to this hackathon on the [issue
+tracker](https://github.com/U-Shift/cyclingpotential-hack/issues).

code/estimate-uptake-pct.R

@@ -12,8 +12,8 @@ library(sf)
 
 u3 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/routes_fast.geojson"
 route_segments_fast = sf::read_sf(u3)
-routes_fast = route_segments_fast %>% 
-  group_by(DICOFREor11, DICOFREde11) %>% 
+routes_fast = route_segments_fast %>%
+  group_by(DICOFREor11, DICOFREde11) %>%
   summarise(
     Origem = first(DICOFREor11),
     Destino = first(DICOFREde11),
@@ -22,7 +22,7 @@ routes_fast = route_segments_fast %>%
     Length_fast_m = sum(distances),
     Hilliness_average = mean(gradient_segment),
     Hilliness_90th_percentile = quantile(gradient_segment, probs = 0.9)
-  ) 
+  )
 
 unique(sf::st_geometry_type(routes_fast))
 nrow(routes_fast)
@@ -59,7 +59,7 @@ points(routes_fast$Length_fast_m, pcycle_pct_godutch, col = "green")
 points(routes_fast$Length_fast_m, pcycle_pct_govtarget, col = "grey")
 
 routes_fast$slc_godutch = routes_fast$All * pcycle_pct_godutch
-length(unique(routes_fast$geometry)) 
+length(unique(routes_fast$geometry))
 
 rnet_fast = overline(sf::st_cast(routes_fast, "LINESTRING"), attrib = "slc_godutch")
 rnet_fast$slc_godutch = round(rnet_fast$slc_godutch)

code/reproducible-example.R

@@ -3,39 +3,58 @@
 install.packages("remotes", quiet = TRUE)
 remotes::install_cran(c("sf", "stplanr", "pct", "tmap", "dplyr"), quiet = TRUE)
 
+# piggyback::pb_upload("od_data_final.csv")
+# piggyback::pb_upload("desire_lines_final.geojson")
+# piggyback::pb_upload("districts.geojson")
+# # piggyback::pb_download(file = "CENTROIDS.Rds", repo = "U-Shift/pct-lisbon2")
+# # centroids = readRDS("CENTROIDS.Rds")
+# st_write(centroids, "centroids.geojson")
+u_od = "od_data_final.csv"
+od_data = read.csv("od_data_final.csv")
+head(od_data)
+plot(od_data$Length_euclidean, od_data$pcycle_current)
+
+library(sf)
 # test the sf package
-u1 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/city_centroids.geojson"
-centroids = sf::read_sf(u1)
+# u1 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/city_centroids.geojson"
+u1 = "districts.geojson"
+u1b = "centroids.geojson"
+districts = read_sf(u1)
 plot(centroids)
+centroids_geo = st_centroid(districts)
+plot(districts$geometry)
+plot(centroids$geometry, add = TRUE)
+plot(centroids_geo$geometry, add = TRUE, col = "red")
 
 # check interactive mapping with tmap
 library(tmap)
 tmap_mode("view")
-u2 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/desire_lines_integers.geojson"
+# u2 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/desire_lines_integers.geojson"
+u2 = "desire_lines_final.geojson"
 desire_lines = sf::read_sf(u2)
 tm_shape(desire_lines) +
   tm_lines(col = "Bike", palette = "viridis", lwd = "Car", scale = 9)
 
 # check route network generation with stplanr
 library(stplanr)
-u3 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/routes_integers_cs_balanced.geojson"
+# u3 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/routes_integers_cs_balanced.geojson"
+u3 = "routes_fast.geojson"
 routes = sf::read_sf(u3)
+tm_shape(routes[]) +
+  tm_lines()
 rnet = overline(routes, "Bike") 
 tm_shape(rnet) +
-  tm_lines(lwd = "Bike", scale = 5, col = "Bike", palette = "viridis")
+  tm_lines(scale = 5, col = "Bike", palette = "viridis")
 
 # check analysis with dplyr and estimation of cycling uptake with pct function
 library(pct)
 library(dplyr)
-u3 = "https://github.com/U-Shift/cyclingpotential-hack/releases/download/1.0/routes_integers_cs_balanced.geojson"
 route_segments_balanced = sf::read_sf(u3)
 routes_balanced = route_segments_balanced %>% 
-  group_by(Origem, Destino) %>% 
+  group_by(DICOFREor11, DICOFREde11) %>% 
   summarise(
-    Origem = first(Origem),
-    Destino = first(Destino),
     Bike = mean(Bike),
-    All = mean(Bike) + mean(Car) + mean(Motorcycle) + mean(Transit) + mean(Walk) + mean(Other),
+    All = sum(Total),
     Length_balanced_m = sum(distances),
     Hilliness_average = mean(gradient_segment),
     Hilliness_90th_percentile = quantile(gradient_segment, probs = 0.9)
@@ -53,4 +72,22 @@ tm_shape(rnet_balanced) +
   tm_lines(lwd = "Potential", scale = 9, col = "Potential", palette = "viridis", breaks = b)
 
 # generate output report
-# knitr::spin(hair = "code/reproducible-example.R")
+# knitr::spin(hair = "code/reproducible-example.R")
+
+# # to convert OD data into desire lines with the od package you can uncomment the following lines
+# # system.time({
+# test_desire_lines1 = stplanr::od2line(od_data, centroids)
+# # })
+# # system.time({
+# test_desire_lines2 = od::od_to_sf(x = od_data, z = centroids)
+# # })
+# plot(test_desire_lines2)
+
+# test routing on a single line (optional - uncomment to test this)
+# warning you can only get a small number, e.g. 5, routes before this stops working!
+# library(osrm)
+# single_route = route(l = desire_lines[1, ], route_fun = osrm::osrmRoute, returnclass = "sf")
+# mapview::mapview(desire_lines[1, ]) +
+#   mapview::mapview(single_route)
+# see https://cran.r-project.org/package=cyclestreets and other routing services
+# for other route options, e.g. https://github.com/ropensci/opentripplanner