Explore partially reproducing od2net approach

[Robinlovelace]

In conversation with @HemantSharmaTP the idea is to prototype the approach developed by @dabreegster in R, just for fun.

We really like the global applicability of this, example from Leeds:

https://urban-analytics-technology-platform.github.io/od2net//interactive.html#16.52/53.810078/-1.542723

[HemantSharmaTP]

That's how this looks in New Delhi. it's much more crowded though.

[Robinlovelace]

Also: not tied to R. Could use another language, key thing is ease of iteration. Maybe that it's worth contributing directly to the od2net project but for the use case we're talking about I suspect it will be quicker to rapidly prototype in a new repo. R/Python/other fine by me though!

[dabreegster]

What can you share about the use case, so I can understand od2net's shortcomings?

[Robinlovelace]

What can you share about the use case, so I can understand od2net's shortcomings?

Use case: being able to generate route networks from the command line in a programmable and easy to extend way. I think that is doable with od2net, we'd just need to learn a lot of Rust or work on bindings to R/Python. Could be that working on Urban-Analytics-Technology-Platform/od2net#28 is a quicker way to what we're after (a way to generate route networks in a way that's flexible and easy to modify/automate/extend for sustainable transport planning) that starting from scratch. Happy to have a go at a basic R (or even Python further outside my comfort zone) interface, especially following @JosiahParry's impressive work in this space but not really sure where to start.

[Robinlovelace]

So not really about shortcomings of od2net, more our ability to contribute to it. That is not fixed and we can think about contributing here... I think validation and generating plausible baseline scenarios and scenarios of change would be a good starting point, validated against counter datasets (which is where some data science/stats could come in handy).

[Robinlovelace]

Another use case: refactoring the backend code that generates the data shown in www.npt.scot

[dabreegster]

I don't think we really need bindings in another language to use od2net. You have to generate input data and config, then call one command to run od2net and produce output files. https://github.com/Urban-Analytics-Technology-Platform/od2net/blob/main/examples/oneshot/oneshot.py is a recent example that extracts a bounding box from Overpass and plugs in a basic OD pattern.

I started od2net largely to help stop reinventing the wheel with projects like NPT. If the current examples in there aren't adequate or the docs aren't enough (very true), I'd like to improve them.

[dabreegster]

I think validation and generating plausible baseline scenarios and scenarios of change would be a good starting point, validated against counter datasets

Agreed. I don't think any of those tasks belong in od2net itself; it's just a tool to do one particular calculation. The larger validation exercise or generating good OD pattern input is up to a per-use case project.

[Robinlovelace]

Happy to give it a go, currently hitting

./run_all.sh 
Checking dependencies
+ run_example york
+ echo 'Running example york'
Running example york
+ cd york
+ rm -rf input/ intermediate/ output/
+ python3 setup.py
Traceback (most recent call last):
  File "/home/robin/github/people/dabreegster/od2net/examples/york/setup.py", line 62, in <module>
    run(["mkdir", "-p", "input"])
NameError: name 'run' is not defined

on my laptop on latest version of main. If we create a minimal input dataset for Leeds would that be a good starting point? Creating an example/leeds could be a good way to get my head round it, debugging and providing feedback as I go.

Agreed, would be great to have a flexible tool that gets around reinventing the wheel.

[Robinlovelace]

od2net would be great as the modular + flexible network generation part of the wider process, all in on that, if I can get it working!

[dabreegster]

That's the same Python import error you've hit before. (I'll try and look at it tomorrow, but no promises with workload) You can also forget about the example scripts and just generate input yourself in the way that's comfortable.

[JosiahParry]

since od2net is made available as a lib we could, in theory, create R bindings to avoid general python dep tomfoolery, right? I havent had a chance to dive into this much

[dabreegster]

I think the right way to call od2net is as an executable. Calling the Rust API directly isn't really helpful. The steps to using it are conceptually:

1. Create input files however you like -- an osm.pbf extract, a .geojson file with points for origins and destinations, optionally a .csv for some origin/destination data, a config.json specifying some settings
2. cargo build the executable, then run and point to config.json
3. Use the output files

The Python examples are just doing step 1, and additionally taking dependencies on other tools like ogr2ogr and osmium. You can ignore the Python examples or the way they do things entirely if you like.

Edit: and I can wire up Github actions to build the executable for Mac/Linux/Windows if cargo build in step 2 is a hassle

[dabreegster]

I dunno if I've linked it, but https://github.com/Urban-Analytics-Technology-Platform/od2net/blob/main/docs/tutorial_new_area.md should be up-to-date and relevant here

[Robinlovelace]

od2net executable sounds fine, qgisprocess is close to best practice in terms of how to interface.. Open to ideas and keen to learn!