Expand on docker/binder descriptions #6
Conversation
|
Oh, one thing more: Currently, using the MyBinder version is a bit annoying as it (i think) requires first downloading the OSM history file to one's local machine to then upload it back into the MyBinder VM. @amandasaurus is there a way to get authenticated/signed links for downloading the Bypassing the down-to-then-upload step could help in making the process a bit easier/accessible too (one can tell I'm traumatised by German DSL upload speeds - and other places with terrible speeds 😂) |
Yes, that would be easier. There's a python library, |
|
That looks like it could work! I’ll play around with it a bit next week and see I can plug that in! |
|
@gedankenstuecke This is a really good idea! Unfortunately when I upload the 600 MB 
Retrying didn't help. :( |
|
How long does it approx. take until it starts hanging? I know that the MyBinder folks cull their "inactive" containers after ~15 min or so of inactivity, and depending on how it's implemented it might interpret waiting for the upload to finish as "inactivity". But let's see if downloading the files from within the containers can in general help with that, I think generally the approach that @amandasaurus shared should be doable! |
@gedankenstuecke I'd happily measure this, but now the container won't start :( 
|
|
Yeah, I ran into the same issue last night, sometimes MyBinder runs out of compute resources, which leads to this. In that case one can unfortunately just wait! |
|
@gy-mate I managed to incorporate @amandasaurus's suggestion for how one can download files right into the Binder in an experimental branch. You can click here to launch the binder, it should have a new little section for authenticating against Geofabrik's download server to download the files right into the container. In my test on MyBinder earlier, it downloaded the full history file for Argentina in ~25 seconds: Once I've tested a bit more I'll make another PR (likely next week as I have my own belated wedding party to attend over the weekend 😉)! |
|
Awesome and congrats! :) 🎉 |
|
Thanks! <3 |
Following some feedback from folks that have tried the Docker approach, I went ahead and also made the whole image compatible with MyBinder, which allows running containers in a free-to-use cloud infrastructure provided by the Jupyter folks.
The main requirement for that is to have a graphical interface that people can use. For that I included Jupyter notebooks/lab in the Docker image, which made the size go up a bit. But it also allowed to create a more user-friendly entry point for creating the images.
The notebook itself can be seen here, at least for the non-interactive parts. Once executed, it creates a form-based frontend for entering the necessary parameters (see image below). And a live version can be tested by using this link, which will spawn the MyBinder version.
I've then also adapted the version that would run when launching the container locally to run Jupyter with the notebook, as this removes the need for the connecting into the container with a second shell etc. The readme should reflect those changes too 🙂
@gy-mate If you're up for another try, I'd love some more feedback!