Magrit: Cartography Teaching+Learning as part of IIAB [with OSM?]

[holta]

We are hopeful @mir123 can work on this, in a virtual environment (venv) so that https://github.com/riatelab/magrit can avoid "dependency hell" and work with the Python 3.6 it requests.

We should assist @mir123 in identifying the best roles (Ansible playbooks) at https://github.com/iiab/iiab/tree/master/roles to model his work upon.

[holta]

A Docker approach is also possible but not recommended.

Using pip (instead of apt) to install gdal within the venv might be an option:

this requires gdal 2.2 or higher, Raspbian stretch is 2.1. So I could download and compile gdal from source (I actually did, but haven't installed yet) but that probably won't work as a recipe [Ansible playbook / role]. So I have installed gdal 2.4 from the testing repository, but that creates a cascade of updates... I don't know if I'm messing it up too much now

[mir123]

I have successfully installed Magrit in IIAB, it runs really well (seems to use max 20% CPU when loading map layers). I used venv with Python 3.5, compiled gdal 2.2. In order to eventually produce an Ansible recipe I made a .deb installer with the compiled gdal, would this be a feasible procedure? I would appreciate help with Ansible, it's a bit too much for me at this point!

[holta]

I have successfully installed Magrit in IIAB, it runs really well (seems to use max 20% CPU when loading map layers). I used venv with Python 3.5, compiled gdal 2.2. In order to eventually produce an Ansible recipe I made a .deb installer with the compiled gdal, would this be a feasible procedure? I would appreciate help with Ansible, it's a bit too much for me at this point!

Great news! And we will definitely help you with any questions you have about Ansible as it is not hard, and we have lots of experience after about 5 years now!

[mir123]

Would it be ok if I post here a description of my installation procedure for someone else in the team to pick it up? Also: what is a good place to put my gdal_2.2.3-1_armhf.deb?

[holta]

Would it be ok if I post here a description of my installation procedure for someone else in the team to pick it up?

Start with that yes!

Also: what is a good place to put my gdal_2.2.3-1_armhf.deb?

How big is it? We can publish it to http://d.iiab.io/packages if that helps, but there may also be better places long term.

[holta]

Tagging @m-anish, @georgejhunt, @tim-moody & @mapmeld so they might investigate Magrit prior to our Thursday 10AM NYC Time OSM + mapmaking call (http://minutes.iiab.io)

• https://github.com/riatelab/magrit
• http://magrit.cnrs.fr

[mir123]

OK, here are my install steps for Magrit:

$ sudo apt-get install -y libpython3.5-dev python3.5 python3.5-dev python3-pip \
    libopenblas-dev libopenblas-base libfreetype6-dev libfreetype6 libproj-dev \
    libspatialindex-dev libv8-3.14-dev libffi-dev nodejs nodejs-dev node-gyp \
    libuv1-dev libxslt1-dev libxml2 libxml2-dev libkml-dev libgeos-dev \
    npm git gcc redis-server wget libatlas-base-dev
$ sudo dpkg -i gdal_2.2.3-1_armhf.deb

(gdal 2.2 .deb can be found here, it's 34.6 mb)

$ sudo ln -s /usr/include/locale.h /usr/include/xlocale.h
$ sudo npm -g install topojson
$ sudo service redis-server start
$ cd /opt 

(What's a better location than /opt/magrit?)

$ git clone https://github.com/riatelab/magrit
$ cd magrit
$ virtualenv venv -p /usr/bin/python3.5
$ source venv/bin/activate
(venv)$ pip install numpy
(venv)$ pip install -r requirements/dev.txt

Modify /opt/magrit/magrit_app/app.py, lines 1414 and 1545

from: if sock.connect_ex(("0.0.0.0", port_nb)) == 0:
to: if sock.connect_ex(("172.18.96.1", port_nb)) == 0:

from: handler, '0.0.0.0', port)
to: handler, '172.18.96.1', port)

This is hardcoded in my installation, but better to pass the server address.

(venv)$ python setup.py install

Create /etc/systemd/system/magrit.service:

[Unit]
Description=Magrit
After=network-online.target

[Service]
Type=oneshot
ExecStart=/opt/magrit/venv/bin/magrit -p 9999

[Install]
WantedBy=multi-user.target

$ systemctl enable magrit.service

Magrit can be accessed at http://box.lan:9999. I don't know how to add this type of address to a menu def.

[mir123]

investigate Magrit prior to our Thursday 10AM NYC Time OSM + mapmaking call (http://minutes.iiab.io)

There might be other very simple and/or useful mapping apps that might run in IIAB, for example, Mapshaper (https://github.com/mbloch/mapshaper), good for simplifying shapefiles and other vector layers and to export as topoJSON.

[georgejhunt]

Nice work!

I have transcribed the magrit manual install recipe listed in this issue into ansible format. But I'm stuck getting it to execute. It seems to fail with conflicting requirements between magrit which now wants python3.6, and the the raspbian package which do not appear to be complete in python 3.6, the the recipe calls out 3.5 packages

To continue to debug, you will need to
put "magrit_install: True, magrit_enabled: True" into /etc/iiab/local_vars.yml.
in iiab repo, type

git remote add ghunt https://github.com/georgejhunt/iiab
git fetch --all
git checkout -b magrit ghunt/magrit
./runrole magrit

I'm not very good at python version conflicts, virtualenv, and such. But I can tweak the ansible, when we find out what needs to change.

I'm looking forward to playing with magrit when we get it working. I like the way their web site makes projections other than web-mercator visible.

[georgejhunt]

The ansible code that @mir123 and I generated is at https://github.com/iiab/iiab/pull/1579/files

[georgejhunt]

@mir123 Mir,
I need to generate a debian x86_64 version of gdal, similar to the one you generated for rpi3, so that the IIAB playbook can work on all our supported platforms. Have your already done that to run on your laptop?

If not, at least I should select the options for "./configure" similar to the ones you chose -- to try to minimize debugging issues, later in the process.

[holta]

@mapmeld & all:

A critical question we need help to answer is what Android apps to recommend for local community map-making and map-reading — as arose during Thursday's Internet-in-a-Box community call (http://minutes.iiab.io) outlining different approaches to grassroots civic cartography.

Please all make pragmatic recommendations here beyond just MAPS.ME if possible!

As we nurture techne / craftsmanship and cultural technics not just pyrotechnics that build understanding of local geographies, this too is tangentially related:

#1541 Can IIAB recommend/include an Android filesharing app for offline communities?

[mir123]

@mir123 Mir,
I need to generate a debian x86_64 version of gdal, similar to the one you generated for rpi3, so that the IIAB playbook can work on all our supported platforms. Have your already done that to run on your laptop?

@georgejhunt on my Linux Mint 19 (equivalent to Ubuntu Bionic) I just installed gdal from the repos (2.2.3), no need to compile, it's enough that it's at least 2.2 according to Magrit instructions

[mir123]

Also check improvements being considered to Magrit for IIAB deployment:

riatelab/magrit#46 (comment)

[holta]

Possible Magrit (and other OSM-like) Questions for the April 4 call: (http://minutes.iiab.io)

• What projections (mercator, polar, etc) are most useful for teachers?
  • Not just projections...map layers & their formatting & thematic/colors based on population...for teaching
  • Server address now customizable (not just 0.0.0.0)
  • “Sample layers addable soon”
• What OSM-in-Magrit integration should we be shooting for?
  • Mapeo brings much more customizability still...what community frameworks make most sense?
• What "show my GPS location on the map" functionalisty is most realistic within mainstream 2019 browsers, on Android especially?
• What aerial/satellite imagery objectives are most realistic for 2019?
  • Can/Should IIAB make downloading per-continent "cloudless" imagery available from Sentinel ? (Alongside @georgejhunt's per-continent vector map downloads that IIAB's Admin Console will offer soon.)
  • Are layers really needed for only 2 layers, or is a prominent toggle button enough?
  • Mapeo or simple leaflet page? (Simple JavaScript addition..but test first!)
• IIAB long-term roadmap: 10 collaborations/partnerships that Mir recommends towards participatory mapping pragmatism (ordered by potential/realizability for 2019/2020?)
  • QField 1.0 “Efficient field work built for QGIS” for Android appears extremely promising for offline (docs)
• ...?

[above revised with input from 2019-04-04 call]

[holta]

Mapping the Map strategy continues, with the above short-term roadmap ideas quickly discussed this morning with @mir123, @m-anish & @georgejhunt.

Medium-term roadmap ideas:
#877 (comment)

Next OSM/mapmaking tactics call will be Thursday 2019-05-02 (http://minutes.iiab.io)

[holta]

@georgejhunt should I defer this ticket to the IIAB 7.1 milestone, given you closed "Magrit playbook" PR #1579 on 2019-03-24 with the following message?

This PR was just to document my progress. It needs to be universalized to run on ubuntu, and debian, before it's ready for prime time