Add BKG data (Germany)

[janusw]

One of the strengths of OpenTopoData is its ability to handle various data sources (and the documentation on how to do it). I'm interested in adding yet another data source, namely the DGM data from the German 'Bundesamt für Kartographie und Geodäsie' (BKG).

They have data in different resolutions. The free version has a grid size of 200m, see: https://gdz.bkg.bund.de/index.php/default/digitale-geodaten/digitale-gelandemodelle/digitales-gelandemodell-gitterweite-200-m-dgm200.html (unfortunately it seems like this page is only available in German).

Of course this 200m data as such is not very interesting (since EU-DEM covers Germany with 25m resolution), but BKG also offers higher-resolution data commercially (down to 5m at least). Therefore it would be nice if one could make the 200m data work with OpenTopoData as a proof of concept at least.

The data is available in XYZ-ASCII or GRID-ASCII format (ARC/INFO), both of which seem to be supported by gdal (via the XYZ and AAIGrid drivers), so I hope it should be no problem to use them with OpenTopoData.

Feedback welcome ...

[janusw]

Ok, I think I already got it working. Here is what I did ...

• Downloaded the data from https://daten.gdz.bkg.bund.de/produkte/dgm/dgm200/aktuell/dgm200.tm32.gridascii.zip (the zipfile is just around 40MB, the uncompressed data are roughly 140MB).
• Took the two files dgm200_tm32.asc and dgm200_tm32.prj and placed them into the folder opentopodata/data/bkg-dgm200/.
• Added this to config.yaml:

- name: bkg200m
  path: data/bkg-dgm200/

• Rebuilt the docker container and ran it.

That's about it. First tests seem to indicate that these data roughly agree with EU-DEM for the few sample points I tried, which I take as an indication that this works well in principle.

[janusw]

Actually the above procedure only works in this order:

• First place only the asc file into the data directory.
• Then run the docker image.
• Only afterwards add the prj file.

If one places both files into the data directory before starting the server, one gets the error:

Config Error: Unknown dataset type for 'bkg200m'. Dataset should either be a single file, or split up into tiles with the lower-left corner coordinate in the filename like 'N20W120'.

Without the prj file one gets:

Dataset has no coordinate reference system. Check the file '{path}' is a geo raster. Otherwise you'll have to add the crs manually with a tool like gdaltranslate.

@ajnisbet What would be the best way to deal with this data?

[janusw]

One other thing that I noticed is that querying from the asc file seems to be quite slow (compared to EU-DEM, for example). This also means that I cannot query more than 20 points or so at a time, otherwise I get a gateway timeout (504).

I'll try converting it to compressed geotiff, as suggested here: https://www.opentopodata.org/datasets/emod2018/

[janusw]

Converting the asc to a tif solved both the problems with the prj file and the performance issues.

$ gdal_translate -co COMPRESS=DEFLATE -co PREDICTOR=2 dgm200_tm32.asc dgm200_tm32.tif
Input file size is 3918, 4854
0...10...20...30...40...50...60...70...80...90...100 - done.

The tif file alone seems to be sufficient now, and the prj is not needed any more. Also the file size is down from 134M to 30M, and it is much faster indeed.

[ajnisbet]

Thanks for bringing this up! Yeah Open Topo Data doesn't handle auxiliary files like .prj very elegantly, when it sees multiple files it thinks you're using a tiled dataset so complains that the .prj file isn't a proper raster.

The immediate fix is to add ".prj" to

opentopodata/opentopodata/config.py

Line 16 in 3620454

AUX_EXTENSIONS = [".tfw", ".aux", ".aux.xml", ".rdd", ".jpw", ".ovr"]

I can't remember how I made that list of extensions, I thought I pulled it from the gdal source code and that it would be complete but clearly it isn't, so I'd like to do a bit more research to find an authoritative list or method to classify aux files.

[ajnisbet]

And yeah .asc is a non-binary format that is really slow, converting to .geotiff is a good move.

[ajnisbet]

Should be fixed by #20, thanks for the PR!

[ajnisbet]

I added the 200m dataset here: 7bf303b

[janusw]

I added the 200m dataset here: 7bf303b

Nice! Thank you so much.

Btw, I found one really small problem in that commit: The alt-text for the image is wrong ...

  <img src="/img/bkg200m.png" alt="EU-DEM elevation">