Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BKG data (Germany) #19

Closed
janusw opened this issue Dec 8, 2020 · 9 comments
Closed

Add BKG data (Germany) #19

janusw opened this issue Dec 8, 2020 · 9 comments

Comments

@janusw
Copy link
Contributor

janusw commented Dec 8, 2020

One of the strengths of OpenTopoData is its ability to handle various data sources (and the documentation on how to do it). I'm interested in adding yet another data source, namely the DGM data from the German 'Bundesamt für Kartographie und Geodäsie' (BKG).

They have data in different resolutions. The free version has a grid size of 200m, see: https://gdz.bkg.bund.de/index.php/default/digitale-geodaten/digitale-gelandemodelle/digitales-gelandemodell-gitterweite-200-m-dgm200.html (unfortunately it seems like this page is only available in German).

Of course this 200m data as such is not very interesting (since EU-DEM covers Germany with 25m resolution), but BKG also offers higher-resolution data commercially (down to 5m at least). Therefore it would be nice if one could make the 200m data work with OpenTopoData as a proof of concept at least.

The data is available in XYZ-ASCII or GRID-ASCII format (ARC/INFO), both of which seem to be supported by gdal (via the XYZ and AAIGrid drivers), so I hope it should be no problem to use them with OpenTopoData.

Feedback welcome ...

@janusw
Copy link
Contributor Author

janusw commented Dec 8, 2020

Ok, I think I already got it working. Here is what I did ...

- name: bkg200m
  path: data/bkg-dgm200/
  • Rebuilt the docker container and ran it.

That's about it. First tests seem to indicate that these data roughly agree with EU-DEM for the few sample points I tried, which I take as an indication that this works well in principle.

@janusw
Copy link
Contributor Author

janusw commented Dec 9, 2020

Actually the above procedure only works in this order:

  • First place only the asc file into the data directory.
  • Then run the docker image.
  • Only afterwards add the prj file.

If one places both files into the data directory before starting the server, one gets the error:

Config Error: Unknown dataset type for 'bkg200m'. Dataset should either be a single file, or split up into tiles with the lower-left corner coordinate in the filename like 'N20W120'.

Without the prj file one gets:

Dataset has no coordinate reference system. Check the file '{path}' is a geo raster. Otherwise you'll have to add the crs manually with a tool like gdaltranslate.

@ajnisbet What would be the best way to deal with this data?

@janusw
Copy link
Contributor Author

janusw commented Dec 9, 2020

One other thing that I noticed is that querying from the asc file seems to be quite slow (compared to EU-DEM, for example). This also means that I cannot query more than 20 points or so at a time, otherwise I get a gateway timeout (504).

I'll try converting it to compressed geotiff, as suggested here: https://www.opentopodata.org/datasets/emod2018/

@janusw
Copy link
Contributor Author

janusw commented Dec 9, 2020

Converting the asc to a tif solved both the problems with the prj file and the performance issues.

$ gdal_translate -co COMPRESS=DEFLATE -co PREDICTOR=2 dgm200_tm32.asc dgm200_tm32.tif
Input file size is 3918, 4854
0...10...20...30...40...50...60...70...80...90...100 - done.

The tif file alone seems to be sufficient now, and the prj is not needed any more. Also the file size is down from 134M to 30M, and it is much faster indeed.

@ajnisbet
Copy link
Owner

ajnisbet commented Dec 9, 2020

Thanks for bringing this up! Yeah Open Topo Data doesn't handle auxiliary files like .prj very elegantly, when it sees multiple files it thinks you're using a tiled dataset so complains that the .prj file isn't a proper raster.

The immediate fix is to add ".prj" to

AUX_EXTENSIONS = [".tfw", ".aux", ".aux.xml", ".rdd", ".jpw", ".ovr"]

I can't remember how I made that list of extensions, I thought I pulled it from the gdal source code and that it would be complete but clearly it isn't, so I'd like to do a bit more research to find an authoritative list or method to classify aux files.

@ajnisbet
Copy link
Owner

ajnisbet commented Dec 9, 2020

And yeah .asc is a non-binary format that is really slow, converting to .geotiff is a good move.

@ajnisbet
Copy link
Owner

ajnisbet commented Dec 11, 2020

Should be fixed by #20, thanks for the PR!

@ajnisbet
Copy link
Owner

I added the 200m dataset here: 7bf303b

@janusw
Copy link
Contributor Author

janusw commented Jan 19, 2021

I added the 200m dataset here: 7bf303b

Nice! Thank you so much.

Btw, I found one really small problem in that commit: The alt-text for the image is wrong ...

  <img src="/img/bkg200m.png" alt="EU-DEM elevation">

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants