Visible walls at quantized mesh terrain tiles boundary

[xtassin]

After experimenting with several settings for ctb-tiles, I keep getting artefacts along terrain tile junctions (see screenshot)

I managed to reduce these occurrences by using low values (0.2 to 0.5) for -g switch (mesh-qfactor) at the cost of increased tile weight.

Tests were made with several SRTM data sets with similar problem each time.

Aattached, the 1deg SRTM tile used for the screenshot:
N43E007.SRTMGL1.hgt.zip

Ccommand line used to generate terrain tiles:
ctb-tile -f Mesh -C -N -o /terrain N43E007.hgt

[xtassin]

@ahuarte47, if that may be of any indication, lowering mesh-factor below 0.5 actually reversed the walls but did not make them disappear.

[ahuarte47]

@xtassin, I think that mesh-factor is not the way to go. I think it has to be a bug in the code, maybe when working with negative heights (I see some of them in the data). But, sorry I can't study it, my apologies.

[xtassin]

@ahuarte47 thank you for the pointer. Do you think you will work on this project again at some point or is there anybody maintaining it at the moment?

[ahuarte47]

As far I know at the present time nobody maintains this, but it would be very much useful to know if the negative heights are the cause of this issue.

[xtassin]

I would like to keep documenting this issue as I investigate it. Please someone notify me if this is not the right place to do it (as this is related to the master-quantized-mesh pull request)

@ahuarte47 I am wondering if the lod algorithme should takes care of matching vertices at tiles boundaries. As you can see from this screenshot, the walls appear where the mesh is more refined on one neighbour tile than the other.

[ahuarte47]

Hi @xtassin: I downloaded your SRTM tile and I have verified this issue.

You are right, the lod algorithm should takes care of vertices at tiles boundaries. I will try to figure out how to fix it.

[xtassin]

Thank you @ahuarte47 for looking into this. I also noted that the "walls" are accentuated when processing a larger set of SRTM tiles aggregated with gdalbuildvrt. This may simply be caused by the LOD algo taking higher altitudes from the surroundings into account. But it could also be a precision issue.

[ahuarte47]

Hi @xtassin, I think I fixed this issue. If you agree, please, could you test the update?

Before:

After:

Now it is 4x slower than old code :-(. For each tile, the LOD implementation reads their four neighbours. This could be optimised using a single tile-cache in the MeshTiler class (At least with the top-border and bottom-border neighbours).

[xtassin]

Thanks @ahuarte47 ! I am struggling a bit to compile the branch but I will let you know as soon as I have a result.

[ahuarte47]

I am struggling a bit to compile the branch but I will let you know as soon as I have a result.

You can use this dockerfile, it is directly using my branch #64 (comment)

[xtassin]

Thank you. It looks like the docker repo has not been updated yet.

[xtassin]

@ahuarte47 I managed to build your branch and process the test SRTM tile provided in this thread. The result is fine and the walls a not there anymore.

However, when I tried to run ctb-tile on the whole SRTM set (as I did before) I got the following error:

Error: The minimum X value is greater than the maximum X value

I run ctb-tile against a .vrt file that aggregate the whole dataset.

The vrt file was generate with the following command to make sure it is cropped to correct bounds and uses proper projection:

gdalbuildvrt -te -180 -90 180 90 -a_srs EPSG:4326 all.vrt *.hgt

This used to work with the latest stable branch.

[xtassin]

I also need to point out that I built CTB against GDAL 2.0.1 (built from source) in order to bypass a compilation error that occurred with GDAL latest.

[ahuarte47]

Hi @xtassin, Thanks for your tests.

I managed to build your branch and process the test SRTM tile provided in this thread. The result is fine and the walls a not there anymore.

I am glad to read that walls disappeared.

However, when I tried to run ctb-tile on the whole SRTM set (as I did before) I got the following error:

Error: The minimum X value is greater than the maximum X value

You reported same error a time ago #64 (comment) I think this error comes from GDAL when it tries to create tiles of highest zooms (Low levels). May I have a DEM to try fix this error? This old link does not work.

[xtassin]

@ahuarte47. Thank you for your feedback.

I was careful to generate the .vrt file with the -te switch in order to specify bounds. This is how I fixed the issue last time. And this was working before the "walls" fix.

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

For now, I managed to start the process by creating the vrt file with a more agressive crop (-te -179 -89 180 90)

I will update you once I have processed the whole world ;)

Thank you so much for your help fixing this wall problem, this is much appreciated.

[ahuarte47]

Hi @xtassin, thanks you for your feedback!

I have pushed a new commit to speed up the reading of MeshTiler taking care of neighbours. It uses a very simple cache of raster heights in order to avoid the reading of data from the source GDAL dataset as much as possible.

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

I am going to look at this. Thanks for your advice.

[ahuarte47]

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

Fixed

[BWibo]

Hey @xtassin @ahuarte47,
I just updated the Docker image.

@ahuarte47 It should be possible to setup a webhook to trigger a build everytime you push a new commit to your repo.

[ahuarte47]

Hi @BWibo, thanks!
About the webhook, sorry, I don't have any experience, do I have to execute a git command to raise a specific event to your repo?

[BWibo]

Hey there,
this is new for me as well.
I believe you will have to setup a webhook in your git repo (Settings -> Webhooks) that points to my Travis CI repo and triggers a build everytime you push to the repo. I will have to provide a token or some kind of access credentials for you I assume.
I hope to find time to look into this later this week. I will update you here when I know how it's done.

For now I setup a daily build of the image. You can look up when the last build happened here:
https://travis-ci.org/tum-gis/cesium-terrain-builder-docker

[xtassin]

@ahuarte47 thank you so much for your reactivity. I just rebuilt with your latest fix and launched a worldwide SRTM processing. The bounds issue is gone. I can't tell about performance yet but it appears to be much slower than before the tiling fix. I will update you after processing is complete.

@BWibo thank you for stepping in: your docker image makes things a lot easier.

[BWibo]

@ahuarte47 Hey there, I think I found a way to trigger a build of the image every time you push to
https://github.com/ahuarte47/cesium-terrain-builder.
I made contact with you on Linkedin as I need to share non-public information with you to do this.

You will receive a trigger URL that you need to send a POST to. So you simply need to setup an empty webhook for this URL for the push event (Settings -> Webhooks).

[BWibo]

Thx @ahuarte47,
the trigger has been setup successfully, the Docker image should now be updated on every push to https://github.com/ahuarte47/cesium-terrain-builder.

The Docker image is built on Docker Cloud's infrastructure. Depending on the current load, the build process takes roughly 10 to 20 minutes to complete. Check this page for the build status:
https://hub.docker.com/r/tumgis/ctb-quantized-mesh/builds

[ahuarte47]

Thank you very much @BWibo !

[xtassin]

@ahuarte47, I just finished processing a worldwide SRTM dataset. The resulting mesh seems to be fine: no walls or cracks to be seen. I believe it took about twice the time to process the set compared to previous builds. Thank you so much for you help.

[ahuarte47]

Hi @xtassin, if you agree, you could close this issue.

[xtassin]

I am re-opening this issue as there are still some case where the mesh matching between tiles is failing. I will try to gather more data and find a test case to reproduce the problem.

[ahuarte47]

Hi @xtassin, is it possible that these gaps are in the border of your raster? Are the adjacent tiles in the image outside of your no-data input raster and come them from global Cesium grid? If this is true, I am sorry there is no solution, they are never taken care of them when the mesh tile creation is performed.

Anyway in order to run faster, neighborhood is only considered for tile levels higher than 6 and some gaps can be appear at high camera zooms.

[xtassin]

Here is a srtm file that produces walls http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_35_06.zip

However, I am not sure this wall is caused by unmatched mesh vertices as before.

The wall (in the screenshot) is visible on the main runway at Agadir airport 30°22'48.9"N 9°32'10.0"W
(30.380244, -9.536106)

The wall is "inside" the srtm tile bounds (not at data boundary)

[ahuarte47]

Thanks @xtassin , I will have a look