Custom hill shading algorithm

[Sublimis]

Creating a custom hill shading algorithm is currently severely restricted due to HgtCache.HgtFileInfo class being package private. Would it be possible to somehow lift this restriction (making the class public would suffice, but it might not be the most elegant solution)?

[devemux86]

@Sublimis

What customizations do you have in mind? Hillshade colors or something else?
Add more parameters in the algorithms?

It would be nice if we integrate these customizations so all users can use them.
Making some classes public is also possible.

[Sublimis]

So, we just started experimenting on whether hill shading can be improved (not sure about the outcome), and wanted to gain full control over the values written to the byte buffer used for creating AndroidHillshadingBitmap (ALPHA_8 config). However, this proved to be impossible because subclassing either public abstract class AbsShadingAlgorithmDefaults implements ShadingAlgorithm or even the public interface ShadingAlgorithm is futile.

To be more precise, implementing/overriding protected abstract byte[] convert(ByteBuffer map, int axisLength, int rowLen, int padding, HgtCache.HgtFileInfo source) -or- public RawShadingResult transformToByteBuffer(HgtCache.HgtFileInfo source, int padding) is impossible due to the aforementioned issue with HgtCache.HgtFileInfo privacy which is taken in as a parameter...

Many thanks!

[devemux86]

(Btw. just noticed, Jitpack build for the latest commit failed)

The first time JitPack builds the artifacts, if they have not been previously requested.
So it may take some time, now it seems to work correctly.

[Sublimis]

Please see here: https://jitpack.io/com/github/mapsforge/mapsforge/204c94093a/build.log 🤷

[devemux86]

JitPack master-SNAPSHOT was built green and also tested in vtm-android-example:

https://jitpack.io/#mapsforge/mapsforge/master-SNAPSHOT

[demiantres]

Is the new hillshading implementation ready (except bug fixes), or are the bigger changes expected?

[devemux86]

@demiantres

All hillshading improvements and the new algorithm are already available in the master branch.

Some extra improvements are prepared by @Sublimis, see the discussion here:

• Hill shading improvements + new algorithm #2 #1521 (comment)

[Sublimis]

Here are the results.
All screenshots were made using the latest Motorider theme included with the library (except for the Google screenshot, of course).

The next several screenshots are using the hill shading magnitude scales of both 3 and 1 for better comparison.

All screenshots are for illustrative purposes.

First our hill shading algorithm as finally used in the app (scale = 3):

Compare with the default Simple algorithm (scale = 3),

the default default Simple algorithm (scale = 1),

the default Diffuse Light algorithm (scale = 3) -- please observe how particularly bad the Lim and Raša fjords look like:

and the default default Diffuse Light algorithm (scale = 1):

Here is our algorithm with scale = 1 (just for completeness):

For completeness, here is the screenshot of Google terrain map ("legacy" renderer):

and again the Motorider map but without any hill shading:

Our biggest objection with the current hill shading algorithms is that they always show a gray overlay over an otherwise flat terrain, reducing contrast of the map, making it darker and, well, uglier.

Our algorithm is based on the Simple one, nothing too fancy, but because of a change in paradigm it won't ever put a shade over a flat terrain. Additional bonus is that this also makes hills and fjords look nicer, and closer to what we see in Google maps (we can provide additional screenshots). If you're interested in including the code in the library, we need a few more days to pretty up the code etc. Also the algorithm doesn't have a name yet, so suggestions are welcome.

[Sublimis]

Screenshots are just for illustrative purposes, the Google logo indicates that Google library was used for making tile screenshots.

The SimpleShadingAlgorithm algorithm was a starting point, but it is in fact a completely new algorithm, in its own java file, extending AbsShadingAlgorithmDefaults.

The performance is about the same, no notable difference when used. Instead of sin(sin(sin())) it computes one abs() and one sqrt(). Sqrt is just our choice for non-linearity, it could as well be a linear (or any other) function, but this is not the bottleneck.

The real bottleneck for all these algorithms is reading DEM files into a memory, which is ~25MB per file for a 1" resolution.
We noticed that the library maps DEM files from file system to memory, only to later read them sequentially byte by byte (short by short to be exact). This feels like a waste of resources, and can be implemented more efficiently using a simple BufferedInputStream. The DemFileFs.openInputStream() creating a BufferedInputStream is already there. Our algorithm also makes use of this, but it required some changes to the aforementioned files not directly related to the algorithm. You can see some details here: master...Sublimis:mapsforge:master. How do you feel about such a change, abandoning memory mapped files and using a simple stream?

[devemux86]

@Sublimis There is no problem to add a new algorithm in the library.

abandoning memory mapped files and using a simple stream?

Nice, it's better to keep fewer things in memory. I suppose this works on Android and Desktop?

Can you create a pull request, so we can check the new algorithm?

[Sublimis]

Nice, it's better to keep fewer things in memory. I suppose this works on Android and Desktop?

InputStream-s are standard java.io classes, so yes.

Can you create a pull request, so we can check the new algorithm?

Done!

[devemux86]

@Sublimis thanks for the pull request! 👍

I merged it in Mapsforge.

[Sublimis]

Thank you, it's great to hear this!

[devemux86]

@Sublimis

Which hgt files did you use for your screenshots above?

[Sublimis]

Those are 1 arcsecond files downloaded from the Viewfinder Panoramas 1" interactive coverage map