alternative base maps: naturalearthdata.com

[Schoeneh]

As discussed in #224 we need map tiles fitting our historical data (at least somewhat).

https://naturalearthdata.com provides vector and raster data useful for historical GIS (but not perfect).
I propose three additional base maps:

a) Similar to the one used by Tom Elliot in his updates for pleiades.stoa.org:

https://pleiades.stoa.org/news/blog/last-week-in-pleiades-12-19-february-2024

https://hcommons.social/@paregorios/111965277256064884

The shaded relief is Natural Earth 1:50 million Gray Earth with Shaded Relief, Hypsography, and Ocean Bottom
naturalearthdata.com/downloads
I use warp to reproject it to an orthographic projection customized to the week's content, and then style it.
Rivers are NE 1:10 million rivers & lakes centerline with scale ranks + tapering: naturalearthdata.com/downloads and styled based on the scaling.
Coastline and graticule are also natural earth.

b) Natural Earth 2

https://www.naturalearthdata.com/downloads/10m-raster-data/10m-natural-earth-2/

Natural Earth II is a raster map dataset that portrays the world environment in an idealized manner with little human influence.

c) Cross-blended Hypsometric Tints

https://www.naturalearthdata.com/downloads/10m-raster-data/10m-cross-blend-hypso/

Cross-blended hypsometric tints instead use modified elevation colors for regions that people presumably associate with the natural environment

[Schoeneh]

I'm planning on being able to provide web map tiles for all three base maps today :)

[Schoeneh]

@mfranke93 could you provide the shaded relief you used in #224 ?
I would like to try combining your shaded relief with some of the base maps

[mfranke93]

The tiles are available here. You will need the download script from the code repository, though.

[Schoeneh]

Using QGIS I was able to reproject the raster image "1:10m Cross-blended Hypsometric Tints" from EPSG:4326 to EPSG:3857 (Web Mercator).
Additionally, using the plugin QTiles, I rendered the whole raster image as a tiled web map (256x256, PNG, zoom levels 0-8):
https://box.hu-berlin.de/d/05cf6a472727471897a9/

I could do the same with "1:10m Natural Earth II" and could even increase the zoom levels to 0-12.
@mfranke93 Is this first attempt useful?

[mfranke93]

@mfranke93 Is this first attempt useful?

@Schoeneh could you maybe upload a ZIP file of the tiles to the cloud? The "download all as ZIP" function of the cloud fails because the files are too large. I would like to download all the files and put it into an interactive map to try it out in full :)

I had a cursory glance at some files. The tiles of levels 7 and 8 already look very pixelated. Level 6 looks okay, but not perfect either. Given the x-axis size of 21,600px of the source data, a maximum zoom level of 6 would seem sensible: log₂(21,600/256) ≈ 6.399. The fact that tiles of level 6 have a width of 337.5px in the source image also explains why the tiles look a bit pixelated: aliasing.

[Schoeneh]

@mfranke93 Here you go; I used different settings with QTiles to render multiple instances of '1:10m Natural Earth II' (zoom 0-7):
https://box.hu-berlin.de/d/b4a7594747b64aca8307

• jpg (quality 100%)
• jpg (100%) and 'Slippy Map Tiles'
• jpg (100%), 'Slippy Map Tiles' and "less jagged"
• png

[mfranke93]

Something seems off with the tiles:

.../{z}/{x}/{y}.png
I think they are just numbered the wrong way around in the y axis. y=0 should be the northernmost tile.

Edit: removed first image since I identified the issue.

[mfranke93]

Switching the numbering around on the y axis worked:

Probably QGIS can do that on export. For reference, I did it like this:

#!/usr/bin/env bash

set -euo pipefail

fromdir=NE2_HR_LC_SR_W_DR/
todir=fixed-naming/
extension=.png

for z in $(ls $fromdir)
do
  for x in $(ls $fromdir/$z)
  do
    numtiles=$(ls $fromdir/$z/$x | wc -l)

    num=$(($numtiles - 1))

    mkdir -p $todir/$z/$x

    for y_in in $(seq 0 $num)
    do
      y_out=$(( $num - $y_in ))
      cp $fromdir/$z/$x/$y_in$extension $todir/$z/$x/$y_out$extension
    done
  done
done

[mfranke93]

The LessJagged version seems broken, though:

[mfranke93]

In total, I like the first version! It is a bit unfortunate that it only goes down to zoom level 7, but it provides a nice reference layer. However, I would suggest using less saturated colors, because the foreground layer (the markers) use color heavily already and need to be distinct from the background. Maybe it does not have to be as greyscale as the Pleiades screenshot you showed, but something in that direction?

[Schoeneh]

I'm back from DHd2024 and able to work on the base maps again :)
Wow - that's really broken! I've no idea as to why - I'm simply using the QTiles Plugin with QGIS...
https://plugins.qgis.org/plugins/qtiles/

But I'll fiddle a bit - at least a less saturated png-version of NE2 is definitely possible!

[Schoeneh]

Saturation = 0

Saturation = -20

Saturation = -40

@mfranke93 I did some testing: -40 seems the best to me

[mfranke93]

@Schoeneh saturation=-40 looks like it could work, but maybe we want to offer an even lower saturation (10–20%) in addition?

Regarding the export: that is really weird, as I also see no option for the Y order in the export dialog in the video. OSM map tiles are definitely numbered north-to-south...

[Schoeneh]

Saturation = -60

Saturation = -80

Saturation = -90

[Schoeneh]

I rendered -40:
https://box.hu-berlin.de/f/c7a8a00a3d384aacbdb6/

and -80:
https://box.hu-berlin.de/f/120a27d21d964488b89d/

[mfranke93]

-40% saturation. I think this looks nice!

-80% saturation. But I think it is good to have this one as well. The contrast to the foreground colors is just nice to have. The light orange and light green colors from the markers sometimes blend a bit too much with the background in the -40% version, for example in the Nile delta.

Coincidentally, the tile numbering was correct this time around. Did you change anything, @Schoeneh?

Zoomed in. Of course, the map still looks a bit sad when you zoom in further, but I suppose that cannot be helped. And it still provides a frame of reference!

---

Sidenote. A slice of the map at the antimeridian is missing in all versions, probably because it is missing in NaturalEarth 2 as well? I suppose it does not matter for the project.

[Schoeneh]

@mfranke93 oh yeah! 🎉 Both seem like a good solution for some screenshots :)

I'm still working on increasing the resolution; I've got some ideas from https://gis.stackexchange.com/questions/313490/increasing-resolution-of-cartopy-stock-background

I unchecked "Use TMS tiles convention (Slippy Map by default)" - that should have done the trick.

RE: Sidenote - yes, that seems to be a problem with the original raster image...

[Schoeneh]

Should I also render a -60-version?

[mfranke93]

Should I also render a -60-version?

I personally prefer the -60 to the -40 one. But there should not be too many options, in my opinion. I would suggest you ask the people who are going to use it more often what they prefer, and what is most readable to them.

As a sidenote: we could also defer the saturation drop to the frontend. It could just be applied to the Leaflet TileLayer via CSS, and we could add a slider to the settings menu. Then, we would only need one set of tiles (100% saturation), which reduces storage space requirements also. That should not be too hard to implement, but the question is whether the CSS desaturation works in the same color space as that of QGIS. Worth exploring with the browser DevTools, I guess. I can take a look at that with the 100% saturation tiles at some point (maybe not today).

[Schoeneh]

I tried adding filter: saturate(0.6) to the .css, but was only able to desaturate the whole map including the markers...
(tried positions div#map and multiple .leaflet-container.$

I'm not up to speed as to how leaflet works - I wasn't able to find the the style-options for the different layers...

But overall, it looks like the saturation via css looks the same as in QGIS.

[mfranke93]

You'd probably want to target the tile pane of Leaflet:

.leaflet-pane.leaflet-tile-pane {
  filter: saturate(40%);
}

But good to know that the results are the same.

[Schoeneh]

That worked - thanks!

A side by side comparison looks fine to me (ignoring the difference in resolution)

• 40% via CSS:
  

• 40% in QGIS:
  

[Schoeneh]

I really like the idea of a slider for the saturation!
We could set the default to 0.5 and let the user choose. That also adds some more accessibility features :)

Maybe even some slider/toggle for the whole map view?
E.g. contrast, grayscale and invert?
(all done via CSS filter)

[mfranke93]

I really like the idea of a slider for the saturation!
We could set the default to 0.5 and let the user choose. That also adds some more accessibility features :)

That would be nice. Of course, it would need to be saved in the user state as well, and have an independent value for each selected base layer of the map (DARE, OSM, ...). That already adds quite a bit of implementation effort... And it needs to be documented as well.

I would be willing to implement that base functionality (at some point).

Maybe even some slider/toggle for the whole map view?
E.g. contrast, grayscale and invert?
(all done via CSS filter)

Nice to have, but I really don't have the time right now. We've thought about light and dark mode for Damast in its entirety as well, but it is a lot of effort. If you (@Schoeneh), or anyone else, wants to submit a PR, I would be happy to review it, though 😅 There is a lot to consider already regarding the default colors for the religious groups and contrast.

[Schoeneh]

I agree completely: Those features would be nice, but are need a considerable effort. I'm no frontend developer; therefore, I don't see myself submitting any PRs...

[Schoeneh]

But would it be possible to add a simple slider for the whole tiles: a kind of minimal viable product?
Using .leaflet-pane.leaflet-tile-pane {filter: saturate(0.5)} and a way of changing the value.

[mfranke93]

Reproducibility has been a key design aspect in Damast. With this minimum viable product, it is no longer possible to exactly reproduce screenshots from a state file. Besides, not all tile sets need the saturation reduced, so putting it to 50% for all tiles by default is not a good idea.

[Schoeneh]

I see your point :)
I'm mostly thinking about the screenshots we need for the upcoming publication - but could that would be solved by including a NE2-tiles with 50% saturation?
I could do another render with -50 and we could call it a day?

[mfranke93]

If it is primarily for the screenshots, it could also be done as you have during testing. Especially if you have the DevTools open anyways. You could also (temporarily) add the CSS to the user stylesheet.

I have to ask, though: if all of this is just for the screenshots, why not use the existing shaded relief map tiles, which are in higher level of detail (up to level 12) as well?

[Schoeneh]

It's not "just for the screenshots", those are just the only kind of time sensitive issues I've got in the back of my mind. :)
We (in Berlin) we're also talking/thinking about tiles similar to the ones of the 'Digital Atlas of the Roman Empire', only more general. That was when I thought/proposed 'Natural Earth 2' and wrote my first comment.

From my POV: We've got some working solutions. I'm working on a version of NE2 with higher resolution. The implementation of the slider etc. would be nice, but is not required. It would be nice to have a version of NE2 implemented, but a simple tile set with 50% saturation would be enough - I would rather not stress your limited time too much :)

But lets talk about that with the others on March 19th

[mfranke93]

But lets talk about that with the others on March 19th

So, to clarify: We should not make any screenshots for our chapter before that, then? That's why I was asking.

[Schoeneh]

Well, I'm not the one who ultimately decides...
Imho the screenshots should match the default view anyone will encounter on their first look at DAMAST. But in the end I can always recreate screenshots. I've done this multiple times over the years (to get a higher resolution, etc.).
My thoughts: Make screenshots as you need them / see fit. The potential work of recreating them with a different base map won't be yours. And I don't want to impede your work with something that ultimately doesn't matter for the publication :)

[Schoeneh]

@mfranke93 I finally created a base map as proof of concept that I'm happy with:
A combination of NE2 with Ocean Bottom and hillshaded via SARTM (zoom 0-10)

You can find the map tiles here:
https://box.hu-berlin.de/f/503b691a54c84a2ab359/

And a short documentation:
https://box.hu-berlin.de/f/b782a2ac15174ddeaeb7/

Unfortunately it took so long to render that only a part of it is finished - but at least I have something to show...

[mfranke93]

Seeing the screenshot now, I have some concerns regarding lightness. We already addressed saturation, but I feel like the dark parts and especially the high-frequency changes between light and dark patches (e.g., the southern Sinai peninsula in your screenshot) could interfere visually with the foreground. I would suggest reducing the contrast of the overall image so that at most 50% of the lightness range (probably 50%–100%) are used. Is that something that can be done in QGIS directly? Otherwise, we could take a look at CSS filters for that as well, but I don't know if it is possible to map lightness precisely that way. For my tiles, I actually only used the upper third of the lightness spectrum.

[Schoeneh]

That should be possible directly in QGIS.
I will try some things this Saturday/Sunday - both overall contrast and clipping the lightness spectrum.
Imho this shouldn't be done via CSS...

Your input is exactly why I rendered only a proof-of-concept :)

[Schoeneh]

Simply setting the contrast of the SRTM-layer to -50 seems to have done the trick:

The whole proof-of-concept as web-tiles:
https://box.hu-berlin.de/f/2e0ef68c10364bdb8b0b/

[mfranke93]

That looks good contrast-wise. Comparison of the previous version, this version, and my shaded relief tiles regarding hue, saturation, and value distribution:

Yours without contrast reduction (comment)

Yours with contrast reduction (comment)

SRTM shaded relief (screenshot used below)

Screenshot of SRTM shaded relief + Natural Earth water features

[mfranke93]

Some screenshots from within Damast:

Without reduced saturation

I think reducing saturation is still needed. And maybe the water color is too saturated overall, especially for the rivers. The blue color of the Jewish communities clashes with the Nile, I think.

With an additional filter: saturate(50%) on the tile layer