Repository navigation
Baker
vgl-bake is the command-line tool that turns source rasters (digital elevation models, bathymetry grids, satellite imagery, a geoid grid) into vgl tile packs, and checks packs on disk and on web servers. It is the only part of vgl that uses GDAL, and it runs on Windows and Linux only. This page covers building it, every command and option, the configuration file, recipes for the datasets vgl was designed around (Copernicus GLO-30, GEBCO, EGM2008, Blue Marble, Sentinel-2 cloudless) and for your own GeoTIFFs, choosing levels and regions, resuming bakes, what a bake writes, and a full regional bake from scratch. The project hosts no data: you bake the packs you need and host them yourself (Hosting packs).
The tool's own reference is tools/baker/README.md; the format it writes is specified in docs/pack-format.md and summarized in Pack format overview.
- What the baker does
- Building it
- Commands
- The configuration file
- Baking elevation
- Baking imagery
- The geoid pack
- Choosing levels, regions and shards
- Resuming and rebaking
- What a bake writes
- Checking packs
- Performance and disk space
- Worked example: a regional bake from scratch
- Pitfalls
A bake reads a JSON configuration that lists sources in priority order and the pack's settings, and writes one pack: a directory with a manifest (pack.vglm) and shard files (*.vgls). Each pack holds one layer: elevation, imagery or the geoid. The runtime stacks several packs per layer by priority, so a global base pack (say GEBCO) and regional detail packs (say GLO-30 over one country) combine at run time without rebaking either.
For every tile, the baker resamples the sources directly onto that tile's own sample grid with GDAL's warper, using a fixed kernel scale so that each sample depends only on its position. Neighbouring tiles, even in different shards, therefore compute identical shared borders, and shards bake independently and in parallel. Near a source's own resolution it interpolates (bilinear by default); at levels coarser than 1.5 times the source's pixel size it filters by area averaging (or by the maximum, if you ask). Heights referenced to EGM2008 (mean sea level) are converted to ellipsoidal heights with a geoid pack, with the same interpolation the runtime uses.
Bakes are deterministic: the same sources and configuration give byte-identical packs, whatever the number of threads. For imagery this holds for the same JPEG and PNG encoders, which come with GDAL.
The baker is off by default. Turn it on with the CMake option VGL_BUILD_BAKER. Its requirements:
| Dependency | Version | Where it comes from |
|---|---|---|
| GDAL | 3.7 or later | Found, never downloaded: system packages on Linux, conda-forge on Windows. CMake stops with an error if it is missing. |
| nlohmann/json | 3.10 or later | Used if installed, otherwise downloaded (3.11.3), unless VGL_FETCH_DEPENDENCIES is off |
| Zstandard | As for the rest of vgl | |
| libcurl | Only for vgl-bake check. It is the libcurl of the defaults module's HTTP reader, so VGL_DEFAULTS_HTTP must be on. Without it, the baker builds, and check says it was built without libcurl. |
The two baker presets set VGL_BUILD_BAKER=ON, VGL_DEFAULTS_HTTP=ON and a static build (BUILD_SHARED_LIBS=OFF), and the matching workflow presets configure, build and run the tests in Release. CI builds both (baker-linux, baker-windows in .github/workflows/ci.yml).
| Preset | Platform | Notes |
|---|---|---|
linux-gcc-baker / workflow linux-gcc-baker-release
|
Linux, GCC | System GDAL |
windows-msvc-baker / workflow windows-msvc-baker-release
|
Windows, MSVC | GDAL from conda-forge; CMAKE_PREFIX_PATH is $env{CONDA_PREFIX}/Library, so the environment must be active |
The packaging recipes (vcpkg port, Conan recipe) build vgl without the baker; build it from the repository as below.
On Ubuntu 24.04 (what CI uses):
sudo apt-get install libgdal-dev nlohmann-json3-dev libzstd-dev libcurl4-openssl-dev ninja-build
cmake --workflow --preset linux-gcc-baker-releaseThe tool is then build/linux-gcc-baker/bin/Release/vgl-bake.
GDAL comes from conda-forge, through micromamba. The environment file tools/baker/environment.yml pins libgdal-core=3.10, nlohmann_json=3.11 and zstd=1.5. Create it once, activate it so that CONDA_PREFIX is set, and use the baker preset:
micromamba create -p D:\vgl-env -f tools\baker\environment.yml
micromamba activate D:\vgl-env
cmake --workflow --preset windows-msvc-baker-releaseThe tool is then build\windows-msvc-baker\bin\Release\vgl-bake.exe. To run it outside the test preset, activate the environment first, so that GDAL's DLLs and the PROJ database are found. Without the environment active, the program cannot load its DLLs.
Keep the environment out of normal (non-baker) builds: if it is active when you configure another preset, CMake may pick up its packages, such as its Zstandard, and the resulting programs then fail to start without the environment's DLLs on the path.
The repository documents only the two setups above. The CMake code calls find_package(GDAL 3.7 CONFIG) and then the module mode, so any GDAL 3.7 or later that CMake can find through CMAKE_PREFIX_PATH (a vcpkg gdal package, an OSGeo4W or self-built installation) should work, with -DVGL_BUILD_BAKER=ON -DVGL_DEFAULTS_HTTP=ON. Those setups are not tested in CI.
vgl-bake bake <config.json> [--jobs N] [--resume] [--max-level L] [--cache MB] [--verbose]
vgl-bake inspect <pack>
vgl-bake validate <pack>
vgl-bake query <pack> < points.txt
vgl-bake check <url> [--sample N] [--timeout S]
vgl-bake --version
vgl-bake --help
<pack> is a pack's directory on disk (the one that holds pack.vglm). Exit codes: 0 on success, 1 on an error (printed as vgl-bake: <message>) or a failed check, and 2 for a usage error such as an unknown command.
Builds the pack a configuration describes.
| Option | Default | Meaning |
|---|---|---|
--jobs <n> |
all cores | Worker threads. Work is split by shard, so more threads than shards do not help. |
--resume |
off | Continue an interrupted bake, keeping the shards it finished. Also required to bake into a directory that already holds a bake. |
--max-level <l> |
the configuration's max_level
|
Preview: bake only down to level l (never deeper than the configuration). |
--cache <MB> |
2048 | GDAL's raster block cache, in megabytes. |
--verbose |
off | Report every shard (level, position, tiles, size, time) instead of a progress counter. |
It prints progress (N of M shards) on standard error and a summary at the end:
baked 234778 tiles in 206 shards, 1777.0 MB, in 60.2 s
Prints the manifest (format version, layer, tile size, levels, shard level, bounds, metadata), the shard count and total size, a table per level (tiles, partial tiles, size, bytes per tile, bits per sample and, for heights, the quantization step), and a table of the shard counts and median, largest and base shard sizes that every other shard level would give. Use it to choose shard_level (see below).
Reads and decodes every tile, which checks every CRC and structure. For heights it then checks that shared borders are identical (across the antimeridian too), that pole rows are uniform, and that child masks and child height bounds agree with the tiles. For imagery it checks that each image has the tile's size plus its border, that borders repeat the neighbouring tiles' pixels (to within a mean difference of 12 per channel, which JPEG's losses stay well inside), and that child masks agree. It exits with 1 if any check fails. Run it on every pack before you publish it.
Reads latitude longitude lines in degrees from standard input and prints, for each, the latitude, longitude, the ellipsoidal height from the deepest tile that covers it, and that tile's level:
echo "61.6363 8.3125" | vgl-bake query packs/glo30-norwayUseful to spot-check a pack against known summits or benchmarks. Remember the heights are ellipsoidal, not above mean sea level (in Norway, roughly 40 m above heights over mean sea level).
Checks a pack on a web server, by the URL of its directory, as vgl and browsers read it: the manifest, byte ranges, compression, CORS and caching headers, every shard's size and digest (or a sample), and one shard's directory and a tile. See Checking a host.
| Option | Default | Meaning |
|---|---|---|
--sample <n> |
every shard | Check n shards spread over the pack, for large packs |
--timeout <s> |
30 | Seconds a request may take |
vgl-bake sample-tile <config.json> <level> <x> <y> is an undocumented diagnostic command, not shown in --help: it prints one elevation tile's heights before quantization, one row per line, as the bake would compute them. It does not apply to imagery.
A configuration is a JSON file. Comments (// and /* */) are allowed. Relative paths (output, geoid, source files) are relative to the configuration file, not the working directory. The configurations of the test bake are in tools/baker/examples, and those of the sample data in samples/data.
| Key | Default | Meaning |
|---|---|---|
output |
required | Directory the pack is written to |
layer |
"elevation" |
"elevation", "imagery" or "geoid"
|
tile_size |
129; imagery 256
|
Heights: samples per tile side, 2^k + 1 with 2 <= k <= 10 (65, 129, 257, ...). Imagery: pixels per side, a power of two from 64 to 2048 |
border |
imagery 1
|
Imagery only: pixels repeated from the neighbouring tiles around each tile, 0 or 1 |
encoding |
"auto" |
Imagery only: "auto" (JPEG for tiles with data everywhere, PNG with alpha for the others), "jpeg" (everywhere, gaps filled with black) or "png" (everywhere, alpha only where needed) |
jpeg_quality |
85 |
Imagery only: JPEG quality from 1 to 100 |
min_level, max_level
|
0, 12
|
Range of levels in the pack, at most 28 |
shard_level |
max_level - 5 (0 if max_level <= 5) |
Level of the shard roots. Above max_level, the whole pack is one base shard |
region |
the globe |
[west, south, east, north] in degrees: only tiles that intersect it are baked |
filter |
"average" |
Heights only: how coarser levels are filtered: "average", or "maximum" so that terrain is never shown lower than it is (aviation) |
height_steps |
see below | Heights only: one quantization step in metres per level from min_level to max_level, each a power of two |
zstd_level |
19 |
Zstandard compression level |
geoid |
Path of a geoid pack, required if any source uses "datum": "egm2008"
|
|
metadata |
Object of strings or lists of strings: name, description, attribution, license, source, ... |
|
sources |
required | Sources, highest priority first |
The baker adds a generator metadata entry (vgl-bake <version>) itself. It does not write created, which would make identical bakes differ. The runtime reports the attribution entries of the packs in view through vgl_context_attribution_get (Attribution and licensing), so put the credit line each dataset's license requires there, verbatim.
Default height steps are 0.5 m to level 5, 0.25 m to level 9 and 0.125 m beyond, and 2^-8 m (3.9 mm) for geoid packs. A tile's heights must fit in 65,534 steps; one that does not fails the bake with a message naming the tile, rather than being clipped. If that happens (very steep terrain or deep trenches next to summits in one coarse tile), set height_steps with a coarser step at that level.
The border, encoding and jpeg_quality keys are rejected for heights, and filter, height_steps, geoid, datum and sea_blend are rejected for imagery.
| Key | Default | Meaning |
|---|---|---|
name |
source N |
Name used in messages |
files |
required | A path or a list of paths. The file name (last component only) may contain * and ?. Matches are sorted. A source matching no files is an error |
datum |
"ellipsoid" |
Heights only: "ellipsoid" (heights above WGS84) or "egm2008" (heights above mean sea level, as in Copernicus DEM and GEBCO) |
max_level |
the pack's | Deepest level this source contributes to |
resampling |
"bilinear" |
GDAL resampling method near the source's resolution, such as "cubic"
|
fill_only |
false |
Fill only the gaps of tiles that earlier sources cover, adding no tiles of its own |
sea_blend |
0 |
Heights only: a distance in metres; see Sea at the edge of a regional pack |
Where a source has no data (its nodata value, or NaN), the next source fills in. A tile is baked only if it intersects the region and some source that is not fill_only covers it at that level.
-
Coordinates. Sources must be north-up rasters in geographic WGS84 coordinates, such as EPSG:4326. Reproject others with
gdalwarpfirst. A plain image without georeferencing (such as NASA's PNGs) can be georeferenced withgdal_translate -a_srs EPSG:4326 -a_ullr <west> <north> <east> <south>. - Format. Anything GDAL reads: GeoTIFF, Cloud Optimized GeoTIFF (COG), VRT, even a GDAL WMTS description. All files of one source must have the same number of bands.
-
Imagery. 1 (grey) or 3 (RGB) bands of 8-bit values. A pack is grey only if all its sources are. Convert others with
gdal_translate(for example-b 1 -b 2 -b 3 -ot Byte -scale). Four-band RGBA sources are rejected: drop the alpha band and mark missing pixels with a nodata value instead. - Speed. Tiled files with overviews (COGs with average overviews) make the coarse levels fast. Files stored in strips without overviews, such as GEBCO's GeoTIFFs as distributed, make every coarse tile read the full-resolution data.
GLO-30 is a global 1 arc-second (about 30 m) surface model, heights above EGM2008, with the sea set to exactly 0 m. Its tiles are on the AWS open data mirror as COGs:
https://copernicus-dem-30m.s3.amazonaws.com/<tile>/<tile>.tif
with the tile names (such as Copernicus_DSM_COG_10_N61_00_E007_00_DEM) in https://copernicus-dem-30m.s3.amazonaws.com/tileList.txt. Tiles that would be all sea do not exist. Use "datum": "egm2008" and a geoid pack. Level 12 (about 38 m) matches the source; level 13 nearly triples the pack while mostly interpolating (see Performance and disk space).
Attribution: Produced using Copernicus WorldDEM-30 © DLR e.V. 2010-2014 and © Airbus Defence and Space GmbH 2014-2018 provided under COPERNICUS by the European Union and ESA; all rights reserved.
The GEBCO grid covers land and ocean floor at 15 arc-seconds (about 463 m), heights above mean sea level (treated as EGM2008). Download the "ice surface elevation" GeoTIFF from gebco.net (4.2 GB zip for GEBCO 2026). Its GeoTIFFs are stored in strips without overviews, so convert each once to a tiled COG with average overviews:
mkdir -p sources/gebco-cog
for f in gebco_2026_*.tif; do
gdal_translate -of COG -co COMPRESS=ZSTD -co PREDICTOR=2 -co BLOCKSIZE=512 \
-co OVERVIEW_RESAMPLING=AVERAGE "$f" "sources/gebco-cog/$f"
doneA global GEBCO base pack is tools/baker/examples/gebco-global.json: levels 0 to 8, which is slightly coarser than the source. Level 9 oversamples GEBCO and triples the pack; use it only if bathymetry is seen up close.
{
"output": "packs/gebco-global",
"layer": "elevation",
"tile_size": 129,
"min_level": 0,
"max_level": 8,
"geoid": "packs/egm2008-geoid",
"metadata": {
"name": "GEBCO 2026",
"attribution": "GEBCO Compilation Group (2026) GEBCO 2026 Grid",
"source": "GEBCO 2026 Grid, ice surface elevation (mean sea level)"
},
"sources": [
{ "name": "GEBCO 2026", "files": "sources/gebco-cog/gebco_2026_*.tif", "datum": "egm2008" }
]
}The runtime can draw an optional sea surface over bathymetry (Atmosphere, stars and clouds and the backend pages).
There are two ways to combine datasets, and they are often used together:
-
Separate packs, stacked at run time. Bake a global base pack and regional detail packs, and add them all to the context with priorities (
vgl_pack_create_info.priority, see Context API). Where the regional pack has data it wins; elsewhere, and in its nodata gaps, the base pack shows. This keeps each pack small and lets you update regions independently. -
Several sources in one pack. List them in
sources, highest priority first. Where the first has no data, the next fills in. With"fill_only": true, a source fills gaps in tiles the earlier sources cover but adds no tiles of its own, so the pack keeps to the first source's footprint. Usemax_levelon a source to stop a coarse source from being sampled at deep levels.
Copernicus DEM sets the sea to exactly 0 m, while GEBCO, which fills in around a regional GLO-30 pack, has the seabed there, often hundreds of metres lower. Where the GLO-30 pack ends at sea, the two surfaces would meet with a step, which the runtime's skirts fill as a wall.
With "sea_blend": 10000 on the GLO-30 source and GEBCO as the next source with "fill_only": true, GLO-30's sea within 10 km of the pack's edge takes GEBCO's seabed at the edge, eased to sea level at 10 km. Only the sea near the edge changes: fjords and lakes inside the coverage keep GLO-30's flat water. This is tools/baker/examples/glo30-norway.json:
{
"output": "packs/glo30-norway",
"layer": "elevation",
"tile_size": 129,
"min_level": 6,
"max_level": 12,
"region": [4, 57, 32, 72],
"geoid": "packs/egm2008-geoid",
"metadata": {
"name": "Copernicus GLO-30, Norway",
"attribution": "Produced using Copernicus WorldDEM-30 © DLR e.V.",
"source": "Copernicus DEM GLO-30 2021_1 (EGM2008)"
},
"sources": [
{
"name": "Copernicus GLO-30",
"files": "sources/glo30/Copernicus_DSM_COG_10_*.tif",
"datum": "egm2008",
"sea_blend": 10000
},
{
"name": "GEBCO 2026",
"files": "sources/gebco-cog/gebco_2026_*.tif",
"datum": "egm2008",
"fill_only": true
}
]
}Do not use sea_blend when the pack below is much coarser than the full GEBCO grid (as with the tiny sample GEBCO pack): blending would only pull the fjords near the edge down to a coarse depth.
Any GDAL-readable DEM works once it meets the source requirements:
# A projected DEM (for example a national grid in UTM) to geographic WGS84, as a COG.
gdalwarp -t_srs EPSG:4326 -r bilinear -of COG -co COMPRESS=ZSTD -co PREDICTOR=2 \
-co OVERVIEW_RESAMPLING=AVERAGE my-dem.tif sources/mine/my-dem-4326.tifChoose the datum by what the heights are measured from:
- Heights above the WGS84 ellipsoid (GNSS-derived, lidar in ellipsoidal heights):
"datum": "ellipsoid", no geoid needed. - Heights above EGM2008:
"datum": "egm2008"with a geoid pack. - Heights in another vertical datum (EGM96, a national datum): the baker supports only these two, so convert to one of them first, for example with GDAL's vertical transformations (
gdalwarpwith a compound source CRS and a target CRS such as EPSG:4979), and check a few points afterwards withvgl-bake query.
Mark voids with the band's nodata value so that the next source or pack fills them. For a fine source (say 1 m lidar) over a small area, raise max_level to match its resolution (level reference) and use a region and min_level that fit the area.
Imagery packs use "layer": "imagery". The defaults are 256-pixel tiles with a 1-pixel border, JPEG at quality 85 where a tile has data everywhere and PNG with alpha elsewhere. Each imagery pack is a layer of its own at run time, blended over the layers of lower priority, with an opacity and priority the host changes at run time (vgl_pack_opacity_set, vgl_pack_priority_set). A tile is written only if one of its own pixels has data.
The format also defines WebP, BC1 and BC7 tile encodings, but the baker writes only JPEG and PNG, and the runtime reads only those two today; the design's optional BC1/BC7 pre-encoding is not implemented yet.
NASA's Blue Marble: Next Generation (with topography and bathymetry) has about 15 arc-seconds (about 460 m) per pixel at full resolution; its deepest useful level is 7 (about 610 m per pixel). The sample pack uses the 5400 x 2700 PNG for July (world.topo.bathy.200407.3x5400x2700.png, Visible Earth record 73751), georeferenced to the whole globe:
gdal_translate -a_srs EPSG:4326 -a_ullr -180 90 180 -90 \
world.topo.bathy.200407.3x5400x2700.png sources/bluemarble/world.topo.bathy.200407.3x5400x2700.tifand samples/data/bluemarble-sample.json:
{
"output": "sample-packs/bluemarble",
"layer": "imagery",
"tile_size": 256,
"border": 1,
"min_level": 0,
"max_level": 3,
"shard_level": 4,
"jpeg_quality": 85,
"metadata": {
"name": "Blue Marble: Next Generation, July (sample, to level 3)",
"attribution": "NASA Earth Observatory, Blue Marble: Next Generation (Reto Stöckli)",
"source": "world.topo.bathy.200407.3x5400x2700.png, NASA Visible Earth"
},
"sources": [
{
"name": "Blue Marble: Next Generation",
"files": "sources/bluemarble/world.topo.bathy.200407.3x5400x2700.tif",
"resampling": "cubic"
}
]
}For a full-resolution global pack, georeference each of NASA's 21600 x 21600 pieces of the mosaic with its own bounds (each covers 90 by 90 degrees), list them all in files (a wildcard works), and set max_level to 7 with the default shard_level (2).
EOX's Sentinel-2 cloudless 2016 mosaic is licensed CC BY 4.0 (later years are CC BY-NC-SA). The sample pack reads EOX's WMTS layer s2cloudless at zoom 11 of its WGS84 tile matrix set, which is exactly the pack tiling's level 11, through a GDAL WMTS description, s2cloudless.xml:
<GDAL_WMTS>
<GetCapabilitiesUrl>https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml</GetCapabilitiesUrl>
<Layer>s2cloudless</Layer>
<TileMatrixSet>WGS84</TileMatrixSet>
<ZoomLevel>11</ZoomLevel>
<BandsCount>3</BandsCount>
</GDAL_WMTS>Save the region to a GeoTIFF once, with a margin of 256 pixels at level 11 (0.087890625 degrees) on every side so that the tiles at the region's edges get their borders:
# Region 5.625..8.4375 E, 60.46875..61.875 N, plus the margin. -projwin is west north east south.
gdal_translate -projwin 5.537109375 61.962890625 8.525390625 60.380859375 \
-co COMPRESS=DEFLATE -co TILED=YES -co PREDICTOR=2 \
s2cloudless.xml sources/s2cloudless/s2cloudless-2016-sognefjord.tifThat reads 612 tiles of EOX's service. Larger regions read many more, so check EOX's terms of use for bulk access first. The pack is samples/data/s2cloudless-sample.json, levels 7 to 11 with the region set, attributed Sentinel-2 cloudless by EOX IT Services GmbH (Contains modified Copernicus Sentinel data 2016).
Note that the 2016 mosaic shows glaciers and snowfields that its processing took for cloud as a flat near-black fill (RGB 11, 18, 26); the baker keeps it as it is.
Orthophotos and other GeoTIFFs work once they are 8-bit grey or RGB, in EPSG:4326:
# Reproject to WGS84 geographic, keep RGB, mark outside pixels as nodata 0, tile with overviews.
gdalwarp -t_srs EPSG:4326 -r cubic -dstnodata 0 -of COG -co COMPRESS=DEFLATE \
-co OVERVIEW_RESAMPLING=AVERAGE ortho.tif sources/ortho/ortho-4326.tifIf your file is RGBA, select the colour bands with -b 1 -b 2 -b 3 (with gdal_translate) and use a nodata value instead of alpha. Choose max_level so that the pack's pixel size at that level is close to the source's: a 256-pixel tile at level L spans 180 / 2^L degrees, so the pixel size is 180 / 2^L / 256 degrees (level 11: about 38 m north to south; level 13: about 9.5 m; level 15: about 2.4 m). Over a region, keep the source a little larger than region (one tile's margin, as above), so the edge tiles' borders have data.
Packs hold ellipsoidal heights. A geoid pack holds the EGM2008 geoid height and is needed twice: by the baker, to convert sources referenced to mean sea level ("datum": "egm2008"), and by the runtime, if the host wants heights above mean sea level. Bake it first, because the other bakes read it.
The source is NGA's 2.5 arc-minute EGM2008 grid as distributed by PROJ (80 MB, public domain): https://cdn.proj.org/us_nga_egm08_25.tif. The configuration is tools/baker/examples/egm2008-geoid.json:
{
"output": "packs/egm2008-geoid",
"layer": "geoid",
"tile_size": 129,
"min_level": 0,
"max_level": 5,
"shard_level": 6,
"metadata": {
"name": "EGM2008 geoid",
"attribution": "EGM2008 geoid model, National Geospatial-Intelligence Agency (NGA)",
"source": "EGM2008 2.5' grid (NGA)"
},
"sources": [
{
"name": "EGM2008 2.5'",
"files": "sources/geoid/us_nga_egm08_25.tif",
"datum": "ellipsoid",
"resampling": "cubic"
}
]
}Its source datum is "ellipsoid" because the values are themselves geoid heights above the ellipsoid. shard_level 6, above max_level, makes the whole pack one base shard. A geoid pack must cover the whole globe at every level without nodata, so do not give it a region. The result is about 22 MB, and differs from GeographicLib's evaluation of the same grid by 3.3 mm RMS (at most 5.2 cm, near Mount Everest).
Ship the same geoid pack with your application that you baked your elevation with: the conversion round-trips exactly only with the same geoid tiles.
Levels. Match max_level to the source's resolution. For 129-sample elevation tiles at the equator:
| Level | Tile span | Sample spacing | Typical source |
|---|---|---|---|
| 5 | 5.625° | 4.9 km | EGM2008 geoid |
| 8 | 0.703125° | 611 m | GEBCO (463 m) |
| 11 | 0.0879° | 76 m | |
| 12 | 0.0439° | 38 m | Copernicus GLO-30 (31 m) |
| 13 | 0.0220° | 19 m | Finer national DEMs |
Going one level deeper than the source quadruples the deepest level's tile count and mostly stores interpolation. For a quick look, bake with --max-level a few levels short.
Regions and min_level. A regional pack should start at a level whose tiles are not much larger than the region. A coarse tile that the region only partly covers holds data only there (a partial tile), and the area outside the region but inside that tile shows coarse detail at that level. The Sognefjord sample starts at level 7 for this reason: a level-6 tile would hold its whole tile, so the half south of the region would show level-6 detail. Tiles intersecting region are baked; the region also becomes the manifest's (informative) bounds.
Shard level. The default max_level - 5 gives shards of about 10 to 25 MB for every dataset in the test bake, and directories that fit in one 64 KiB range read. Override it only after looking at vgl-bake inspect, whose last table shows the shard counts and sizes of every other shard level. For small packs (samples, tests, the geoid), a shard level above max_level makes the whole pack one base shard. Smaller shards make regional updates and partial downloads finer, at the cost of more files.
Each finished shard is recorded in <output>/.bake (a small text file per shard, plus one for the base shard), and every file is written to a .partial name first and then renamed, so an interrupted bake never leaves a half-written shard behind. Continue it with:
vgl-bake bake glo30-norway.json --resumeShards with a record are kept; the rest are baked, then the base shard and the manifest. Without --resume, a bake into a directory that already holds pack.vglm or .bake stops with ... already holds a bake; pass --resume to continue it, or remove the directory.
--resume needs the same configuration and --max-level as the run that started the bake: they are recorded in .bake/settings.txt, and a resume after either has changed stops with ... holds a bake with another configuration or --max-level; --resume needs the same ones, or remove the directory to bake again. The configuration is compared as parsed, so comments and spacing do not matter. The source files themselves are not compared: after replacing a source file under the same name, remove the output directory (or bake into a new one) instead of resuming. Bake a --max-level preview into a directory of its own.
There is no command for rebaking only one region of an existing pack. Rebake the whole pack into a new directory: because bakes are deterministic, shards whose sources did not change come out byte-identical with the same file names, so an upload of the new pack (see Updating packs) transfers only the shards that changed, plus the manifest and the base shard if those changed.
<output>/
pack.vglm Manifest: layer, levels, height steps, shard index, metadata
base-<digest>.vgls Base shard: all tiles above the shard level (if any)
shards/<y>/<x>-<digest>.vgls One shard per level-S tile with data, for its subtree
.bake/ Resume records: shard-<x>-<y>.txt, base.txt and settings.txt
<digest> is eight hexadecimal digits of the shard's CRC-32C header checksum, so a rebaked shard with different content gets a different name. The .bake directory is only for --resume: keep it while you might resume, and do not publish it (the upload commands in Hosting packs copy only pack.vglm and *.vgls, and the reference server never serves hidden paths). A pack directory without .bake is complete and self-contained; copy it as a whole.
To use a pack locally, pass its directory to vgl_pack_create (uri), or to the samples' --pack option (Samples).
Before publishing:
vgl-bake validate packs/glo30-norway # every tile, seam, pole and child mask
vgl-bake inspect packs/glo30-norway # sizes per level and shard level
echo "61.6363 8.3125" | vgl-bake query packs/glo30-norwayAfter uploading, check the host as vgl and browsers read it:
vgl-bake check https://packs.example.org/glo30-norway
vgl-bake check https://packs.example.org/glo30-norway --sample 50 # large packsFigures from the M1 test bake (docs/test-bake.md), on 32 logical CPUs with 31 GB of RAM under WSL 2, GDAL 3.8.4, 129-sample tiles:
| Pack | Levels | Tiles | Size | Bake time |
|---|---|---|---|---|
| EGM2008 geoid | 0 to 5 | 2,730 | 21.8 MB | 12 s |
| GEBCO global | 0 to 8 | 174,762 | 2,166 MB | 60 s |
| GEBCO global | 0 to 9 | 699,050 | 6,793 MB | 237 s |
| GLO-30 Norway (337 source tiles, 5.4 GB) | 6 to 12 | 234,778 | 1,777 MB | 60 s |
| GLO-30 Norway | 6 to 13 | 934,408 | 5,276 MB | 226 s |
- Throughput was 3,000 to 4,200 tiles per second. A global GLO-30 bake at level 12 (about 18 million tiles) would take a bit over an hour on that machine, and level 13 about five hours.
- Pack size. Elevation tiles take about 7 to 10 KB each at 129 samples (about 3.4 bits per sample at GLO-30's level 12). GLO-30 at level 12 extrapolates to about 140 GB globally, and level 13 to about 410 GB. A global GEBCO base at level 8 is 2.2 GB.
- Disk space. Plan for the sources (GLO-30 is about 16 MB per 1-degree tile; GEBCO's COGs a few GB), the packs, and the GEBCO COG conversion. The bake itself needs no scratch space beyond the shard being written.
- Sources. Prepare sources as tiled COGs with average overviews; this made GEBCO's coarse levels fast. Keep sources on a local SSD rather than a network share.
-
Memory. GDAL's block cache is 2 GB by default (
--cache). Raise it for many large sources if memory allows; lower it, and--jobs, on small machines. - Tile size. Keep 129 for elevation: 65 costs 5 to 9% more space and four times the tiles, and 257 saves only 3 to 5% while coarsening the level-of-detail steps.
This bakes the full test bake set on Linux: the geoid, a global GEBCO base and GLO-30 over Norway with GEBCO at its edges, then validates and previews them. Adapt the region for your own area.
1. Build the baker (see Building it) and put it on the path:
cmake --workflow --preset linux-gcc-baker-release
export PATH="$PWD/build/linux-gcc-baker/bin/Release:$PATH"2. Lay out a data directory with the configurations from tools/baker/examples:
mkdir -p ~/vgl-data/sources/{geoid,gebco-cog,glo30}
cp tools/baker/examples/*.json ~/vgl-data/
cd ~/vgl-data3. Download the geoid grid:
curl -fL -o sources/geoid/us_nga_egm08_25.tif https://cdn.proj.org/us_nga_egm08_25.tif4. Download GEBCO from gebco.net (the ice surface elevation GeoTIFF zip), unzip it, and convert it to COGs as shown in GEBCO bathymetry, into sources/gebco-cog.
5. Download the GLO-30 tiles for the region. This loop requests every 1-degree tile from 57 to 71 N and 4 to 31 E (the region [4, 57, 32, 72]); tiles over open sea do not exist and are skipped:
for lat in $(seq 57 71); do
for lon in $(seq 4 31); do
t=$(printf "Copernicus_DSM_COG_10_N%02d_00_E%03d_00_DEM" "$lat" "$lon")
curl -fsL -o "sources/glo30/$t.tif" \
"https://copernicus-dem-30m.s3.amazonaws.com/$t/$t.tif" || rm -f "sources/glo30/$t.tif"
done
done
ls sources/glo30 | wc -l # the test bake had 337 tiles for this regionFor a region in the western or southern hemisphere, the names use S and W (for example Copernicus_DSM_COG_10_S34_00_W071_00_DEM); tileList.txt on the mirror lists every name.
6. Bake and validate, geoid first:
vgl-bake bake egm2008-geoid.json && vgl-bake validate packs/egm2008-geoid
vgl-bake bake gebco-global.json && vgl-bake validate packs/gebco-global
vgl-bake bake glo30-norway.json && vgl-bake validate packs/glo30-norway
vgl-bake inspect packs/glo30-norwayIf a bake is interrupted, run the same bake command with --resume.
7. Look at the result with a sample, stacking the packs (later --pack options win over earlier ones; see Samples). The baker presets do not build the samples; use a build that does, such as linux-gcc-shared:
build/linux-gcc-shared/bin/Release/vgl-headless --pack ~/vgl-data/packs/gebco-global --pack ~/vgl-data/packs/glo30-norway \
--pack ~/vgl-data/packs/egm2008-geoid8. Publish the packs/* directories (Hosting packs) and check them with vgl-bake check.
For your own region, copy glo30-norway.json, change output, region, metadata and the files, and pick min_level so that a tile at that level is not much larger than the region (level 6 tiles are 2.8 degrees, level 7 tiles 1.4 degrees).
-
Paths are relative to the configuration file, not to the directory you run
vgl-bakefrom. - "source ... matches no files". Wildcards apply to the file name only, not to directories, and the directory must exist.
-
"must use geographic coordinates on WGS84". Reproject with
gdalwarp -t_srs EPSG:4326first. -
"sources referenced to egm2008 need "geoid"". Bake the geoid pack first and point
geoidat its directory. -
"already holds a bake". Pass
--resumeto continue, or remove the output directory for a fresh bake. A resume after changing the configuration or--max-levelis refused. -
A tile's heights do not fit. Coarsen
height_stepsat the level the message names. -
Seams in imagery at a region's edge. Give the source a margin of one tile beyond
region, so the edge tiles' borders have data. -
Windows:
vgl-bake.exefails to start, often silently (exit code 0xc0000135). Activate the conda environment so GDAL's DLLs are found. -
checksays it was built without libcurl. Reconfigure withVGL_DEFAULTS_HTTP=ON(the baker presets do).
See also Troubleshooting, Pack format overview, Hosting packs and Attribution and licensing.
Getting vgl
Using the library
Rendering
Data
Reference
Specifications