Skip to content

Core Concepts

Jump to bottom
Jeremiah van Oosten edited this page Oct 19, 2026 · 3 revisions

Core concepts

This page explains how vgl is put together, for developers who embed it in their own application or engine. It covers who owns what (the host, the core, the backends and the defaults module), the data vgl reads (packs, layers, tiles and levels of detail), the WGS84 ellipsoid and the coordinate frames, the depth conventions, the CPU and GPU budgets, what happens in each frame, the threading model, results and logging, and how versioned structures keep the ABI stable. The specifications in the repository are the source of truth and are linked throughout: the design document, the C API conventions, the render interface and the pack format.

The host owns the application

vgl is a library that cooperates with an application, not a framework that takes it over. The host (your application) owns:

  • the window and the graphics device or context,
  • the main loop and its timing,
  • the camera, which it passes to vgl in double precision each frame,
  • the render target, including clearing it, its depth buffer and presenting it.

Each frame, the host sets the view, calls vgl_context_update, and then asks a backend to draw the globe into its current render pass. vgl never clears, swaps buffers or presents, and it draws together with whatever else the host draws, depth-tested against it (see depth conventions).

The library also follows a set of rules that make it predictable to embed:

Rule What it means for you
The C API is the ABI One binary serves C and C++. The C++ wrapper is header-only and adds no second ABI.
No exceptions across the API Every function returns a vgl_result. The library builds with exceptions and RTTI off (VGL_DISABLE_EXCEPTIONS).
No global state All state lives in a vgl_context. Several contexts are independent.
No threads, no network connections The core creates neither. You supply task scheduling and I/O, or use the defaults module that does.
All memory through your allocator Every allocation on behalf of a context goes through the allocator you give it (or the C heap).
Bounded memory Decoded data and GPU resources stay within budgets you set.
Portable core The core uses only the C and C++ standard libraries: no operating system calls, no std::filesystem or std::thread.

The pieces: core, backends and the defaults module

flowchart TB
    Host["Host application"]
    Core["vgl core (vgl::vgl)<br/>context, packs, selection, cache,<br/>decoding, meshes, queries"]
    Defaults["Defaults module (vgl::defaults)<br/>thread pool, file reader,<br/>HTTP reader, disk cache"]
    RI["Render interface<br/>(vgl/render.h)"]
    GL["OpenGL / ES backend (vgl::gl)"]
    DX["Direct3D 11 backend (vgl::d3d11)"]
    OG["Ogre backend (vgl::ogre)"]
    Own["Host's own renderer"]
    Packs[("Packs on disk or a web server")]

    Host --> Core
    Host -. optional .-> Defaults
    Defaults -- "task and I/O interfaces" --> Core
    Defaults --> Packs
    Core --> RI
    RI --> GL
    RI --> DX
    RI --> OG
    RI --> Own
Loading
Piece Library and header Role
Core vgl::vgl, <vgl/vgl.h> (and <vgl/vgl.hpp>) Turns packs into GPU-ready data: reads and decodes tiles, selects them for the view, builds meshes and textures, answers height and ray queries. It calls no graphics API.
Render interface Part of the core, <vgl/render.h> The contract between the core and whatever draws: per-frame lists of resources to create and destroy, and a draw list. See Render Interface.
Backends vgl::gl (<vgl/gl.h>), vgl::d3d11 (<vgl/d3d11.h>), vgl::ogre (<vgl/ogre.hpp>) Consumers of the render interface that draw the globe with a graphics API or engine. Each is a library of its own, built on the public C API only, with its headers in backends/<backend>/include/vgl.
Defaults module vgl::defaults, <vgl/defaults.h> Optional ready-made implementations of the task and I/O interfaces: a thread pool, a file reader, an HTTP reader (with VGL_DEFAULTS_HTTP) and a disk cache. Unlike the core, it creates threads and opens files and connections.

The backends are the OpenGL / OpenGL ES backend (OpenGL 3.3 core or OpenGL ES 3.0, and WebGL 2 in a browser), the Direct3D 11 backend (feature level 11_0) and the Ogre backend (classic Ogre 14, on its OpenGL 3+ and Direct3D 11 render systems on Windows and Linux; Vulkan, macOS and the mobile platforms follow after version 1, as milestone M3b). A host with its own renderer uses the render interface directly, exactly as the built-in backends do.

Packs, layers and tiles

Packs

All data reaches the runtime as packs, made offline by the Baker. A pack is a directory with a small manifest (pack.vglm) and one or more range-readable shard files (base-<digest>.vgls, shards/<y>/<x>-<digest>.vgls). The runtime only ever reads byte ranges of these files, so a pack can sit on a local disk or on any web server that serves HTTP range requests. The format is specified in pack-format.md; the Pack Format Overview summarizes it.

There are three kinds of pack, recorded in the manifest's layer_type. You do not say which kind you add; the pack's manifest does:

Kind Holds What the context does with it
Elevation Heights above the ellipsoid, as 129 x 129 sample grids per tile Shapes the terrain; answers height and ray queries
Imagery Images, 256-pixel tiles with a 1-pixel border by default (JPEG, or PNG where there is alpha) One imagery layer, blended over the layers of lower priority
Geoid The geoid height (EGM2008) on the same grid as elevation Converts heights to mean sea level; places the optional sea surface. At most one per context.

The project hosts no data. You bake the elevation and imagery you need and host the packs yourself (Hosting Packs); the repository's samples/data holds a small sample set for the samples and tests.

Layering by priority

Several packs of the same kind stack by priority (vgl_pack_create_info::priority), so that a regional high-resolution pack refines a global base without rebaking:

  • Where packs overlap, the one with the higher priority wins where it has data. Among equal priorities, the pack created later wins.
  • Where the winning elevation pack has nodata, the next one fills in, so regional packs blend into the base without holes.
  • Each imagery pack is a separate layer. Layers draw bottom (lowest priority) to top, each blended by its image's alpha times an opacity the host sets (vgl_pack_opacity_set). At opacity 0 a layer is hidden: neither loaded nor drawn nor credited.
  • Priority and opacity can change at run time. See Context API.

Tiles and levels of detail

Terrain and imagery share one geographic (latitude and longitude) quadtree:

  • Level 0 has two tiles of 180 by 180 degrees: (0, 0, 0) is the western hemisphere and (0, 1, 0) the eastern. Each level splits every tile into four.
  • Column x counts east from the antimeridian, row y south from the north pole. Level L has 2L+1 columns and 2L rows.
  • With 129-sample elevation tiles, level 8 has about 611 m between samples at the equator, level 12 about 38 m (the deepest level the test bake chose for Copernicus GLO-30).

Pyramids are sparse: fine levels exist only where a pack has data, and each tile records which children exist and their height range, so the runtime can cull tiles before loading them. Geodesy and Tiling covers the scheme in detail.

Selection picks the tiles to draw by screen-space error: a tile's geometric error (its sample spacing) projected to pixels. A tile is refined when that exceeds vgl_view::max_screen_space_error, 2 pixels by default. Tiles outside the frustum or below the horizon are culled. Detail arrives level by level: a tile is refined only once its own data has loaded, an unloaded child is drawn from its nearest loaded ancestor, and a newly selected tile gets a small placeholder mesh at once, so no part of the globe is ever empty while it loads. Regions no pack covers draw as the bare ellipsoid. The details are in the design document, section 7.3.

The ellipsoid and coordinate frames

Frame Definition Where it appears
Geodetic Latitude and longitude in radians, height in metres above the ellipsoid (vgl_geodetic) Height queries, ray hits, places
ECEF Earth-centred, Earth-fixed, right-handed, metres: origin at the ellipsoid's centre, +Z through the north pole, +X through latitude 0, longitude 0 The view matrix, ray queries, projection of positions, draw items' centre
ENU Local East-North-Up at a point (vgl_ellipsoid_enu_frame_get) Placing a camera or a vehicle on the ground
Camera space Whatever the host's view matrix maps ECEF to, in any handedness The model-view matrices of the draw list
  • The reference ellipsoid is WGS84 unless the context is created with another (vgl_context_create_info::ellipsoid); packs must use the same one.
  • All heights are ellipsoidal unless the name says otherwise. Heights above mean sea level need the geoid pack (VGL_HEIGHT_REFERENCE_MEAN_SEA_LEVEL); the geoid and the ellipsoid differ by up to about 100 m.
  • Angles are radians everywhere. VGL_DEGREES_TO_RADIANS, VGL_RADIANS_TO_DEGREES, vgl_degrees_to_radians and vgl_radians_to_degrees convert.
  • Matrices (vgl_dmat4) are column-major and multiply column vectors: element (row r, column c) is m[c * 4 + r], translation in m[12] to m[14], as in GLM and OpenGL. A host with row-major, row-vector matrices, common with Direct3D math libraries, passes its matrices unchanged: the two differences cancel out.

Double precision and tile-relative vertices

The Earth's radius is about 6.4 million metres, and a float has about seven significant digits, so single-precision ECEF positions would jitter by metres near the ground. vgl avoids this (requirement NFR-09, no visible jitter at any distance):

  • Every position and distance that crosses the API is double, including the camera.
  • Mesh vertices are floats relative to each tile's centre, so their values stay small.
  • Each tile's model-view matrix (view * translate(centre)) is composed on the CPU in double precision and rounded to float only at the end. The GPU never sees large coordinates.

Hosts drawing their own geometry next to the globe should do the same: compose model-view matrices in double precision relative to the camera, and convert to float last. Geodesy and Tiling has the conversion functions.

Depth conventions

A globe seen from orbit down to the ground needs an enormous depth range. The host owns the depth buffer and the projection, so it chooses the convention when it creates a backend (vgl_depth_convention, in <vgl/render.h>). Every backend supports all three:

Convention Value Projection Depth buffer Use it
Logarithmic (default) VGL_DEPTH_CONVENTION_LOGARITHMIC The host's, conventional; near and far planes do not matter Any, cleared to 1, compare LESS; the shader writes depth from the camera distance Everywhere, including OpenGL ES, WebGL 2 and macOS
Reversed-Z VGL_DEPTH_CONVENTION_REVERSED_Z Reversed: near to 1, far (ideally infinite) to 0, in a depth range 0 to 1 32-bit float, cleared to 0, compare GREATER Direct3D 11; OpenGL 4.5, GL_ARB_clip_control or GL_EXT_clip_control
Conventional VGL_DEPTH_CONVENTION_CONVENTIONAL The host's as it is Any, cleared to 1, compare LESS Small scenes only: distant terrain z-fights

The logarithmic convention writes log2(1 + w) / log2(1 + far) as window depth, with w the distance along the view axis and far the far distance passed when drawing (VGL_LOGARITHMIC_DEPTH_FAR_DEFAULT, 1.0e8 m, when the host passes none). Host geometry that must depth-test against the globe, such as an aircraft on the runway, writes the same depth. In GLSL, with p the camera-space position:

// log_depth_coefficient = 2.0 / log2(far + 1.0)
// Vertex shader:
v_log_depth = 1.0 + max(-p.z, 0.0);
gl_Position.z = (log2(v_log_depth) * log_depth_coefficient - 1.0) * gl_Position.w;
// Fragment shader:
gl_FragDepth = log2(v_log_depth) * log_depth_coefficient * 0.5;

Reversed-Z gives the best precision where it is available. A backend asked for a convention the device cannot do fails to create with VGL_RESULT_NOT_SUPPORTED. Section 8 of the render interface has the details.

Budgets

Memory is bounded by budgets set when the context is created (requirement NFR-04). A zero value selects the default:

Budget Field Default What it bounds
CPU vgl_context_create_info::cpu_budget 256 MiB Decoded tiles, encoded imagery tiles, pack metadata and meshes not yet uploaded
GPU vgl_context_create_info::gpu_budget 512 MiB Meshes and textures the consumer was asked to create
Update time vgl_update_info::time_budget 2 ms Decoding inside vgl_context_update when there is no task interface
Upload vgl_gl_draw_info::upload_budget (and the other backends' equivalents, vgl_frame_begin_info::upload_budget) 16 MiB (VGL_UPLOAD_BUDGET_DEFAULT) Bytes of new resources per frame, so a big view change spreads its uploads over several frames

How they work:

  • Over the CPU budget, decoded tiles and meshes are evicted least recently used, but never those the current view needs.
  • Once a mesh or texture is uploaded, its CPU copy is freed and it counts against the GPU budget instead. Over the GPU budget, the core destroys resources not drawn in the current frame, least recently drawn first. Resources that selected tiles still wait for are exempt.
  • The consumer never decides what to evict: it only applies the destroy list it is given.
  • Lost GPU resources (a lost device or OpenGL context) are rebuilt from the decoded tiles after vgl_context_resources_reset, which the backends' device_reset functions call.

vgl_context_statistics_get reports use against both budgets (Context API). On mobile devices, smaller budgets than the defaults are usually right.

The frame lifecycle

sequenceDiagram
    participant H as Host
    participant C as vgl context
    participant T as Tasks and I/O (host or defaults)
    participant B as Backend
    H->>C: vgl_context_view_set(view)
    H->>C: vgl_context_update()
    C->>C: take completed reads and decodes
    C->>C: select tiles, request missing ones
    C->>T: read byte ranges, submit decodes and mesh builds
    T-->>C: vgl_io_request_complete, task done (any thread)
    H->>B: draw (render thread)
    B->>C: vgl_context_frame_begin
    C-->>B: destroys, creates (within the upload budget), draw list
    B->>B: apply resources, draw tiles
    B->>C: vgl_frame_end
Loading

What vgl_context_update does

In order, each update:

  1. Takes completions. Reads the host completed and tasks that finished since the last update are processed in full: manifests parsed, shard directories indexed, tiles queued for decoding.
  2. Decodes. Without a task interface, it decodes and builds meshes itself, within time_budget (at least one job runs if any is waiting). With one, this work runs on the host's threads and the update only collects the results.
  3. Selects. For the view set with vgl_context_view_set, it selects tiles by screen-space error, with frustum and horizon culling, and builds placeholders for new tiles.
  4. Requests. It requests the tiles selection needs that are not loaded, coarser levels first, then larger errors first, with at most 64 reads outstanding.
  5. Evicts decoded data beyond the CPU budget.

How a tile streams in

stateDiagram-v2
    [*] --> Unloaded
    Unloaded --> Requested: selected
    Requested --> Loading: I/O issued
    Loading --> Decoded: data decoded, mesh built
    Decoded --> Uploaded: created by the consumer, CPU copy freed
    Uploaded --> Decoded: device lost (vgl_context_resources_reset)
    Uploaded --> Evicted: over GPU budget, least recently drawn
    Decoded --> Evicted: over CPU budget
    Evicted --> Unloaded
    Loading --> Unloaded: cancelled
Loading

A tile's data is read in a few steps: the pack's manifest once, then the directory of the shard that holds the tile (usually one range request), then the tile's own byte range. Imagery tiles stay encoded in the cache, a few kilobytes each, and are decoded into RGBA8 textures with a full mip chain on a task thread when a selected tile wants them.

What a frame does

The backend's draw function (for example vgl_gl_backend_draw) brackets a frame with vgl_context_frame_begin and vgl_frame_end. The frame lists what to destroy, then the index lists, meshes and textures to create (within the upload budget), then the tiles to draw, nearest first, each with its float model-view matrix and imagery layers. A host with its own renderer walks the same lists itself (Render Interface).

The update and the frame must not overlap. A host that updates on one thread and renders on another serializes the two.

Threading model

The library creates no threads. Every function documents one of these categories (C API conventions, section 11):

Category Rule Examples
Free-threaded Any thread, concurrently with anything vgl_context_create, vgl_context_allocator_get, vgl_context_log, vgl_view_project, vgl_view_geodetic_project, the geodesy functions, vgl_io_request_complete, vgl_io_request_create, vgl_result_name_get, vgl_version_get
Context-synchronized Calls on one context, or on objects created from it, must not overlap; any thread, one at a time vgl_context_update, vgl_context_view_set, vgl_pack_create, the queries, vgl_context_statistics_get, vgl_context_destroy
Render thread Context-synchronized, and on the thread that owns the graphics device vgl_context_frame_begin and the frame functions, vgl_context_resources_reset, every backend function
Externally synchronized For objects of no context, such as the defaults module's: calls on one object must not overlap vgl_thread_pool_destroy, vgl_file_reader_destroy

Callbacks the library calls:

Callback Where it runs
vgl_io_interface.read, .cancel, .destroy The calling thread, inside a context function (usually vgl_context_update or a query)
vgl_task_interface.submit, .destroy The calling thread, inside a context function
The task function you are given A task thread of yours, possibly concurrently with others; tasks never block on each other or on the host
vgl_log_fn and the allocator Any thread: they must be thread-safe if a task interface is installed

Tasks. With a task interface, decoding and mesh building run on the host's threads, and the update just collects results. Without one (tasks null), the update does that work itself, within its time budget, which suits a single-threaded host (NFR-10). If submit returns anything but VGL_RESULT_SUCCESS, the library does the work itself.

I/O. The library asks for byte ranges by URI; the host completes each request exactly once with vgl_io_request_complete, from any thread, immediately or later. Completions only queue the result; it takes effect at the next update. A context may be destroyed with reads outstanding or tasks queued: late completions find the context gone and free themselves, and the host never has to drain its queues first. Host Interfaces covers writing your own; the Defaults Module provides ready ones.

Results and logging

Every function returns vgl_result: zero is success, positive values are statuses that still succeeded, negative values are errors.

Code Value Meaning
VGL_RESULT_SUCCESS 0 Succeeded
VGL_RESULT_INCOMPLETE 1 Succeeded, but an output array was too small
VGL_RESULT_NOT_READY 2 Succeeded, but finer data is still loading; the output holds the best answer so far
VGL_RESULT_INVALID_ARGUMENT -1 An argument is null, out of range or otherwise invalid
VGL_RESULT_OUT_OF_MEMORY -2 An allocation failed, or a budget does not allow it
VGL_RESULT_NOT_SUPPORTED -3 Valid, but not supported by this build, backend or device; or a structure uses fields of a newer version
VGL_RESULT_INVALID_OPERATION -4 Not allowed in the object's current state
VGL_RESULT_NOT_FOUND -5 An item, file or tile does not exist
VGL_RESULT_IO_ERROR -6 Reading failed
VGL_RESULT_CORRUPT_DATA -7 Data was read but failed validation or decoding
VGL_RESULT_VERSION_MISMATCH -8 A pack, interface or structure version this binary does not support
VGL_RESULT_CANCELLED -9 An asynchronous operation was cancelled
VGL_RESULT_BACKEND_ERROR -10 A graphics API or engine call failed
VGL_RESULT_INTERNAL_ERROR -11 A bug in vgl; please report it
  • Test with VGL_SUCCEEDED(r) and VGL_FAILED(r) rather than comparing with VGL_RESULT_SUCCESS, so that VGL_RESULT_NOT_READY counts as success. Treat unknown negative codes as failures and unknown positive codes as success.
  • vgl_result_name_get gives the name, such as "VGL_RESULT_OUT_OF_MEMORY".
  • A failed function leaves its objects unchanged, and the library stays usable after any failure, including running out of memory.

There is no "last error" state. The details go to the log callback (vgl_context_create_info::log, with log_user_data): every failure of a function that has a context logs one error message that names the function, says what was wrong and, where known, how to fix it. Without a callback the library is silent; it never prints by itself. The minimum level is vgl_context_create_info::log_level; VGL_LOG_LEVEL_DEFAULT selects warnings. VGL_LOG_LEVEL_INFO is useful while integrating. Backends log through the same callback (vgl_context_log).

Versioned structures and ABI stability

Structures that may grow start with uint32_t struct_size, which the caller sets to sizeof the structure. Every other field has a default selected by its zero value. So the idiom is: zero the structure, set struct_size, then set only what you need.

vgl_context_create_info info = {.struct_size = sizeof(info)}; /* Every other field zero. */
info.cpu_budget = 128ull * 1024 * 1024;
auto info = vgl::init<vgl_context_create_info>(); // Zeroed, with struct_size set.
info.cpu_budget = 128ull * 1024 * 1024;
  • Always zero the whole structure. On 32-bit targets, including WebAssembly, each pointer in a structure with 64-bit fields is followed by a padding field (VGL_POINTER_PADDING), so code that sets fields one by one on an uninitialized structure, or a designated initializer that names every field in order, can miss one. Zeroing first is always correct.
  • Input structures. A smaller struct_size (a host built against older headers) gives the missing fields their defaults. A larger one is accepted if the unknown bytes are zero, and fails with VGL_RESULT_NOT_SUPPORTED otherwise.
  • Output structures (vgl_context_statistics, vgl_ray_hit, vgl_view from vgl_context_view_get): set struct_size, at least 8. The library writes the fields that fit and sets struct_size to the bytes it wrote.
  • Interfaces (vgl_io_interface, vgl_task_interface, vgl_allocator_interface) carry a version field instead, set to the VGL_<NAME>_INTERFACE_VERSION macro. The library copies the table; its user_data must stay valid until the context is destroyed.

From version 1.0, a host built against headers X.Y runs with any binary X.Z where Z >= Y. Breaking changes come only with a new major version, and the changelog lists them. vgl_version_get reports the binary's version, and VGL_VERSION_MAJOR, VGL_VERSION_MINOR, VGL_VERSION_PATCH and VGL_VERSION_STRING the headers'.

Object lifetimes

  • Objects are opaque handles with create and destroy functions. Destroying null does nothing.
  • Objects belong to their parent: destroying a context destroys its remaining packs, with a warning for each, since that usually means a leak. Destroy children first.
  • A backend must be destroyed before its context, on the render thread, with its graphics context current. Destroying the context destroys none of the consumer's GPU objects.
  • Host interfaces must outlive the context that uses them: destroy the context, then the defaults module's readers and cache, then the thread pool.
  • Strings the library returns (attribution lines, result names) are owned by the library, and each function says how long they stay valid.

See also

vgl

Getting vgl

Using the library

Rendering

Data

Reference

Specifications

Clone this wiki locally