Web Builds

Web builds

vgl builds for WebAssembly with Emscripten and draws in browsers on WebGL 2, the browser's OpenGL ES 3.0. This page covers the emscripten preset and the tested emsdk version, why the build uses threads and what that asks of the page (cross-origin isolation), running the tests on Node.js, the OpenGL sample's page in samples/gl/web and how to serve it, streaming packs over HTTP from the browser (CORS), and how to use vgl in an Emscripten application of your own.

The web build is part of vgl 1.0, and is checked in CI on every pull request.

What runs in the browser

The same code as on the desktop: the core, the defaults module and the OpenGL backend.

Threads. The build uses Emscripten's pthreads, so the defaults module's thread pool decodes tiles on Web Workers just as it does on native threads.
HTTP. With VGL_DEFAULTS_HTTP (on in the preset), the defaults module's HTTP reader is built on the browser's fetch (emscripten_fetch) instead of libcurl, with the same API (vgl_http_reader_create and friends). No library is needed.
Rendering. The OpenGL backend takes its OpenGL ES path on WebGL 2. Reversed-Z needs clip control, which WebGL 2 lacks, so the logarithmic depth convention (VGL_DEPTH_CONVENTION_LOGARITHMIC, the default) applies.
Not in the browser. The Direct3D 11 and Ogre backends, the baker, and the pure-C sample (its loop does not fit a page).

Supported browsers are those with WebGL 2 and shared memory between workers: Chrome and Edge 92, Firefox 79 and Safari 15.2, or later (design document, section 4).

Building

Install emsdk, activate version 6.0.11 (the version CI tests), and build the emscripten preset. It needs Ninja on the path, and the EMSDK environment variable that activating emsdk sets.

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install 6.0.11
./emsdk activate 6.0.11
source ./emsdk_env.sh            # .\emsdk_env.ps1 in PowerShell
cd <vgl>
cmake --workflow --preset emscripten-release

The workflow configures, builds, and runs the tests on Node.js. Step by step:

cmake --preset emscripten
cmake --build --preset emscripten-release
ctest --preset emscripten-release

What the preset sets:

Setting	Value	Why
Toolchain	`$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake`	Emscripten's CMake toolchain
Generator	Ninja Multi-Config
`BUILD_SHARED_LIBS`	`OFF`	The web build is static
`VGL_ZSTD_PROVIDER`	`BUNDLED`	Zstandard is compiled from source into vgl
`VGL_DEFAULTS_HTTP`	`ON`	The HTTP reader, on the browser's fetch
`CMAKE_C_FLAGS`, `CMAKE_CXX_FLAGS`	`-pthread`	Every object is compiled with threads, so every object that links with vgl must be too

The outputs are in build/emscripten/bin/Release. The OpenGL sample becomes a page: vgl-gl.html with vgl-gl.js, vgl-gl.wasm, and vgl-gl.data (the package holding the clouds' image). The headless sample and the tests become programs for Node.js.

Tests on Node.js

The Emscripten toolchain sets Node.js (from emsdk) as CMake's cross-compiling emulator, so ctest runs every test with it. The tests and the headless sample are linked with:

Flag	Purpose
`-sPROXY_TO_PTHREAD`	`main` runs on a worker, so it may block while the threads it starts come up
`-sPTHREAD_POOL_SIZE=8`	Workers created up front
`-sEXIT_RUNTIME`	The program exits with `main`'s result
`-sINITIAL_MEMORY=512MB`	Room for the tests' packs and caches
`-sNODERAWFS`	The real file system, so the file reader reads `samples/data` directly

The http test (the HTTP reader against a local server) is not built for Emscripten. The headless test runs the headless sample's check on the sample data, as on every platform.

The browser check

With VGL_TEST_BROWSER on, the build adds a browser test, tests/browser/page_check.mjs. It serves the page with Caddy and the sample's Caddyfile, opens it in headless Chrome on its software renderer (no GPU needed), and drives it over the DevTools protocol. It fails if the page is not cross-origin isolated, does not draw tiles from orbit, or does not fly to the Sognefjord on the key 7 and load the tiles there, or if anything throws, a WebGL call fails or a read fails. It leaves a screenshot, browser-check.png, in build/emscripten/samples/gl.

cmake --preset emscripten -DVGL_TEST_BROWSER=ON
cmake --build --preset emscripten-release
ctest --preset emscripten-release -R browser

It needs Chrome and Caddy 2.11, found on the path or named with -DVGL_CHROME=<path> and -DVGL_CADDY=<path>. CI runs it on Linux.

Cross-origin isolation (COOP and COEP)

Threads need SharedArrayBuffer, which browsers give only to pages that are cross-origin isolated: served with both of these headers.

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Without them, the page cannot start its workers. The sample's page checks window.crossOriginIsolated before it starts and shows a message naming the headers if the page is not isolated, or if the browser has no WebGL 2. Opening the HTML file directly from disk never works, since file:// pages are not isolated.

Isolation has a consequence for data: under require-corp, every resource the page loads from another origin must opt in, with CORS or with Cross-Origin-Resource-Policy: cross-origin. See Streaming packs from another server.

The web sample

The OpenGL sample (samples/gl) builds into the page, with the same panel, navigation, atmosphere, stars and clouds as the native sample. The web-specific parts are:

File	Role
`samples/gl/web/shell.html`	The page: a full-window canvas, a status line, the isolation and WebGL 2 checks, and the address-to-options translation
`samples/gl/web/Caddyfile`	Serves the page cross-origin isolated, with the sample packs at `data/packs` next to it
`samples/gl/web/README.md`	Building, running, options and checking

The sample is linked with these options (from samples/gl/CMakeLists.txt), which are a good starting point for your own page:

Flag	Purpose
`-sMIN_WEBGL_VERSION=2`, `-sMAX_WEBGL_VERSION=2`	WebGL 2 only
`-sGL_ENABLE_GET_PROC_ADDRESS`	Lets the backend load its functions through a `GetProcAddress` (here `glfwGetProcAddress`)
`-sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency`	The thread pool's workers, started with the page
`-sINITIAL_MEMORY=768MB`	The context's CPU budget, the meshes and the clouds. Growing memory with threads is slow, so the page starts with what it needs
`-sSTACK_SIZE=1MB`
`-sENVIRONMENT=web,worker`	A page, not Node.js
`--shell-file=.../web/shell.html`	The page template
`--preload-file=.../samples/data/clouds.jpg@/data/clouds.jpg`	The clouds' image, in `vgl-gl.data`
`--use-port=contrib.glfw3`	Emscripten's GLFW port, on a canvas with WebGL 2 (compile and link)
`-sFETCH=1`	The HTTP reader's fetch. This one comes from `vgl::defaults` itself

The sample is compiled with VGL_SAMPLE_PAGE, which makes it fetch every pack, and VGL_SAMPLE_DATA_DIR="data", so the sample set is fetched from data/packs relative to the page.

Serving it

From the repository root, with Caddy 2.11:

caddy run --config samples/gl/web/Caddyfile

Then open http://localhost:8090/vgl-gl.html. The Caddyfile reads three environment variables:

Variable	Default	Meaning
`VGL_SITE`	`:8090`	The address to serve
`VGL_WEB`	`build/emscripten/bin/Release`	The directory of the built page
`VGL_PACKS`	`samples/data/packs`	The packs served at `/data/packs` next to the page

It sends the two isolation headers and Cache-Control: no-cache for everything, serves only GET and HEAD, serves only pack.vglm and *.vgls files under /data/packs (as application/octet-stream, as the reference server does), and only the four files of the page at the root.

Any other server works if it sends the same two headers for the page, its script, its WebAssembly and its package. Simple static servers such as python -m http.server do not send them, so the page reports that it is not cross-origin isolated.

Options in the address

The page passes its address's parameters to the sample as command-line options: a parameter with a value becomes --name value, one without becomes --name.

Address	Native equivalent
`vgl-gl.html?pack=https://packs.example.org/gebco&pack=https://packs.example.org/glo30`	`--pack https://packs.example.org/gebco --pack https://packs.example.org/glo30`
`vgl-gl.html?depth=conventional`	`--depth conventional`

Without pack, the page loads the sample set from data/packs. Relative pack URIs are fetched from the page's own address. --check is not available in the page, since its loops would stop it.

Streaming packs over HTTP

In the browser, every pack is read with fetch, using HTTP range requests on the pack's shards exactly as the native HTTP reader does (Hosting packs). Created without a fallback interface, the reader fetches every URI, and relative URIs resolve against the page's address:

// In a page: every pack is on a web server, and relative URIs are fetched from the page's
// own address (samples/common/pack_reader.hpp).
vgl::expected<vgl::http_reader> http = vgl::http_reader::create();

With a fallback, as on native platforms, URIs that do not start with http:// or https:// go to the fallback instead (for example a file reader on Emscripten's virtual file system).

Things that differ from the libcurl reader:

Fetches start and finish on the main thread, between the page's frames. Reads requested on other threads are passed to the main thread, so the main thread must keep returning to the browser for reads to progress.
Reads that fail without an answer are tried again a few times (for example when Chrome's cache refuses a range of a file it is still storing).
max_connection_count and user_agent in vgl_http_reader_create_info are ignored: the browser chooses connections and the User-Agent header.
The browser allocates the data delivered, so the reader's allocator serves only the reader itself.

The disk cache (vgl_disk_cache) is meant for real file systems. In a page, Emscripten's default file system lives in memory and is gone with the page, so a disk cache there would not outlive it; the browser's own HTTP cache does that job, which is why the reference server sends caching headers that let browsers keep shards.

Streaming packs from another server

A page served from one origin can stream packs from another (a CDN, an object store, a reference server), provided that server allows it. Because the page is cross-origin isolated, the pack server must send either CORS headers or Cross-Origin-Resource-Policy: cross-origin, and for range requests from another origin it must answer CORS preflights that ask for the Range header. The reference server's Caddyfile (tools/server/Caddyfile) does all of this:

Access-Control-Allow-Origin: *            (or VGL_CORS_ORIGIN)
Cross-Origin-Resource-Policy: cross-origin
Access-Control-Expose-Headers: Accept-Ranges, Content-Length, Content-Range, ETag
Access-Control-Allow-Methods: GET, HEAD, OPTIONS          (preflight)
Access-Control-Allow-Headers: Range, If-None-Match, If-Range   (preflight)

Check any host before pointing a page at it with vgl-bake check <url>, which reports missing CORS and caching headers as warnings (Baker, and the hosting guide). The project hosts no data: you bake and host your own packs (Hosting packs).

Using vgl in your own Emscripten application

Build setup

The simplest way is to build vgl as part of your project with the Emscripten toolchain, with add_subdirectory or FetchContent (Installing vgl). Compile everything with -pthread, your code included, as the preset does:

cmake_minimum_required(VERSION 3.24)
project(my_globe LANGUAGES C CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Before vgl is added, so that every object is compiled with threads.
string(APPEND CMAKE_C_FLAGS " -pthread")
string(APPEND CMAKE_CXX_FLAGS " -pthread")

set(BUILD_SHARED_LIBS OFF)
set(VGL_ZSTD_PROVIDER BUNDLED)
set(VGL_DEFAULTS_HTTP ON)      # the HTTP reader on the browser's fetch
add_subdirectory(third_party/vgl)

add_executable(my_globe main.cpp)
set_target_properties(my_globe PROPERTIES SUFFIX ".html")
target_link_libraries(my_globe PRIVATE vgl::vgl vgl::defaults vgl::gl)
target_link_options(my_globe PRIVATE
    "-pthread"
    "-sMIN_WEBGL_VERSION=2"
    "-sMAX_WEBGL_VERSION=2"
    "-sGL_ENABLE_GET_PROC_ADDRESS"
    "-sPTHREAD_POOL_SIZE=navigator.hardwareConcurrency"
    "-sINITIAL_MEMORY=768MB")

Configure it with the toolchain:

emcmake cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build

Notes:

vgl::defaults adds -sFETCH=1 to your link when the HTTP reader is built, so you do not need to.
Size PTHREAD_POOL_SIZE for the threads you create: a vgl_thread_pool with thread_count 0 starts one thread less than the hardware has, and a pool of navigator.hardwareConcurrency workers covers that.
Size INITIAL_MEMORY for your context's cpu_budget (256 MiB by default) plus meshes and your own data, rather than relying on memory growth, which is slow with threads.
If you install a web build of vgl and use find_package(vgl) instead, note that Emscripten's toolchain file sets CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to ONLY, so packages are searched only under CMAKE_FIND_ROOT_PATH (Emscripten's sysroot). Add your install prefix to CMAKE_FIND_ROOT_PATH, or install into the sysroot.

The main loop

A browser runs the page on its main thread, which must return to the browser after each frame: a while loop that never returns freezes the page, and the HTTP reader's fetches only progress between frames. Turn your loop body into a frame callback with emscripten_set_main_loop_arg, as the sample does:

#include <emscripten/emscripten.h>

#include <vgl/defaults.h>
#include <vgl/gl.h>
#include <vgl/vgl.h>

struct app
{
    vgl_context* context = nullptr;
    vgl_gl_backend* backend = nullptr;
    // ... the thread pool, the reader, the packs, the window and the camera
};

// Your set-up: the defaults, the context, the packs and the backend. Returns false on failure.
bool app_create(app& a);

static void frame(void* user_data)
{
    app& a = *static_cast<app*>(user_data);
    // 1. Set the view from the camera (vgl_context_view_set), as on native platforms.
    // 2. Update: completions, decoding within the time budget, selection.
    static_cast<void>(vgl_context_update(a.context, nullptr));
    // 3. Clear the host's framebuffer, then draw the globe into it.
    static_cast<void>(vgl_gl_backend_draw(a.backend, nullptr));
    // No swap: the browser shows the canvas when the callback returns.
}

int main()
{
    static app a;
    if (!app_create(a))
    {
        return 1;
    }
    // 0 frames per second: use requestAnimationFrame. true: simulate an endless loop, so that
    // main's stack is kept (nothing after this line runs).
    emscripten_set_main_loop_arg(frame, &a, 0, true);
    return 0;
}

app_create stands for the setup described in Getting started and Core concepts: a thread pool, the HTTP reader, the context with the pool's task interface and the reader's I/O interface, the packs, and the OpenGL backend created on the current WebGL 2 context. With GLFW (Emscripten's port), create the window with GLFW_CLIENT_API set to GLFW_OPENGL_ES_API and version 3.0, as the sample does for --es, and pass a loader that returns glfwGetProcAddress(name).

Do not block the main thread for long anywhere: in particular, avoid waiting for packs to load in a loop at start-up. Check vgl_pack_state_get or vgl_context_statistics_get from frame to frame instead.

Depth and drawing

Create the backend with the default VGL_DEPTH_CONVENTION_LOGARITHMIC (zero-initialized depth_convention). VGL_DEPTH_CONVENTION_REVERSED_Z needs clip control, which vgl does not get on WebGL 2; vgl_gl_backend_create then fails with VGL_RESULT_NOT_SUPPORTED and logs why.
On WebGL 2 the backend uploads index data through GL_ELEMENT_ARRAY_BUFFER (WebGL ties a buffer to the kind of target it was first bound to) and skips the primitive restart switch, which WebGL does not have. Nothing is needed from the host for this.
The backend restores the state it changes, as on every platform (OpenGL backend).

Data in the page

Packs are best fetched from a web server, as above. For a page with its own small packs, serve them next to the page and use relative URIs.
Other files, such as a cloud coverage image, can be packaged with --preload-file and read from Emscripten's virtual file system, as the sample does with clouds.jpg.

Web Builds

Web builds

What runs in the browser

Building

Tests on Node.js

The browser check

Cross-origin isolation (COOP and COEP)

The web sample

Serving it

Options in the address

Streaming packs over HTTP

Streaming packs from another server

Using vgl in your own Emscripten application

Build setup

The main loop

Depth and drawing

Data in the page

See also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally