Skip to content

Commit

Permalink
Merge 5a75b21 into 05a2afb
Browse files Browse the repository at this point in the history
  • Loading branch information
@ibgreen
ibgreen committed Dec 30, 2019
2 parents 05a2afb + 5a75b21 commit 886b94d
Show file tree
Hide file tree
Showing 14 changed files with 242 additions and 172 deletions.
64 changes: 41 additions & 23 deletions docs/specifications/category-3d-tiles.md
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,24 @@
# Category: 3D Tiles
# 3D Tiles Loaders

> The 3D tiles category is still under development.
The 3D Tiles category defines a generalized, slightly abstracted representation of hierarchical geospatial data structures.
The 3D Tiles category defines a generalized representation of hierarchical geospatial data structures.

It is being defined to be able to represent the [OGC 3D Tiles](https://www.opengeospatial.org/standards/3DTiles) standard but is intended to be generalized and extended to handle the similar formats, potentially such as [OGC i3s](https://www.opengeospatial.org/standards/i3s) standard and the `potree` format as well.
## 3D Tiles Category Loaders

| Loader | Notes |
| ----------------------------------------------------------------- | ----- |
| [`Tileset3DLoader`](modules/gltf/docs/api-reference/gltf-loader) | |
| [`I3SLoader`](modules/i3s/docs/api-reference/i3s-loader) | |
| [`PotreeLoader`](modules/potree/docs/api-reference/potree-loader) | |

## Overview

The 3D Tiles category is can represent the

- [OGC 3D Tiles](https://www.opengeospatial.org/standards/3DTiles) standard
- [OGC i3s](https://www.opengeospatial.org/standards/i3s) standard
- `potree` format as well.

## Concepts

Expand All @@ -13,24 +27,7 @@ It is being defined to be able to represent the [OGC 3D Tiles](https://www.openg
- **Tile Cache** - Since the number of tiles in big tilesets often exceed what can be loaded into available memory, it is important to have a system that releases no-longer visible tiles from memory.
- **Tileset Traversal** - Dynamically loading and rendering 3D tiles based on current viewing position, possibly triggering loads of new tiles and unloading of older, no-longer visible tiles.

## Tileset Traversal Support

To start loading tiles once a top-level tileset file is loaded, the application can instantiate the `Tileset3D` class and start calling `tileset3D.traverse(camera_parameters)`.

Since 3D tiled data sets tend to be very big, the key idea is to only load the tiles actually needed to show a view from the current camera position.

The `Tileset3D` allows callbacks (`onTileLoad`, `onTileUnload`) to be registered that notify the app when the set of tiles available for rendering has changed. This is important because tile loads complete asynchronously, after the `tileset3D.traverse(...)` call has returned.

## Coordinate Systems

To help applications process the `position` data in the tiles, 3D Tiles category loaders are expected to provide matrices are provided to enable tiles to be used in both fixed frame or cartographic (long/lat-relative, east-north-up / ENU) coordinate systems:

- _cartesian_ WGS84 fixed frame coordinates
- _cartographic_ tile geometry positions to ENU meter offsets from `cartographicOrigin`.

Position units in both cases are in meters.

For cartographic coordinates, tiles come with a prechosen cartographic origin and precalculated model matrix. This cartographic origin is "arbitrary" (chosen based on the tiles bounding volume center). A different origin can be chosen and a transform can be calculated, e.g. using the math.gl `Ellipsoid` class.
## Data Format

## Tileset Fields

Expand All @@ -48,8 +45,6 @@ For cartographic coordinates, tiles come with a prechosen cartographic origin an
| ---------------- | -------- | -------- |
| `boundingVolume` | `Object` | |

## Tile Fields

### Common Fields

| Field | Type | Contents |
Expand Down Expand Up @@ -101,3 +96,26 @@ Following vis.gl conventions, `attributes` are represented by "glTF-style" acces
| `size` | No | `Number` | Number of components, `1`-`4`. |
| `byteOffset` | Yes | `Number` | Starting offset into the bufferView. |
| `count` | Yes | `Number` | The number of elements/vertices in the attribute data. |

## Helper Classes

Tileset Traversal Support

To start loading tiles once a top-level tileset file is loaded, the application can instantiate the `Tileset3D` class and start calling `tileset3D.traverse(camera_parameters)`.

Since 3D tiled data sets tend to be very big, the key idea is to only load the tiles actually needed to show a view from the current camera position.

The `Tileset3D` allows callbacks (`onTileLoad`, `onTileUnload`) to be registered that notify the app when the set of tiles available for rendering has changed. This is important because tile loads complete asynchronously, after the `tileset3D.traverse(...)` call has returned.

## Additional Information

### Coordinate Systems

To help applications process the `position` data in the tiles, 3D Tiles category loaders are expected to provide matrices are provided to enable tiles to be used in both fixed frame or cartographic (long/lat-relative, east-north-up / ENU) coordinate systems:

- _cartesian_ WGS84 fixed frame coordinates
- _cartographic_ tile geometry positions to ENU meter offsets from `cartographicOrigin`.

Position units in both cases are in meters.

For cartographic coordinates, tiles come with a prechosen cartographic origin and precalculated model matrix. This cartographic origin is "arbitrary" (chosen based on the tiles bounding volume center). A different origin can be chosen and a transform can be calculated, e.g. using the math.gl `Ellipsoid` class.
17 changes: 11 additions & 6 deletions docs/specifications/category-gis.md
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
# Category: GIS
# Geospatial Loaders

> The GIS category is highly experimental and may be removed in a future release
> The Geospatial category is experimental
Several geospatial formats return data in the form of lists of lng/lat encoded geometric objects.

## GeoJSON Conversion
## Geospatial Category Loaders

GIS category data can be converted to GeoJSON (sometimes with a loss of information). Most geospatial applications can consume geojson.
| Loader | Notes |
| -------------------------------------------------------- | ----- |
| [`KMLLoader`](modules/kml/docs/api-reference/kml-loader) | |
| [`WKTLoader`](modules/wkt/docs/api-reference/wkt-loader) | |

## Data Format

## Data Structure

Expand All @@ -22,6 +27,6 @@ A JavaScript object with a number of top-level array-valued fields:
| `folders` | |
| `links` | |

## Loaders
### GeoJSON Conversion

- [KMLLoader](/docs/api-reference/kml/kml-loader)
Geospatial category data can be converted to GeoJSON (sometimes with a loss of information). Most geospatial applications can consume geojson.
96 changes: 56 additions & 40 deletions docs/specifications/category-image.md
View file
Original file line number Diff line number Diff line change
@@ -1,38 +1,23 @@
# Category: Image
# Image Loaders

The image category documents a common data format, options, conventions and utilities for loader and writers for images that follow loaders.gl conventions.
The image loader category documents a common data format, options, conventions and utilities for loader and writers for images that follow loaders.gl conventions.

Participating Loaders
## Image Category Loaders

| Loader | Notes |
| --------------------- | ----- |
| ImageLoader | |
| CompressedImageLoader | |
| BasisLoader | |

## Features and Capabilities

Apart from providing a set of image loaders that integrate with loaders.gl, there are a number of capabilities that are provided for

- Worka under both node and browser.
- Transparently uses ImageBitmap on supporting browsers
- Loads images on workers
- Handles SVG images
- Image type detection (without loading images)

## Installation
| Loader | Notes |
| --------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| [`ImageLoader`](modules/images/docs/api-reference/image-loader) | Loads compressed images (PNG, JPG, etc) |
| [`CompressedTextureLoader`](modules/basis/docs/api-reference/compressed-texture-loader) | Parses compressed textures to image data mipmap array |
| [`BasisLoader`](modules/basis/docs/api-reference/basis-loader) | Transpiles into supported compressed texture format |

Core image category support is provided by the `@loaders.gl/images` module:

```bash
npm install @loaders.gl/core @loaders.gl/images
```

## Usage

Individual loaders for specific image formats can be imported for `@loaders.gl/images`:

```js
import '@loaders.gl/polyfills'; // Only required if loading images under Node.js
import {ImageLoader} from '@loaders.gl/images';
import {registerLoaders, load} from '@loaders.gl/core';
registerLoaders(ImageLoader);
Expand All @@ -42,44 +27,75 @@ const image = await load('image.jpeg');
However since each image loader is quite small (in terms of code size and bundle size impact), most applications will just install all image loaders in one go:

```js
import '@loaders.gl/polyfills'; // Only required if loading images under Node.js
import {ImageLoaders} from '@loaders.gl/images';
import {registerLoader, load} from '@loaders.gl/core';
registerLoaders(ImageLoader);
const image = await load('image.jpeg');
```

## Data Formats
## Image Types

Images can be loaded as image data or as opaque image objects (`Image` or `ImageBitmap`), and the image _type_ option can be used to control the type of image object produced by the `ImageLoader`.

A loaded image can always be returned as an _image data_ object (an object containing a `Uint8Array` with the pixel data, and metadata like `width` and `height`, and in Node.js images are always loaded as image data objects).

In the browser, the `ImageLoader` uses the browser's native image loading functionality, and if direct access to the image data is not required, it is more efficient to load data into an opaque image object. The `ImageLoader` prefers `ImageBitmap` when supported, falling back to `Image` (aka `HTMLImageElement`) on older browsers.

Note that _type_ is independent of the _format_ of the image (see below).

| Image Type | Class | Availability | Workers | Description |
| ------------- | -------------------------------------------------------------------- | -------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `data` | Object with `{width: Number, height: Number, data: Uint8Array, ...}` | Node.js and browsers | No | Compatible with headless gl. |
| `imagebitmap` | `ImageBitmap` | Chrome/Firefox | Yes: **transferrable** | A newer JavaScript class designed for efficient loading of images, optimized for use in worker threads and with WebGL |
| `image` | `Image` (aka `HTMLImageElement`) | All browsers | No | The traditional HTML/JavaScript class used for image loading into DOM trees. WebGL compatible. |

## Image Data

Image data objects are images loaded as data, represented by an object that contains a typed array with the pixel data, size, and possibly additional metadata `{width: Number, height: Number, data: Uint8Array, ...}`

The loaded image representation can vary somewhat based on your environment. For performance, image loaders use native image loading functionality in browsers. Browsers can load into two types of image classes (`ImageBitmap` and `HTMLImageElement`) and on Node.js images are represented using `ndarray`. The following table summarizes the situation:
To get an image data object from a loaded `Image` or `ImageBitmap`, call `getImageData(image)`. To load an image data object directly, set the `image.type: 'data'` option when loading the image.

| Format Name | Format | Availability | Workers | Description |
| ------------- | -------------------------------- | -------------------------------------- | ---------------------- | -------------------------------------------------------------------------------- |
| `imagebitmap` | `ImageBitmap` | Chrome/Firefox | Yes: **transferrable** | A newer class designed for efficient loading of images for use with WebGL |
| `html` | `Image` (aka `HTMLImageElement`) | All browsers | No | The original HTML class used for image loading into DOM trees. WebGL compatible. |
| `ndarray` | ndarray | Node only, via `@loaders.gl/polyfills` | No | Used to load images under node. Compatible with headless gl. |
## Image Format

The _format_ of the image describes how the memory is laid out. It is mainly important when working with `data` _type_ images. The default format / memory layout for image data is `RGBA` and `UNSIGNED_BYTE` i.e. four components per pixel, each a byte.

Some loaders may add additional fields to the image data structure to describe the data format. Currently the image category does not provide any documentation for how to describe alternate formats/memory layouts, however a preliminary recommendation is to follow OpenGL/WebGL conventions.

## Compressed Images

Compressed images are always returned as image data objects. They will have an additional field, `compressed: true`, indicating that the typed array in the `data` field contains compressed pixels and is not directly indexable.

Applications that use e.g. the `CompressedTextureLoader` and/or the `BasisLoader` together with the `ImageLoader` can check this flag before attempting to access the image data.

## Options

The image category support some generic options (specified using `options.image.<option-name>`), that are applicable to all (or most) image loaders.

| Option | Default | Type | Availability | Description |
| --------------------------- | ------------- | ------- | -------------- | ----------------------------------------------- |
| `options.image.type` | `'auto'` | string | See table | One of `auto`, `imagebitmap`, `html`, `ndarray` |
| `options.image.decodeHTML` | `true` | boolean | No: Edge, IE11 | Wait for HTMLImages to be fully decoded. |
| `options.image.crossOrigin` | `'anonymous'` | boolean | All Browsers | Sets `crossOrigin` field for HTMLImage loads |
| Option | Default | Type | Availability | Description |
| -------------------------- | -------- | ------- | -------------- | --------------------------------------------- |
| `options.image.type` | `'auto'` | string | See table | One of `auto`, `data`, `imagebitmap`, `image` |
| `options.image.decodeHTML` | `true` | boolean | No: Edge, IE11 | Wait for HTMLImages to be fully decoded. |

## Notes

### About worker loading

- Worker loading is only supported for the `imagebitmap` format (on Chrome and Firefox).
- `ImageBitmap` is **transferrable** and can be moved back to main thread without copying.

Since image worker loading is only available on some browsers, the `ImageLoader` dynamically determines if worker loading is available. Use `options.worker: false` to disable worker loading of images.
Worker loading is only supported for the `data` and `imagebitmap` formats. Since image worker loading is only available on some browsers (Chrome and Firefox), the `ImageLoader` dynamically determines if worker loading is available. Use `options.worker: false` to disable worker loading of images.

## Image API

The image category also provides a few utilities:

- Detecting ("sniffing") mime type and size of image files before parsing them
- Getting image data (arrays of pixels) from an image without knowing which type was loaded (TBA)

## Remarks

- image data objects have the same fields as the browser's built-in `ImageData` class, but are not actual instances of `ImageData`. It is however easy to create an `ImageData` instance from an image data object:

```js
const imageData = new ImageData(imagedata.data, imagedata.width, imagedata.height);
```

Note that this assumes its image _format_ is the default 8-bit RGBA.
Loading

0 comments on commit 886b94d

Please sign in to comment.