Skip to content

Latest commit

 

History

History
212 lines (153 loc) · 6.45 KB

requirements.md

File metadata and controls

212 lines (153 loc) · 6.45 KB
Raw

COPC Validation

Table of Contents

  1. Introduction
    1. Purpose
    2. Scope
    3. Intended Audience
    4. Intended Use
    5. Definitions & Acronyms
  2. Overall Description
    1. User Needs
    2. Assumptions & Dependencies
  3. Features & Requirements
    1. Scans
      1. Quick scan
      2. Full scan
    2. Report
  4. Software
    1. Checks
    2. Report schema

Introduction

COPC Validator (name TBD) is a NPM package that performs extended validation checking against COPC type files

(See README.md)

Purpose

The purpose of this package is to allow COPC users to verify that a given file is genuinely COPC; the software will provide a detailed report on the integrity of the file, pointing out where and why there may be issues. This will protect the COPC type from malicious implementations and provide a centralized arbiter of what is and is-not proper COPC.

Scope

This software will be an NPM-published package (library?) that inputs a file, performs a quick scan of the file header or a full scan of the entire contents, and reports the status of each check performed via JSON output.

Indended Audience

The main audience for COPC Validator is internal to Hobu, for a revamped copc.io website that includes a file validator.
However, the package will also be published as an open source command-line utility to allow users to verify files locally.

Indended Use

The intended use for most users will be through the copc.io validator webpage which will utilize this package to scan provided the file(s) and report the checks completed, the status (pass/fail and/or a message) of each check, and other detailed information about the COPC file that can be determined; and then allow the user to download the report as a PDF.

The other possible use case will be users who install the command-line utility from NPM and use it to validate COPC files with their own machine. The utility should still provide the same information as the website, sans maybe the PDF (if it's not possible).

Definitions & Acronyms

  • LAS = binary file format for 3D point cloud data
  • LAZ = LAS but compressed
  • VLR = Variable Length Record
    • 54+X bytes/each
  • EVLR = Extended Variable Length Record
    • 60+X bytes/each at End of file
  • PDR = Point Data Record
  • PDRF = PDR Format
  • CRS = Coordinate Reference System
  • WKT = Well Known Text CRS (required by PDRF 6-10)

Overall Description

User Needs

Ease of use

The process of getting a file validated should be simple and easy. It should be a single command (with minimal options), along the lines of:

$ copc-validator [...filepaths] --full

Detailed reporting

The output should include detailed information about the file metadata, contents, and checks completed (with pass/fail status, and (possibly) informational messages).

Usable in the Browser

Ultimately, this package will be implemented onto copc.io, so it should be easily usable in a browser setting.

Assumptions & Dependencies

  • Operating System: Unix-based (MacOS/Linux)

This package will be written on MacOS but should be fully compatible with Linux. No guarentees about Windows compatibility.

  • Technologies:
    • NodeJS
    • TypeScript
  • Dependencies:

Features & Requirements

Scans

All scans perform checks according to LAS 1.4 and COPC 1.0 specifications.

Quick scan

Errors

Validation fails if:

  • LAS version is not 1.4
  • LAS PDRF is not 6, 7, or 8
  • COPC info VLR is missing/mispositioned
  • COPC hierarchy VLR is missing
  • COPC VLRs are unique
  • Fixed chunk size
  • Shape mismatches chunking
  • Dead hierarchy space
  • CRS = Geotiffs (vs WKT)
  • Octree center no match header bounds
  • Root hierarchy points to incorrect location
  • Unreachable hierarchy nodes
  • Multiple COPC nodes
  • Too many nodes in same page
  • Unreachable point data (not addressed by hierarchy)

Warnings

Validation passes but adds message to report if:

  • Missing SRS
  • 8-bit RGBI values
  • Extra bytes VLRs
  • Zero point nodes

Full scan

Errors

In addition to performing a Quick Scan, validation fails if:

  • GPS time outside GPS time extents
  • LAS metadata extents & ranges
  • Chunk count mismatch
  • Points outside node bounds

Warnings

Validation passes but adds message to report if:

  • Unsorted GPS times
  • Unutilized RGB in PDRF

Report

Validation report will include:

  • File name
  • Scan start/end times
  • Coordinate info
  • GPS Timer
  • Position on globe (image)
  • Summary of checks ran
  • Status/info of non-passing checks
  • VLR(s) info
    • LAS VLR
      • Public header
        • version = 1.4
        • PDRFs = 6 | 7 | 8
      • CRS VLR
      • Classification VLR
      • Extra Bytes
      • Textarea description
    • COPC VLR
      • First after header
      • info & hierarchy VLRs exist
    • LAZ compression
      • version = 1.3
    • PDAL
      • Metadata
      • Pipeline

Reports will be output via JSON (see README.md)

Software

NPM package with command-line functionality to check local files and generate JSON (and PDF?) reports, as well as in-browser capability to validate dropped/selected files and download a PDF report

Checks

Check functions maintain the following properties:

  • Syncronous or Asyncronous
  • Output: { status: 'pass' | 'fail' | 'warn', info?: unknown } (or a Promise)
  • Pure function

TypeScript:

namespace Check {
  type Status = {
    status: 'pass' | 'fail' | 'warn'
    info?: unknown
  }
  type Function<T> =
    | (c: T) => Status
    | (c: T) => Promise<Status>
  type Check = Status & {id: string}
  type Suite<T> = Record<string, checkFunction<T>>
}

pass means file definitely matches COPC specificiations
fail means file does not match any COPC specifications
warn means file may not match current COPC specifications (out-dated), or may cause issues (extra-bytes?)

All checks are located in src/checks