OpenSpecLib v0.0.6

OpenSpecLib v0.0.6

Amalgamated spectral library containing 32940 spectra from the following sources:

• ecosis (v1.0): 27032 spectra
• ecostress (v1.0): 3451 spectra
• usgs_splib07 (v7a): 2457 spectra

Artifacts

• openspeclib-catalog-0.0.6.json — Catalog (metadata index, no spectral arrays)
• One Parquet file per source (row-grouped for partial HTTP Range reads):
• ecosis.parquet — 27032 spectra from ecosis
• ecostress.parquet — 3451 spectra from ecostress
• usgs_splib07.parquet — 2457 spectra from usgs_splib07
• checksums.txt — SHA-256 checksums for all of the above

Usage

Read the catalog for metadata queries:

import json, urllib.request
url = "https://github.com/<owner>/<repo>/releases/download/v0.0.6/openspeclib-catalog-0.0.6.json"
catalog = json.load(urllib.request.urlopen(url))
print(f"Total spectra: {catalog['statistics']['total_spectra']}")

Per-source files are Apache Parquet (zstd-compressed) with one row per spectrum
and dot-separated columns (e.g. material.name, spectral_data.wavelengths).
Each source is a single file — category filtering is a column predicate, not a
file selection. Read directly from the release URL:

import pyarrow.parquet as pq, pyarrow.fs as pafs
# pyarrow supports HTTP(S) sources; reads only the row groups matching your query
table = pq.read_table(
    "https://github.com/<owner>/<repo>/releases/download/v0.0.6/usgs_splib07.parquet",
    filters=[("material.category", "=", "mineral")],
)

Or query with DuckDB directly over HTTP (httpfs extension reads Parquet footers
and row groups via Range requests, no full download):

INSTALL httpfs; LOAD httpfs;
SELECT id, "material.name"
FROM 'https://github.com/<owner>/<repo>/releases/download/v0.0.6/usgs_splib07.parquet'
WHERE "material.category" = 'mineral'
  AND "spectral_data.wavelength_min" < 0.4;

See docs/data-structure.md and
schemas/library.parquet-schema.md for the
full schema specification and column reference.