Repository navigation
data/ is Parquet-only: the 550 legacy .pb.gz files are removed, leaving the same 53 .parquet datasets that v0.1.1 already carried.
This is the compatibility break — v0.0.0 through v0.1.1 all resolve .pb.gz paths, and this release does not.
Breaking changes
data/ no longer contains .pb.gz files, and message_helpers.load_message has no Parquet path. Read datasets with ord_schema.datasets.load_dataset(path), which dispatches on suffix and returns a streaming DatasetView; pass as_dataset=True for a materialized Dataset.
from ord_schema.datasets import load_dataset
dataset = load_dataset("data/4d/ord_dataset-4d431564f3ef4e9c91d8da5836f4eae6.parquet")The legacy .pb.gz files are still retrievable
They are removed from data/ at this tag, not deleted from storage. A checkout of v0.1.1, or of any earlier commit, still resolves every one of them:
git checkout v0.1.1
git lfs pull --include="data/**/*.pb.gz"Both LFS stores retain the objects. The Hugging Face mirror serves all 603 objects referenced by v0.1.1 even though its HEAD no longer references the 550 .pb.gz, and GitHub's LFS store serves them as well:
git -c lfs.url="https://github.com/open-reaction-database/ord-data.git/info/lfs" \
lfs pull --include="data/**/*.pb.gz"Retired dataset IDs
499 dataset IDs no longer address a file, because two groups of shards are published as single consolidated datasets:
| Dataset | ID | Sources |
|---|---|---|
uspto-grants |
ord_dataset-1158e351757f315b93cbcbe7bc55f38e |
489 monthly uspto-grants-YYYY_MM datasets |
Training data from https://doi.org/10.1039/C8SC04228D |
ord_dataset-e7830cd6b11158b43994ccfb5ee9acb3 |
10 (N/10) shards |
retired_datasets.csv maps each retired ID to the dataset that replaced it. Every other dataset kept its ID; only the suffix changed. Per-reaction patent provenance for the USPTO data is on Reaction.provenance.patent, so the monthly bucket a reaction came from stays recoverable.
No reactions were lost in the removal: the two consolidated datasets match their shard totals exactly, and every dataset deleted here has a Parquet counterpart with identical reactions.
Reaction data lives in Git LFS, so the auto-generated source archives below contain pointer files, not reactions. Clone the repository, or run python scripts/download_from_huggingface.py; LFS reads are served by the Hugging Face mirror.
What's Changed
- Remove legacy pb.gz datasets in favor of their Parquet replacements by @skearnes in #284
- Assert validation coverage and finish the Parquet-only migration by @skearnes in #256
Full Changelog: v0.1.1...v0.2.0
🤖 Generated with Claude Code