ARVA (Agricultural Robustness across Varied Areas) evaluates how frozen Earth-observation representations behave when a deployment population changes geographically. It compares split constructions, source-only transfer, controlled source/target label allocation, local-training references, worst-region behavior, and a no-encoder control without updating the encoder.
Dataset: ARVA on Hugging Face
The reusable package is designed for researchers evaluating their own numeric features or cached representations. The paper-specific benchmark loaders, model wrappers, experiment grid, and machine orchestration remain in the source repository and are not imported by the public package.
ARVA answers three questions:
- Are target-region labels more useful than the same number of source-region labels?
- How much does the reported result depend on split construction?
- Do learned representations improve on task-matched raw inputs, and do benchmark averages conceal severe regional failures?
ARVA does not establish universal model rankings, isolate architecture from pretraining or preprocessing, equate label costs across tasks, or turn a target-trained score into an oracle ceiling. Results remain conditional on the declared target population, inputs, probe, metric, and available labels.
- Distribution:
arva-eo - Import:
arva - Version source:
arva._version.__version__ - Supported Python: 3.11
The shorter arva distribution name is already used by an unrelated project,
so this project uses arva-eo while retaining the paper-aligned arva import.
From a source checkout:
python3.11 -m venv .venv
.venv/bin/python -m pip install .For development:
.venv/bin/python -m pip install ".[dev]"
.venv/bin/python -m pytest tests
.venv/bin/ruff check src tests examplesThe base package installs only the numerical dependencies needed by the public CPU evaluation path. Dataset libraries, coordinate projection, visualization, and model frameworks are optional.
Run the installed synthetic extension workflow from any directory:
python -m arva.examples.syntheticIt defines a benchmark and representation adapter, creates a deterministic geographic split, fits a logistic probe, evaluates two metrics across source, fixed-budget, additive, and complete-pool routes, writes checksummed artifacts, and reads the result back through the public API.
The complete executable source is
src/arva/examples/synthetic.py. The same
workflow is an integration test, so adding the example required no edits to
paper-specific registries or orchestration.
The stable public surface is exported from arva:
- Contracts:
Benchmark,RepresentationAdapter,IdentityRepresentation - Configuration:
ProbeConfig,LabelAllocationConfig,EvaluationConfig,Regime - Split strategies:
RandomSplit,PublishedSplit,GeographicHoldout,SpatialClustering - Split artifacts:
SplitArtifact,generate_splits,load_splits - Evaluation:
run_evaluation - Results:
ResultArtifact,read_results - Extension:
Registry - Errors:
ContractError,ArtifactError,EvaluationError,MissingOptionalDependencyError,RegistrationError
Registries are explicit instances, not mutable process-wide state. Duplicate registration fails unless the caller deliberately requests replacement.
| Paper term | Program identifier | Target-label access | Role |
|---|---|---|---|
| Stratified random split | random_id |
No | Within-domain reference |
| Published benchmark split | official |
No | Dataset-released setting |
| Geographic holdout split | geographic_ood |
Yes | Named-region deployment transfer |
| Spatial clustering split | spatial_cluster_ood |
No | Coordinate-derived split sensitivity |
Only the geographic holdout supplies a disjoint target-label pool. Spatial clustering is zero-shot split sensitivity and never gains target labels. Positive purge distances require coordinates and are recorded in the split artifact.
- Architecture and package boundaries
- Benchmark, representation, probe, and metric contracts
- Split semantics and label access
- Artifact and result schemas
- Reproducibility and seed behavior
- Adding benchmarks, models, probes, metrics, or regimes
- Optional dependencies
- Troubleshooting
- Paper experiment relationship
Maintainers build both release artifacts with:
python -m buildRelease verification installs the wheel in a clean environment, runs
pip check, imports from outside the repository, executes the complete
synthetic workflow, installs the geo extra, and inspects wheel and source
distribution contents before any publication.
ARVA software and documentation are available under the MIT License. Original project-authored data and released data artifacts are available under CC BY 4.0. Third-party datasets, model weights, and assets retain their original terms.