What models, datasets, and tasks should AgML benchmark next?

[masonearles]

AgML's leaderboard is intended to make model evaluation across agricultural datasets more transparent, comparable, and reproducible—not simply to declare a universal winner.

Current results already show why this matters: performance can vary substantially by dataset, task, label ontology, metric, prompt, and evaluation protocol. Aggregate rankings are useful, but per-dataset behavior and methodological details are essential.

What should we benchmark next?

We welcome proposals involving:

• New open or closed foundation models
• Vision-language models and agricultural-domain models
• Classification, detection, segmentation, counting, localization, and image-text tasks
• Zero-shot, few-shot, in-context, fine-tuned, and parameter-efficient adaptation
• Geographic, temporal, sensor, cultivar, and environment domain shifts
• Open-ended versus constrained-label prompting
• Context injection using crop, phenology, geography, weather, or management information
• Robustness, calibration, uncertainty, efficiency, and failure analysis
• Comparisons with task-specific supervised baselines

Propose or share an evaluation

Please include as much of the following as possible:

• Model name, exact version, and access conditions
• Dataset and configuration
• Task and agricultural use case
• Train, validation, and test protocol
• Prompt, label ontology, adaptation, or training setup
• Metrics and averaging method
• Hardware and inference settings when relevant
• Reproducibility artifacts
• Known limitations and the research question being tested

Explore the AgML leaderboard and see the result contribution format.

Preliminary results are welcome when clearly labeled. Before a result is incorporated into the public leaderboard, we will ask for enough methodological detail to make it interpretable and reproducible.