[Paper] · [Data] · [Repository] · [Quick start] · [Citation]
FinEvolveBench evaluates whether language-model agents can turn delayed, noisy market outcomes into useful experience during chronological replay. It aligns public financial news with daily observations for 31 Chinese A-share industry indices and evaluates predictive market-sentiment factors against future market-adjusted returns.
Important
Data release status (August 2026). The Google Drive folder currently contains a one-month preview (January 2026). The complete benchmark dataset described in the paper is scheduled for public release in September 2026. The preview is intended to make the data format and released pipeline inspectable in advance; it is not the full 300-day experimental dataset.
| Component | Description | Status |
|---|---|---|
| News corpus | Time-stamped public financial news with topic, industry, and importance annotations | January 2026 preview on Google Drive |
| Technical data | Daily OHLC, trading fields, and technical indicators for the benchmark universe | January 2026 preview on Google Drive |
| Prediction pipeline | Original 40-trading-day, industry-level prediction workflow, with experience operations disabled | Available in this repository |
| Evaluation tools | Time-series and cross-sectional information-coefficient evaluation | Available under tools/tEval |
| Full benchmark data | The complete temporal coverage described in the paper | Planned for September 2026 |
FinEvolveBench/
├── pipeline.py # 40-trading-day prediction workflow
├── prediction_prompt.md # prompt used by the predictor
├── sector_map.csv # industry-name to index-code mapping
├── tools/
│ ├── tEval/ # benchmark evaluation utilities
│ └── philo/ # bundled agent runtime
├── News Courps/ # downloaded data; one Parquet file per day
└── Technical Data/ # downloaded data; one CSV per trading day
The two data directories are distributed through Google Drive rather than tracked in Git. Download them from the data folder and place them at the repository root using the names shown above. News Courps retains the directory name used by the released pipeline.
-
Download the January 2026 preview from Google Drive and place
News Courps/andTechnical Data/besidepipeline.py. -
Install the runtime dependencies:
python -m pip install pandas pyarrow openai
-
Configure an OpenAI-compatible endpoint and run a trading date included in the preview:
export FIN_EVOLVE_API_KEY="..." export FIN_EVOLVE_BASE_URL="https://your-openai-compatible-endpoint/v1" export FIN_EVOLVE_MODEL="your-model-name" python pipeline.py 20260105 --output-dir forecasts
The output is written to forecasts/20260105.csv with columns l1sector and forecast. API credentials are read only from environment variables and are not included in the release.
Each daily Parquet file contains the original article and its preprocessing annotations.
| Field | Description |
|---|---|
id |
News-record identifier |
news_datetime, news_date |
First-publication time and date |
news_src |
Source label |
news_title, news_content |
Original title and full article body |
preprocess_data |
Complete preprocessing record serialized as UTF-8 JSON |
importance_score, importance_reason |
Importance annotation and rationale |
primary_topic, secondary_topics, topic_confidence |
Topic annotations |
industry_tags, industry_type, industry_confidence |
Industry annotations |
The preprocess_data JSON has the following top-level structure:
{
"importance": {"score": "...", "reason": "..."},
"industry": {"tags": ["..."], "type": "...", "confidence": "...", "rationale": "..."},
"topic": {"primary": "...", "secondary": ["..."], "confidence": "..."},
"duplicate_key": {"event_date": "...", "event_type": "...", "key_entities": ["..."]}
}Each CSV is a cross-sectional snapshot for one trading day, keyed by ts_code and trade_date. Fields include OHLC prices (open, high, low, close, pre_close), return and trading fields (change, pct_change, vol, amount), and precomputed indicators including moving averages, MACD, RSI, KDJ, Bollinger bands, ATR, CCI, OBV, and WR. Weekends and market holidays intentionally have no technical-data file.
The released pipeline follows five stages:
- Get data — load the date's news annotations and technical-data CSV.
- Recall experience — disabled in this release.
- Compose data — experience-based composition is disabled.
- Predict — group news by industry, create the original industry-news context, request a structured 40-trading-day prediction, validate it, and write the forecast CSV.
- Update experience — disabled in this release.
This compact workflow exposes the paper's experience-free baseline. sector_map.csv maps annotation labels (l1name) to the index identifiers (l1sector) used by the prediction output. For evaluator input formats and multi-horizon evaluation, see the tEval documentation.
The benchmark design, chronological replay protocol, data statistics, experimental horizons, and evaluation methodology are described in:
FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards
Zihao Deng, Yining Zhu, Leiming Wang, Jingfei Lu, Junbo Wang, Chuncheng Ran, Yu Yang, Dixuan Yang, and Jikun Shen.
arXiv:2606.06960v2
If you use FinEvolveBench, please cite:
@article{deng2026finevolvebench,
title = {FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewards},
author = {Deng, Zihao and Zhu, Yining and Wang, Leiming and Lu, Jingfei and Wang, Junbo and Ran, Chuncheng and Yang, Yu and Yang, Dixuan and Shen, Jikun},
journal = {arXiv preprint arXiv:2606.06960},
year = {2026},
url = {https://arxiv.org/abs/2606.06960}
}FinEvolveBench is provided for research and evaluation. Its forecasts are model outputs, not financial advice, and must not be used as the sole basis for investment decisions.