"I visualise a time when we will be to robots what dogs are to humans, and I’m rooting for the machines." — Claude Shannon
"We can only see a short distance ahead, but we can see plenty there that needs to be done." — Alan Turing
Our Mission: to efficiently automate trading. We continuously develop and share codes for finance.
Our Vision: AI community has accumulated an open-source code ocean over the past decade. Applying these intellectual and engineering properties to finance will initiate a paradigm shift from the conventional trading routine to an automated machine learning approach, even RLOps in finance.
FinRL (website) is the first open-source project to explore the great potential of deep reinforcement learning in finance. We help practitioners pipeline a trading strategy using deep reinforcement learning (DRL).
The FinRL ecosystem is a unified framework, including various markets, state-of-the-art algorithms, financial tasks (portfolio allocation, cryptocurrency trading, high-frequency trading), live trading, etc.
| Roadmap | Level | Users | Example | Desription |
|---|---|---|---|---|
| 0.0 (Preparation) | preparation | practitioners of financial big data | FinRL-Meta | a universe of market environments |
| 1.0 (Proof-of-Concept) | entry-level | beginners | this repo | demonstration, education |
| 2.0 (Professional) | intermediate-level | full-stack developers, professionals | ElegantRL | financially optimized DRL algorithms |
| 3.0 (Production) | advance-level | investment banks, hedge funds | Podracer | cloud-native solution |
- Overview
- File Structure
- Supported Data Sources
- Installation
- Status Update
- Contributions
- Tutorials
- Publications
- News
- Citing FinRL
- Welcome Contributions
- Sponsorship
- LICENSE
FinRL has three layers: applications, agents, and market environments.
For a trading task (on the top), an agent (in the middle) interacts with an environment (at the bottom), making sequential decisions.
Run FinRL_StockTrading_NeurIPS_2018.ipynb step by step for a quick start.
A video about FinRL library at the AI4Finance Youtube Channel.
Correspondingly, the main folder finrl has three subfolders applications, agents, finrl_meta.
We employ a train-test-trade pipeline by three files: train.py, test.py, and trade.py.
FinRL
├── finrl (main folder)
│ ├── applications
│ ├── cryptocurrency_trading
│ ├── high_frequency_trading
│ ├── portfolio_allocation
│ └── stock_trading
│ ├── agents
│ ├── elegantrl
│ ├── rllib
│ └── stablebaseline3
│ ├── finrl_meta
│ ├── data_processors
│ ├── env_cryptocurrency_trading
│ ├── env_portfolio_allocation
│ ├── env_stock_trading
│ ├── preprocessor
│ ├── data_processor.py
│ └── finrl_meta_config.py
│ ├── config.py
│ ├── config_tickers.py
│ ├── main.py
│ ├── plot.py
│ ├── train.py
│ ├── test.py
│ └── trade.py
│
├── tutorial (tutorial notebooks and educational files)
├── unit_testing (make sure verified codes working on env & data)
│ ├── test_env
│ └── test_env_cashpenalty.py
│ └── test_marketdata
│ └── test_yahoodownload.py
├── setup.py
├── requirements.txt
└── README.md
| Data Source | Type | Max Frequency | Raw Data | Preprocessed Data |
|---|---|---|---|---|
| Alpaca | US Stocks, ETFs | 1 min | OHLCV | Prices, indicators |
| Baostock | CN Securities | 5 min | OHLCV | Prices, indicators |
| Binance | Cryptocurrency | 1 s | OHLCV | Prices, indicators |
| CCXT | Cryptocurrency | 1 min | OHLCV | Prices, indicators |
| IEXCloud | NMS US securities | 1 day | OHLCV | Prices, indicators |
| JoinQuant | CN Securities | 1 min | OHLCV | Prices, indicators |
| QuantConnect | US Securities | 1 s | OHLCV | Prices, indicators |
| RiceQuant | CN Securities | 1 ms | OHLCV | Prices, indicators |
| Tushare | CN Securities | 1 min | OHLCV | Prices, indicators |
| WRDS | US Securities | 1 ms | Intraday Trades | Prices, indicators |
| YahooFinance | US Securities | 1 min | OHLCV | Prices, indicators |
| AkShare | CN Securities | 1 day | OHLCV | Prices, indicators |
| findatapy | CN Securities | 1 day | OHLCV | Prices, indicators |
| pandas_datareader | US Securities | 1 day | OHLCV | Prices, indicators |
| pandas-finance | US Securities | 1 day | OHLCV & Prices, indicators | |
| ystockquote | US Securities | 1 day | OHLCV | Prices, indicators |
| Marketstack | 50+ countries | 1 day | OHLCV | Prices, indicators |
| finnhub | US Stocks, currencies, crypto | 1 day | OHLCV | Prices, indicators |
| Financial Modeling prep | US stocks, currencies, crypto | 1 min | OHLCV | Prices, indicators |
| EOD Historical Data | US stocks, and ETFs | 1 day | OHLCV | Prices, indicators |
| Alpha Vantage | Stock, ETF, forex, crypto, technical indicators | 1 min | OHLCV & Prices, indicators | |
| Tiingo | Stocks, crypto | 1 day | OHLCV | Prices, indicators |
| Quandl | 250+ sources | 1 day | OHLCV | Prices, indicators |
| Polygon | US Securities | 1 day | OHLCV | Prices, indicators |
| fixer | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| Exchangerates | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| Fixer | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| currencylayer | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| currencyapi | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| Open Exchange Rates | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| XE | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
| Xignite | Exchange rate | 1 day | Exchange rate | Exchange rate, indicators |
OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price
Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.
- install description for all system
- FinRL for Quantitative Finance: Install and Setup Tutorial for Beginners
Version History [click to expand]
- 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
- 2020-12-14 Upgraded to Pytorch with stable-baselines3; Remove tensorflow 1.0 at this moment, under development to support tensorflow 2.0
- 2020-11-27 0.1: Beta version with tensorflow 1.5
- FinRL is the first open-source framework to demonstrate the great potential of applying DRL algorithms in quantitative finance. We build an ecosystem around the FinRL framework, which seeds the rapidly growing AI4Finance community.
- The application layer provides interfaces for users to customize FinRL to their own trading tasks. Automated backtesting tool and performance metrics are provided to help quantitative traders iterate trading strategies at a high turnover rate. Profitable trading strategies are reproducible and hands-on tutorials are provided in a beginner-friendly fashion. Adjusting the trained models to the rapidly changing markets is also possible.
- The agent layer provides state-of-the-art DRL algorithms that are adapted to finance with fine-tuned hyperparameters. Users can add new DRL algorithms.
- The environment layer includes not only a collection of historical data APIs, but also live trading APIs. They are reconfigured into standard OpenAI gym-style environments. Moreover, it incorporates market frictions and allows users to customize the trading time granularity.
- [Towardsdatascience] Deep Reinforcement Learning for Automated Stock Trading
- [Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Multiple Stock Trading
- [Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Portfolio Allocation
- [Towardsdatascience] FinRL for Quantitative Finance: Tutorial for Single Stock Trading
- [Towardsdatascience] ElegantRL-Podracer: A Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning
- [Towardsdatascience] ElegantRL: A Lightweight and Stable Deep Reinforcement Learning Library
- [Towardsdatascience] ElegantRL: Mastering PPO Algorithms
- [MLearning.ai] Hyperparameter Optimization using Ray tune for FinRL models
- [MLearning.ai] An Empirical Approach to Explain Deep Reinforcement Learning in Portfolio Management Task
- [MLearning.ai] FinRL for Quantitative Finance: plug-and-play DRL algorithms
- [MLearning.ai] ElegantRL Demo: Stock Trading Using DDPG (Part I)
- [MLearning.ai] ElegantRL Demo: Stock Trading Using DDPG (Part II)
- [DataDrivenInvestor] FinRL-Meta: A Universe of Near Real-Market Environments for Data-Driven Financial Reinforcement Learning
- [DataDrivenInvestor] A Data Scientist’s Approach for Algorithmic Trading using Deep Reinforcement Learning: An End-to-end Tutorial for Paper Trading
- [Analytics Vidhya] Weights and Biases-ify FinRL with Stable Baselines3 models
- [Analytics Vidhya] Hyperparameter tuning using optuna for FinRL
- [Analytics Vidhya] A hitchhikers guide to FinRL: A Deep Reinforcement Learning Framework for Quantitative Finance
- [Analyticsindiamag.com] How To Automate Stock Market Using FinRL (Deep Reinforcement Learning Library)?
| Title | Conference | Link | Citations | Year |
|---|---|---|---|---|
| FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance | NeurIPS 2021 Data-Centric AI Workshop | paper: https://arxiv.org/abs/2112.06753 ; code: https://github.com/AI4Finance-Foundation/FinRL-Meta |
2 | 2021 |
| Explainable deep reinforcement learning for portfolio management: An empirical approach | ICAIF 2021 : ACM International Conference on AI in Finance | paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3958005; code: https://github.com/AI4Finance-Foundation/FinRL |
1 | 2021 |
| FinRL-Podracer: High performance and scalable deep reinforcement learning for quantitative finance | ICAIF 2021 : ACM International Conference on AI in Finance | paper: https://arxiv.org/abs/2111.05188; code: https://github.com/AI4Finance-Foundation/FinRL_Podracer |
2 | 2021 |
| FinRL: Deep reinforcement learning framework to automate trading in quantitative finance | ICAIF 2021 : ACM International Conference on AI in Finance | paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3955949; code: https://github.com/AI4Finance-Foundation/FinRL |
4 | 2021 |
| FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance | NeurIPS 2020 Deep RL Workshop | paper: https://arxiv.org/abs/2011.09607; code: https://github.com/AI4Finance-Foundation/FinRL |
20 | 2020 |
| Deep reinforcement learning for automated stock trading: An ensemble strategy | ICAIF 2020 : ACM International Conference on AI in Finance | paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3690996; code: https://github.com/AI4Finance-Foundation/Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020 |
37 | 2020 |
| Multi-agent reinforcement learning for liquidation strategy analysis | ICML 2019 Workshop on AI in Finance: Applications and Infrastructure for Multi-Agent Learning | paper: https://arxiv.org/abs/1906.11046; code: https://github.com/AI4Finance-Foundation/Liquidation-Analysis-using-Multi-Agent-Reinforcement-Learning-ICML-2019 |
19 | 2019 |
| Practical deep reinforcement learning approach for stock trading | NeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services | paper: https://arxiv.org/abs/1811.07522; code: https://github.com/AI4Finance-Foundation/DQN-DDPG_Stock_Trading |
81 | 2018 |
- [央广网] 2021 IDEA大会于福田圆满落幕：群英荟萃论道AI 多项目发布亮点纷呈
- [央广网] 2021 IDEA大会开启AI思想盛宴 沈向洋理事长发布六大前沿产品
- [IDEA新闻] 2021 IDEA大会发布产品FinRL-Meta——基于数据驱动的强化学习金融风险模拟系统
- [知乎] FinRL-Meta基于数据驱动的强化学习金融元宇宙
- [量化投资与机器学习] 基于深度强化学习的股票交易策略框架（代码+文档)
- [运筹OR帷幄] 领读计划NO.10 | 基于深度增强学习的量化交易机器人：从AlphaGo到FinRL的演变过程
- [深度强化实验室] 【重磅推荐】哥大开源“FinRL”: 一个用于量化金融自动交易的深度强化学习库
- [商业新知] 金融科技讲座回顾|AI4Finance: 从AlphaGo到FinRL
- [Kaggle] Jane Street Market Prediction
- [矩池云Matpool] 在矩池云上如何运行FinRL股票交易策略框架
- [财智无界] 金融学会常务理事陈学彬: 深度强化学习在金融资产管理中的应用
- [Neurohive] FinRL: глубокое обучение с подкреплением для трейдинга
- [ICHI.PRO] 양적 금융을위한 FinRL: 단일 주식 거래를위한 튜토리얼
@article{finrl2020,
author = {Liu, Xiao-Yang and Yang, Hongyang and Chen, Qian and Zhang, Runjia and Yang, Liuqing and Xiao, Bowen and Wang, Christina Dan},
title = {{FinRL}: A deep reinforcement learning library for automated stock trading in quantitative finance},
journal = {Deep RL Workshop, NeurIPS 2020},
year = {2020}
}
@article{liu2021finrl,
author = {Liu, Xiao-Yang and Yang, Hongyang and Gao, Jiechao and Wang, Christina Dan},
title = {{FinRL}: Deep reinforcement learning framework to automate trading in quantitative finance},
journal = {ACM International Conference on AI in Finance (ICAIF)},
year = {2021}
}
We published FinTech papers, check Google Scholar, resulting in this project. Closely related papers are given in the list.
Welcome to the AI4Finance Foundation community!
Join to discuss FinRL: AI4Finance mailing list, AI4Finance Slack channel:
Please check Contributing Guidances.
Thanks!
Welcome gift money to support AI4Finance, a non-profit academic community. Use the links in the right, or scan the following vemo QR code:
Detailed sponsorship records can be found at Issue #425
MIT License
Disclaimer: Nothing herein is financial advice, and NOT a recommendation to trade real money. Please use common sense and always first consult a professional before trading or investing.