Regen is an end-to-end application that showcases how to train and deploy reinforcement learning trading agents on environments that conform to the Farama's Env interface. The tech stack is both modern and open-source, making it easy to work with and build upon. With Regen, you'll gain valuable insight into how to develop powerful and flexible trading agents.
The objective of this project is to offer a set of architecture components that are required to create a complete trading application. This includes ELT pipelines for acquiring data for environments, which comprises both historical data for training agents and backtesting, as well as live data for deployments. The environments serve as interfaces for agents to trade assets, while the agents themselves are designed to define the policies for executing trades.
Version | |
---|---|
Terraform | 1.3.7 |
Docker | 20.10.22 |
All the ELT infrastructure is defined as Terraform code inside the directory
infra
The ELT pipeline in this project is managed through Airflow. The extraction layer includes python tasks that extract data from sources in batches and load it into a Postgres database that serves as the data warehouse. Once the data is in the warehouse, transformations are carried out using dbt modules to create the observations table that is used by the environment. Currently, there is a single pipeline that generates the observations table for the single crypto-asset environment.
Environments are interfaces to expose data that agents need to make decisions, additionally, they execute actions the agents make at a specified step, episode and execution. they are subclasses of Farama's Env. Currently, there is a single environment available, which is the single crypto-asset environment.
An environment that provides data about a specific cryptocurrency asset at a given minute, an observation is made up of 11 features: open value, highest value, lowest value, close value, and 7 moving averages data points (with 7, 25, 100, 300, 1440, 14400, 144000 window sizes) in the last minute.
Agents choose actions in an environment. At the moment, only the deep reinforcement learning agent is available.
An agent that relies on deep reinforcement learning algorithms to build policies. stable-baselines3 implementations are used.
- go to the infra folder
cd infra
- run
terraform apply
to deploy infrastructure
- Trigger the
klines_backfill
,ma_backfill
anddbt_run
DAG's in sequence that should be available in the Airflow server inhttp://localhost:8080/
to put historical data in the observations table that is hosted by theregen_db
container. - Run
docker exec agent python main.py -t
to train the agent.
Run docker exec agent python main.py -l
to run in live mode. This mode
requires load_from_execution_id config variable to be defined.
bnb_base_urls: List of URL's to connect to the Binance api.
bnb_client_key: Binance api secret.
db_file_location: Location of the database file when using SQLite engine.
base_asset: Asset to accumulate in the single crypto-asset environment.
quote_asset: Quote asset in the single crypto-asset environment.
window_size: Number of consecutive records per observation.
ticks_per_episode: Max number of steps per episode.
time_steps: Total number of time steps to train the agent.
output_dir: Path to the directory to store output files.
db_type: Type of database engine to use. Only sql
and postgres
are
supported at the moment.
load_from_execution_id: Id of the previous execution to load a pretrained model from.
env_logging_lvl: Level of the logs to flush into the log files.
obs_table: Name of the observations table.
db_schema: Name of the schema where the observations table is.
This software is for educational purposes only. It is not intended to be used with real money. Do not risk money which you are afraid to lose. USE THE SOFTWARE AT YOUR OWN RISK. THE AUTHOR ASSUME NO RESPONSIBILITY FOR YOUR TRADING RESULTS.