docs/getting_started/backtest_low_level.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Backtest (low-level API)\n",
    "\n",
    "Tutorial for [NautilusTrader](https://nautilustrader.io/docs/) a high-performance algorithmic trading platform and event driven backtester.\n",
    "\n",
    "[View source on GitHub](https://github.com/nautechsystems/nautilus_trader/blob/develop/docs/getting_started/backtest_low_level.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "This tutorial walks through how to use a `BacktestEngine` to backtest a simple EMA cross strategy\n",
    "with a TWAP execution algorithm on a simulated Binance Spot exchange using historical trade tick data.\n",
    "\n",
    "The following points will be covered:\n",
    "- How to load raw data (external to Nautilus) using data loaders and wranglers\n",
    "- How to add this data to a `BacktestEngine`\n",
    "- How to add venues, strategies and execution algorithms to a `BacktestEngine`\n",
    "- How to run backtests with a  `BacktestEngine`\n",
    "- Post-run analysis and options for repeated runs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "- [NautilusTrader](https://pypi.org/project/nautilus_trader/) latest release installed (`pip install -U nautilus_trader`)\n",
    "- [JupyterLab](https://jupyter.org/) or similar installed (`pip install -U jupyterlab`)\n",
    "- Python 3.10+ installed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "## Imports\n",
    "\n",
    "We'll start with all of our imports for the remainder of this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from decimal import Decimal\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "from nautilus_trader.backtest.engine import BacktestEngine\n",
    "from nautilus_trader.backtest.engine import BacktestEngineConfig\n",
    "from nautilus_trader.examples.algorithms.twap import TWAPExecAlgorithm\n",
    "from nautilus_trader.examples.strategies.ema_cross_twap import EMACrossTWAP\n",
    "from nautilus_trader.examples.strategies.ema_cross_twap import EMACrossTWAPConfig\n",
    "from nautilus_trader.model.currencies import ETH\n",
    "from nautilus_trader.model.currencies import USDT\n",
    "from nautilus_trader.model.data import BarType\n",
    "from nautilus_trader.model.enums import AccountType\n",
    "from nautilus_trader.model.enums import OmsType\n",
    "from nautilus_trader.model.identifiers import TraderId\n",
    "from nautilus_trader.model.identifiers import Venue\n",
    "from nautilus_trader.model.objects import Money\n",
    "from nautilus_trader.persistence.wranglers import TradeTickDataWrangler\n",
    "from nautilus_trader.test_kit.providers import TestDataProvider\n",
    "from nautilus_trader.test_kit.providers import TestInstrumentProvider"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## Loading data\n",
    "\n",
    "For this tutorial we'll use some stub test data which exists in the NautilusTrader repository\n",
    "(this data is also used by the automated test suite to test the correctness of the platform).\n",
    "\n",
    "Firstly, instantiate a data provider which we can use to read raw CSV trade tick data into memory as a `pd.DataFrame`.\n",
    "We then need to initialize the instrument which matches the data, in this case the `ETHUSDT` spot cryptocurrency pair for Binance.\n",
    "We'll use this instrument for the remainder of this backtest run.\n",
    "\n",
    "Next, we need to wrangle this data into a list of Nautilus `TradeTick` objects, which can we later add to the `BacktestEngine`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load stub test data\n",
    "provider = TestDataProvider()\n",
    "trades_df = provider.read_csv_ticks(\"binance/ethusdt-trades.csv\")\n",
    "\n",
    "# Initialize the instrument which matches the data\n",
    "ETHUSDT_BINANCE = TestInstrumentProvider.ethusdt_binance()\n",
    "\n",
    "# Process into Nautilus objects\n",
    "wrangler = TradeTickDataWrangler(instrument=ETHUSDT_BINANCE)\n",
    "ticks = wrangler.process(trades_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "See the [Loading External Data](https://nautilustrader.io/docs/latest/concepts/data#loading-data) guide for a more detailed explanation of the typical data processing components and pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "## Initialize a backtest engine\n",
    "\n",
    "Now we'll need a backtest engine, minimally you could just call `BacktestEngine()` which will instantiate\n",
    "an engine with a default configuration. \n",
    "\n",
    "Here we also show initializing a `BacktestEngineConfig` (will only a custom `trader_id` specified)\n",
    "to show the general configuration pattern.\n",
    "\n",
    "See the [Configuration](https://nautilustrader.io/docs/api_reference/config) API reference for details of all configuration options available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configure backtest engine\n",
    "config = BacktestEngineConfig(trader_id=TraderId(\"BACKTESTER-001\"))\n",
    "\n",
    "# Build the backtest engine\n",
    "engine = BacktestEngine(config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {},
   "source": [
    "## Add venues\n",
    "\n",
    "We'll need a venue to trade on, which should match the *market* data being added to the engine.\n",
    "\n",
    "In this case we'll setup a *simulated* Binance Spot exchange."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a trading venue (multiple venues possible)\n",
    "BINANCE = Venue(\"BINANCE\")\n",
    "engine.add_venue(\n",
    "    venue=BINANCE,\n",
    "    oms_type=OmsType.NETTING,\n",
    "    account_type=AccountType.CASH,  # Spot CASH account (not for perpetuals or futures)\n",
    "    base_currency=None,  # Multi-currency account\n",
    "    starting_balances=[Money(1_000_000.0, USDT), Money(10.0, ETH)],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {},
   "source": [
    "## Add data\n",
    "\n",
    "Now we can add data to the backtest engine. First add the `Instrument` object we previously initialized, which matches our data.\n",
    "\n",
    "Then we can add the trade ticks we wrangled earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add instrument(s)\n",
    "engine.add_instrument(ETHUSDT_BINANCE)\n",
    "\n",
    "# Add data\n",
    "engine.add_data(ticks)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {},
   "source": [
    ":::note\n",
    "The amount of and variety of data types is only limited by machine resources and your imagination (custom types are possible).\n",
    "Also, multiple venues can be used for backtesting, again only limited by machine resources.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15",
   "metadata": {},
   "source": [
    "## Add strategies\n",
    "\n",
    "Now we can add the trading strategies we’d like to run as part of our system.\n",
    "\n",
    ":::note\n",
    "Multiple strategies and instruments can be used for backtesting, only limited by machine resources.\n",
    ":::\n",
    "\n",
    "Firstly, initialize a strategy configuration, then use this to initialize a strategy which we can add to the engine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Configure your strategy\n",
    "strategy_config = EMACrossTWAPConfig(\n",
    "    instrument_id=ETHUSDT_BINANCE.id,\n",
    "    bar_type=BarType.from_str(\"ETHUSDT.BINANCE-250-TICK-LAST-INTERNAL\"),\n",
    "    trade_size=Decimal(\"0.10\"),\n",
    "    fast_ema_period=10,\n",
    "    slow_ema_period=20,\n",
    "    twap_horizon_secs=10.0,\n",
    "    twap_interval_secs=2.5,\n",
    ")\n",
    "\n",
    "# Instantiate and add your strategy\n",
    "strategy = EMACrossTWAP(config=strategy_config)\n",
    "engine.add_strategy(strategy=strategy)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17",
   "metadata": {},
   "source": [
    "You may notice that this strategy config includes parameters related to a TWAP execution algorithm.\n",
    "This is because we can flexibly use different parameters per order submit, we still need to initialize\n",
    "and add the actual `ExecAlgorithm` component which will execute the algorithm - which we'll do now.\n",
    "\n",
    "## Add execution algorithms\n",
    "\n",
    "NautilusTrader enables us to build up very complex systems of custom components. Here we show just one of the custom components\n",
    "available, in this case a built-in TWAP execution algorithm. It is configured and added to the engine in generally the same pattern as for strategies:\n",
    "\n",
    ":::note\n",
    "Multiple execution algorithms can be used for backtesting, only limited by machine resources.\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate and add your execution algorithm\n",
    "exec_algorithm = TWAPExecAlgorithm()  # Using defaults\n",
    "engine.add_exec_algorithm(exec_algorithm)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19",
   "metadata": {},
   "source": [
    "## Run backtest\n",
    "\n",
    "Now that we have our data, venues and trading system configured - we can run a backtest\n",
    "Simply call the `.run(...)` method which will run a backtest over all available data by default.\n",
    "\n",
    "See the [BacktestEngineConfig](https://nautilustrader.io/docs/latest/api_reference/config) API reference for a complete description of all available methods and options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the engine (from start to end of data)\n",
    "engine.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21",
   "metadata": {},
   "source": [
    "## Post-run and analysis\n",
    "\n",
    "Once the backtest is completed, a post-run tearsheet will be automatically logged using some\n",
    "default statistics (or custom statistics which can be loaded, see the advanced [Portfolio statistics](../concepts/advanced/portfolio_statistics.md) guide).\n",
    "\n",
    "Also, many resultant data and execution objects will be held in memory, which we\n",
    "can use to further analyze the performance by generating various reports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {},
   "outputs": [],
   "source": [
    "engine.trader.generate_account_report(BINANCE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "engine.trader.generate_order_fills_report()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24",
   "metadata": {},
   "outputs": [],
   "source": [
    "engine.trader.generate_positions_report()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25",
   "metadata": {},
   "source": [
    "## Repeated runs\n",
    "\n",
    "We can also choose to reset the engine for repeated runs with different strategy and component configurations.\n",
    "Calling the `.reset(...)` method will retain all loaded data and components, but reset all other stateful values\n",
    "as if we had a fresh `BacktestEngine` (this avoids having to load the same data again)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For repeated backtest runs make sure to reset the engine\n",
    "engine.reset()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27",
   "metadata": {},
   "source": [
    "Individual components (actors, strategies, execution algorithms) need to be removed and added as required.\n",
    "\n",
    "See the [Trader](../api_reference/trading.md) API reference for a description of all methods available to achieve this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Once done, good practice to dispose of the object if the script continues\n",
    "engine.dispose()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}