Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backtesting results are not the same #175

Closed
jashan1498 opened this issue Dec 20, 2021 · 13 comments
Closed

backtesting results are not the same #175

jashan1498 opened this issue Dec 20, 2021 · 13 comments

Comments

@jashan1498
Copy link

jashan1498 commented Dec 20, 2021 •
edited

@iterativv @eSeR1805
backtesting results are not same as the one presented on github , can you check what might be missing ?

strategy name : NostalgiaForInfinityX
pairlist used : https://github.com/iterativv/NostalgiaForInfinityData/blob/main/binance-usdt-static.json

"max_open_trades": 5,
"stake_currency": "USDT",
"stake_amount": "unlimited",
"dry_run_wallet":1000,
"use_sell_signal":true,
"tradable_balance_ratio": 0.99,
"fiat_display_currency": "USD",
"timeframe": "15m",
"dry_run": true,
@eSeR1805
Copy link
Collaborator

"timeframe": "15m",

@jashan1498
Copy link
Author

jashan1498 commented Dec 20, 2021 •
edited

@eSeR1805 timeframe : "15m" returns this result , still very different from presented backtesting results

=============== SUMMARY METRICS ================
| Metric | Value |
|------------------------+---------------------|
| Backtesting from | 2021-12-10 00:00:00 |
| Backtesting to | 2021-12-20 21:00:00 |
| Max open trades | 5 |
| | |
| Total/Daily Avg Trades | 17 / 1.7 |
| Starting balance | 1000.000 USDT |
| Final balance | 1030.616 USDT |
| Absolute profit | 30.616 USDT |
| Total profit % | 3.06% |
| Trades per day | 1.7 |
| Avg. daily profit % | 0.31% |
| Avg. stake amount | 203.929 USDT |
| Total trade volume | 3466.786 USDT |
| | |
| Best Pair | GXS/USDT 11.76% |
| Worst Pair | EGLD/USDT -22.42% |
| Best trade | CAKE/USDT 8.72% |
| Worst trade | EGLD/USDT -22.42% |
| Best day | 22.204 USDT |
| Worst day | -36.011 USDT |
| Days win/draw/lose | 6 / 1 / 1 |
| Avg. Duration Winners | 10:01:00 |
| Avg. Duration Loser | 2 days, 0:52:00 |
| Rejected Buy signals | 0 |
| | |
| Min balance | 1002.977 USDT |
| Max balance | 1066.628 USDT |
| Drawdown | 22.42% |
| Drawdown | 45.672 USDT |
| Drawdown high | 66.628 USDT |
| Drawdown low | 20.956 USDT |
| Drawdown Start | 2021-12-19 09:15:00 |
| Drawdown End | 2021-12-20 06:00:00 |
| Market change | -12.56% |

@eSeR1805
Copy link
Collaborator

and what timeframe does the strategy file specify?

@jashan1498
Copy link
Author

it specifies .. timeframe = '15m'

@eSeR1805
Copy link
Collaborator

https://github.com/iterativv/NostalgiaForInfinity/blob/main/NostalgiaForInfinityX.py
are you for real? 😄

@jashan1498
Copy link
Author

jashan1498 commented Dec 20, 2021 •
edited

i have tried both 5m and 15m in config file.

i thought you want me to change it to "15m" based on this comment
#175 (comment)

@jashan1498
Copy link
Author

jashan1498 commented Dec 20, 2021 •
edited

@eSeR1805

timeframe : "5m" both in config file and strategy

=============== SUMMARY METRICS ================
| Metric | Value |
|------------------------+---------------------|
| Backtesting from | 2021-12-10 00:00:00 |
| Backtesting to | 2021-12-20 22:00:00 |
| Max open trades | 5 |
| | |
| Total/Daily Avg Trades | 5 / 0.5 |
| Starting balance | 1000.000 USDT |
| Final balance | 958.021 USDT |
| Absolute profit | -41.979 USDT |
| Total profit % | -4.20% |
| Trades per day | 0.5 |
| Avg. daily profit % | -0.42% |
| Avg. stake amount | 199.003 USDT |
| Total trade volume | 995.014 USDT |
| | |
| Best Pair | AR/USDT 6.75% |
| Worst Pair | EGLD/USDT -19.51% |
| Best trade | AR/USDT 6.75% |
| Worst trade | EGLD/USDT -19.51% |
| Best day | 13.416 USDT |
| Worst day | -64.259 USDT |
| Days win/draw/lose | 3 / 4 / 1 |
| Avg. Duration Winners | 1:57:00 |
| Avg. Duration Loser | 4 days, 0:38:00 |
| Rejected Buy signals | 0 |
| | |
| Min balance | 958.021 USDT |
| Max balance | 1022.280 USDT |
| Drawdown | 32.33% |
| Drawdown | 64.259 USDT |
| Drawdown high | 22.280 USDT |
| Drawdown low | -41.979 USDT |
| Drawdown Start | 2021-12-18 17:35:00 |
| Drawdown End | 2021-12-20 22:00:00 |
| Market change | -12.22% |

@eSeR1805
Copy link
Collaborator

we don't have December data in CI backtest yet and strategy is 5m. I'm not sure what you're comparing to what

@jashan1498
Copy link
Author

jashan1498 commented Dec 20, 2021 •
edited

@eSeR1805 backtesting from 2021-11 to 2021-12 .. but still not the same

"max_open_trades": 5,
"stake_currency": "USDT",
"stake_amount": "unlimited",
"dry_run_wallet":1000,
"use_sell_signal":true,
"tradable_balance_ratio": 0.99,
"fiat_display_currency": "USD",
"timeframe": "5m",
"dry_run": true,

=============== SUMMARY METRICS ================
| Metric | Value |
|------------------------+---------------------|
| Backtesting from | 2021-11-01 00:00:00 |
| Backtesting to | 2021-12-01 00:00:00 |
| Max open trades | 5 |
| | |
| Total/Daily Avg Trades | 35 / 1.17 |
| Starting balance | 1000.000 USDT |
| Final balance | 1216.059 USDT |
| Absolute profit | 216.059 USDT |
| Total profit % | 21.61% |
| Trades per day | 1.17 |
| Avg. daily profit % | 0.72% |
| Avg. stake amount | 217.838 USDT |
| Total trade volume | 7624.331 USDT |
| | |
| Best Pair | TLM/USDT 23.30% |
| Worst Pair | ZEN/USDT -9.66% |
| Best trade | TLM/USDT 19.53% |
| Worst trade | ZEN/USDT -9.66% |
| Best day | 88.919 USDT |
| Worst day | -13.362 USDT |
| Days win/draw/lose | 12 / 13 / 1 |
| Avg. Duration Winners | 4:44:00 |
| Avg. Duration Loser | 2 days, 1:25:00 |
| Rejected Buy signals | 7232 |
| | |
| Min balance | 1000.354 USDT |
| Max balance | 1238.799 USDT |
| Drawdown | 9.66% |
| Drawdown | 22.740 USDT |
| Drawdown high | 238.799 USDT |
| Drawdown low | 216.059 USDT |
| Drawdown Start | 2021-11-28 06:00:00 |
| Drawdown End | 2021-11-28 09:50:00 |
| Market change | 15.00% |

@eSeR1805
Copy link
Collaborator

Yeah not enough info for further debug. Either some missing data for slightly different parilist. In any case this isn't the purpose of this issue tracker.

@jashan1498
Copy link
Author

@eSeR1805 can you also mention which timeframes of data we need to download for strat ?
5m , 15m, 1h , 1d , -- are these all ?

@jashan-pointr
Copy link

jashan-pointr commented Dec 21, 2021 •
edited

@eSeR1805 this is the config file , can you please let me know what is missing / incorrect in this ?

{
    "max_open_trades": 5,
    "stake_currency": "USDT",
    "stake_amount": "unlimited",
    "dry_run_wallet":1000,
    "use_sell_signal":true,
    "tradable_balance_ratio": 0.99,
    "fiat_display_currency": "USD",
    "timeframe": "5m",
    "dry_run": true,
    "cancel_open_orders_on_exit": false,
    "unfilledtimeout": {
        "buy": 10,
        "sell": 30,
        "unit": "minutes"
    },
    "order_types": {
        "buy": "market",
        "sell": "market",
        "emergencysell": "market",
        "trailing_stop_loss": "market",
        "stoploss": "market",
        "stoploss_on_exchange": false,
        "stoploss_on_exchange_interval": 60
    },
    "bid_strategy": {
        "price_side": "ask",
        "ask_last_balance": 0.0,
        "use_order_book": false,
        "order_book_top": 1,
        "check_depth_of_market": {
            "enabled": false,
            "bids_to_ask_delta": 1
        }
    },
    "ask_strategy": {
        "price_side": "bid",
        "use_order_book": false,
        "order_book_top": 1
    },
    "exchange": {
        "name": "binance",
        "key": "",
        "secret": "",
        "ccxt_config": {"enableRateLimit": true},
        "ccxt_async_config": {
            "enableRateLimit": true,
            "rateLimit": 200
        },
        "pair_whitelist": [
            "AAVE/USDT",
            "ADA/USDT",
            "AION/USDT",
            "ALGO/USDT",
            "ALICE/USDT",
            "ALPHA/USDT",
            "ANKR/USDT",
            "AR/USDT",
            "ATM/USDT",
            "ATOM/USDT",
            "AUDIO/USDT",
            "AVA/USDT",
            "AVAX/USDT",
            "AXS/USDT",
            "BAKE/USDT",
            "BAT/USDT",
            "BCH/USDT",
            "BTC/USDT",
            "BTCST/USDT",
            "BTG/USDT",
            "BTT/USDT",
            "BURGER/USDT",
            "CAKE/USDT",
            "CELO/USDT",
            "CELR/USDT",
            "CHR/USDT",
            "CHZ/USDT",
            "CKB/USDT",
            "COCOS/USDT",
            "COMP/USDT",
            "CRV/USDT",
            "CTSI/USDT",
            "DASH/USDT",
            "DATA/USDT",
            "DEGO/USDT",
            "DENT/USDT",
            "DGB/USDT",
            "DNT/USDT",
            "DOCK/USDT",
            "DODO/USDT",
            "DOGE/USDT",
            "DOT/USDT",
            "EGLD/USDT",
            "ENJ/USDT",
            "EOS/USDT",
            "ETC/USDT",
            "ETH/USDT",
            "FIL/USDT",
            "FTM/USDT",
            "GRT/USDT",
            "GTO/USDT",
            "GXS/USDT",
            "HARD/USDT",
            "HBAR/USDT",
            "HNT/USDT",
            "HOT/USDT",
            "IOST/USDT",
            "IOTA/USDT",
            "IOTX/USDT",
            "KAVA/USDT",
            "KSM/USDT",
            "LINK/USDT",
            "LPT/USDT",
            "LTC/USDT",
            "LUNA/USDT",
            "MANA/USDT",
            "MASK/USDT",
            "MATIC/USDT",
            "MBL/USDT",
            "MKR/USDT",
            "MTL/USDT",
            "NEO/USDT",
            "NKN/USDT",
            "NMR/USDT",
            "OM/USDT",
            "OMG/USDT",
            "ONE/USDT",
            "ONG/USDT",
            "ONT/USDT",
            "OXT/USDT",
            "PERL/USDT",
            "PERP/USDT",
            "PNT/USDT",
            "POLS/USDT",
            "POND/USDT",
            "QTUM/USDT",
            "REEF/USDT",
            "RLC/USDT",
            "RUNE/USDT",
            "RVN/USDT",
            "SAND/USDT",
            "SC/USDT",
            "SKL/USDT",
            "SLP/USDT",
            "SNX/USDT",
            "SOL/USDT",
            "STMX/USDT",
            "STPT/USDT",
            "STRAX/USDT",
            "STX/USDT",
            "SUPER/USDT",
            "SUSHI/USDT",
            "SXP/USDT",
            "TFUEL/USDT",
            "THETA/USDT",
            "TKO/USDT",
            "TLM/USDT",
            "TRX/USDT",
            "TWT/USDT",
            "UNI/USDT",
            "VET/USDT",
            "WAVES/USDT",
            "WIN/USDT",
            "WING/USDT",
            "WRX/USDT",
            "XEM/USDT",
            "XLM/USDT",
            "XMR/USDT",
            "XRP/USDT",
            "XTZ/USDT",
            "XVS/USDT",
            "YFI/USDT",
            "ZEC/USDT",
            "ZEN/USDT",
            "ZIL/USDT",
            "ZRX/USDT",
        ],
        "pair_blacklist": [
            ]
    },
    "pairlists": [
        {"method": "StaticPairList"}
    ],
    "edge": {
        "enabled": false,
        "process_throttle_secs": 3600,
        "calculate_since_number_of_days": 7,
        "allowed_risk": 0.01,
        "stoploss_range_min": -0.01,
        "stoploss_range_max": -0.1,
        "stoploss_range_step": -0.01,
        "minimum_winrate": 0.60,
        "minimum_expectancy": 0.20,
        "min_trade_number": 10,
        "max_trade_duration_minute": 1440,
        "remove_pumps": false
    },
    "telegram": {
        "enabled": false,
        "token": "",
        "chat_id": ""
    },
    "api_server": {
        "enabled": false,
        "listen_ip_address": "127.0.0.1",
        "listen_port": 8080,
        "verbosity": "error",
        "enable_openapi": false,
        "jwt_secret_key": "",
        "CORS_origins": [],
        "username": "",
        "password": ""
    },
    "bot_name": "freqtrade",
    "initial_state": "running",
    "forcebuy_enable": false,
    "internals": {
        "process_throttle_secs": 5
    }
}

@eSeR1805
Copy link
Collaborator

Yes. you need all 5m, 15m, 1h, 1d.
As I've said there's not enough info to debug this here and it's not really the point of this issue tracker.
To clarify there's nothing here that guarantees you have downloaded enough of the right data. For example if you don't download startup_candle_count worth of informatives before your backtested timerange some indicators will be skewed by fillup data and that will ultimately affect the results also.

@eSeR1805 eSeR1805 closed this as completed Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jashan1498 @eSeR1805 @jashan-pointr