Skip to content
This repository has been archived by the owner on Jun 8, 2023. It is now read-only.

✨ HyperOpt over 3 separate timeranges & individual spaces #51

Open
Rikj000 opened this issue Jun 5, 2021 · 8 comments
Open

✨ HyperOpt over 3 separate timeranges & individual spaces #51

Rikj000 opened this issue Jun 5, 2021 · 8 comments
Assignees
@Rikj000 @3ylo @topscoder
Labels
Feature - New New feature or pull request
Projects
MoniGoMani - Global Development Progress

Comments

@Rikj000
Copy link
Owner

Rikj000 commented Jun 5, 2021

The idea here is to:

HyperOpt over 3 separate timeranges (one representing each individual kind of trend, downwards/sideways/upwards, a timeframe that represents a corresponding trend should be picked). Experimental idea though, might not work at all.

How this would be implemented:

  • Make a version of MoniGoMani with 1 third of the parameters (basically no trend detection, but all the current signals etc remain)
  • HyperOpt that light version 3 times (once for upwards, once for downwards, once for sideways)
  • Combine those 3 hyperopt results (for each trend one) in the main MoniGoMani file again, and cross dem fingers 🤣
@Rikj000 Rikj000 added Feature - New New feature or pull request Planned Planned feature, improvement or bugfix (not being worked on yet) labels Jun 5, 2021
@3ylo
Copy link
Contributor

3ylo commented Jun 10, 2021

Hi Rik, can you please asssign this issue to me?I will start testing it tonight.
Thanks.

@Rikj000 Rikj000 added In Progress This is being worked on and removed Planned Planned feature, improvement or bugfix (not being worked on yet) labels Jun 10, 2021
@Rikj000 Rikj000 mentioned this issue Jun 16, 2021
Closed
@3ylo
Copy link
Contributor

3ylo commented Jun 17, 2021

I can finally post the results of the above test.
3 timeranges have been used that represent a different trend each followed by a final HO on a timerange that represents the current market.
A vanilla HO has been used to compare against.
Lastly a max_open_trades 6 backtest has been performed against both HOs.

Downtrend timerange:20210509-20210524
Uptrend timerange: 20210127-20210221
Sidetrend timerange: 20210518-20210610
Final timerange: 20210425-20210610

Final Trend_HO

strategy_MoniGoManiHyperStrategy_Final

Default go-to-command timerange for backtesting

Trend_HO_Default_Timerange

Default go-to-command timerange for backtesting max_open 6

Trend_HO_Default_Timerange_Max_6

Timerange used to HO

Trend_HO_Timerange

Timerange used to HO max_open 6

Trend_HO_Timerange_Max_6

Downtrend Timerange

Trend_HO_Downtrend

Downtrend TImerange Max_open 6

Trend_HO_Downtrend_Max_6

Uptrend Timerange

Trend_HO_Uptrend

Uptrend Timerange Max_open 6

Trend_HO_Uptrend_Max_6

Sidetrend Timerange

Trend_HO_Sidetrend

Sidetrend Timerange Max_open 6

Trend_HO_Sidetrend_Max_6

@3ylo
Copy link
Contributor

3ylo commented Jun 17, 2021 •
edited

Vanilla test results
Final HO
Vanilla_HO_final

Default go-to-command timerange for backtesting

Vanilla_backtest_default_timerange

Default go-to-command timerange for backtesting Max_open 6

Vanilla_backtest_default_timerange_Max_6

Timerange used to HO

Vanilla_backtest_HO_timerange

Timerange used to HO Max_open 6

Vanilla_backtest_HO_timerange_Max_6

Downtrend Timerange

Vanilla_Backtest_Downtrend

Downtrend Timerange Max_open 6

Vanilla_Backtest_Downtrend_Max_6

Uptrend Timerange

Vanilla_Backtest_Uptrend

Uptrend Timerange Max_open 6

Vanilla_Backtest_Uptrend_Max_6

Sidetrend Timerange

Vanilla_Backtest_Sidetrend

Sidetrend Timerange Max_open 6

Vanilla_Backtest_Sidetrend_Max_6

@3ylo
Copy link
Contributor

3ylo commented Jun 17, 2021

Some more information about the test.
Make signals only trigger once in lookback windows fix has not been implemented as the start day of the test was prior to the fix.
The HO's have been optimized using max_open -1 so the backtest using max_open 6 is not a valid performance indication but a last minute test to check Drawdown improvements.
The pairlist was generated by using the go-to-command and pulling 60 pairs after filtering, reducing that number might be beneficial in order to avoid smaller coins in our pairlist.

@Rikj000
Copy link
Owner Author

Rikj000 commented Jun 17, 2021

Very nice @3ylo! Thank you so much for the amazing test reports upon this issue! 🦾

Review

Final HO:

  • Pro: Avg duration ended up being a little less, which means less risk too!
  • Con: Overall profit seems a bit lower (keep in mind that we recently decided to maybe start using 800ish epochs instead of 1000 to prevent overfitting)

Default go-to-command timerange for backtesting:

  • Some zero-duration trades, but I'll neglect those for now, since they should have mostly been resolved by now with: ⚡️ Make signals only trigger once in lookback windows #43 (However I notice there are slightly more in the new version which we should keep in mind!)
  • Pro: Higher total profit 🥳
  • Pro: Dradwown is decently less!
  • Win/draw/lose ratio is better, but Avg duration winner/loser is not 🤔
  • FYI: As mentioned by the MGM Discord Community, we should aim to try and take rocketing pairs that create insane profits on a trade out by adding them to the blacklist or removing them from the static pairlist to prevent HyperOpt from overfitting on them

Default go-to-command timerange for backtesting Max_open 6:
These results show less profits, but they do look comparable to the previous backtests! Indicating that we could probably safely HyperOpt over 6 pairs instead of the 15 we where doing now and still get comparable end results out of our HyperOpt. This would save us a lot of time HyperOpting!

Further we can conclude that a top 60ish VolumePairList would still be a better idea for dry/live, since we can see that more pairs should result in more profit since it gives MGM more "good" buying options! 🎉

Timerange used to HO:

  • Con: First backtest where the new setup scores lower in profit! 👀
  • Pro: Dradwown is still less!

Timerange used to HO Max_open 6:
Dipping into negatives here 😮
But I think / hope HyperOpting over a smaller pairlist without crazy rocketing pairs, will make HO try to look for better overall buy/sell signals!

Uptrend/Downtrend:
Interesting, here we learn that the downtrend would be the most profitable trend for MGM to trade in at the moment!
However we must keep slippage in mind, because during market crashes / bearish movements our bots might have a harder time to make the orders go through in time

Conclusion

Overall I think it seems like an improvement to the optimization process! 🥳
But I'm not sure if the benefits outweigh the extra time needed for 4 HyperOpt Runs yet 🤔
We learned a ton here on how we can improve though!

HyperOpting separate spaces with the unclogger off in the beginning is also a big thing that plays in my head lately where I think we could see good improvements! Perhaps we could combine these 2 things 👀

Suggesion for new optimization process using this + separate spaces:

  • 1st 3 trend individual HyperOpt Runs only done upon --spaces buy sell, with stoploss at -1 and ROI {"0": 10} (disabled)
  • Combined refinement HyperOpt Run doing --spaces buy sell
  • Individual space HyperOpt Run using --spaces roi
  • Individual space HyperOpt Run using --spaces stoploss trailing
  • Individual space HyperOpt Run using --spaces sell + unclogger enabled

@Rikj000 Rikj000 mentioned this issue Jun 18, 2021
Closed
@Rikj000 Rikj000 changed the title HyperOpt over 3 separate timeranges HyperOpt over 3 separate timeranges & individual spaces Jun 18, 2021
@Rikj000 Rikj000 self-assigned this Jun 18, 2021
@denvers
Copy link
Contributor

denvers commented Jun 19, 2021 •
edited

I'm trying to put my two cents in. Here are some results.
--> Please let me know if I did it right, before I'm making the same mistakes over and over again ;-)

Config

spaces: buy sell
stoploss: -1
roi: {"0": 10}
unclogger: off
epochs: 1000

Downtrend (20210509-20210524)

HO run 1

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_14-27-07.fthypt --best
Schermafbeelding 2021-06-19 om 10 07 14

HO run 2

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_16-10-21.fthypt --best
Schermafbeelding 2021-06-19 om 10 07 52

HO run 3

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_16-44-26.fthypt --best
Schermafbeelding 2021-06-19 om 10 08 29

Backtest results Downtrend timerange (20210101-20210316)

Toggle 
source .env/bin/activate; freqtrade backtesting -s MoniGoManiHyperStrategy -c ./user_data/mgm-config.json -c 
./user_data/mgm-config-private.json -c ./user_data/mgm-config-hyperopt.json --timerange 20210101-20210316
Schermafbeelding 2021-06-19 om 10 34 55

Backtest results Current market (20210425-20210610)

Toggle 
source .env/bin/activate; freqtrade backtesting -s MoniGoManiHyperStrategy -c ./user_data/mgm-config.json -c ./user_data/mgm-config-private.json -c ./user_data/mgm-config-hyperopt.json --timerange 20210425-20210610
Schermafbeelding 2021-06-19 om 10 32 49

@Rikj000
Copy link
Owner Author

Rikj000 commented Jun 19, 2021

I'm trying to put my two cents in. Here are some results.
--> Please let me know if I did it right, before I'm making the same mistakes over and over again ;-)

Config

spaces: buy sell
stoploss: -1
roi: {"0": 10}
unclogger: off
epochs: 1000

Downtrend (20210509-20210524)

HO run 1

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_14-27-07.fthypt --best
Schermafbeelding 2021-06-19 om 10 07 14

HO run 2

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_16-10-21.fthypt --best
Schermafbeelding 2021-06-19 om 10 07 52

HO run 3

Toggle 
source .env/bin/activate; freqtrade hyperopt-list --hyperopt-filename=strategy_MoniGoManiHyperStrategy_2021-06-18_16-44-26.fthypt --best
Schermafbeelding 2021-06-19 om 10 08 29

Backtest results Downtrend timerange (20210101-20210316)

Toggle 
source .env/bin/activate; freqtrade backtesting -s MoniGoManiHyperStrategy -c ./user_data/mgm-config.json -c 
./user_data/mgm-config-private.json -c ./user_data/mgm-config-hyperopt.json --timerange 20210101-20210316
Schermafbeelding 2021-06-19 om 10 34 55

Backtest results Current market (20210425-20210610)

Toggle 
source .env/bin/activate; freqtrade backtesting -s MoniGoManiHyperStrategy -c ./user_data/mgm-config.json -c ./user_data/mgm-config-private.json -c ./user_data/mgm-config-hyperopt.json --timerange 20210425-20210610
Schermafbeelding 2021-06-19 om 10 32 49

Ah sorry, seems the instructions where a little unclear! :)
I meant to open the master file and then find the mgm_trends dictionary.

There just leave 1 trend in, for your 1st 3 HyperOpt runs (downwards timerange + downwards trend, etc)

Then HyperOpt spaces buy sell
Once you done that for each trend, then you combine the 3 json files (with a single trend in each one).

And once you combined them you start to HyperOpt the other spaces remaining one by one (roi, stoploss, trailing, unclogger)! ;)

@topscoder
Copy link
Collaborator

topscoder commented Jun 27, 2021 •
edited

⚠️

⚠️ Currently running HO's.... (will be updated...)

⚠️ Note: hurry commands refer to hurry cli tool at https://github.com/topscoder/hurry-cli

⚠️

Spaces: Buy, Sell

Downtrend

Click to expand ⬇️ 
timerange = 20210509-20210524
mgm_trends = [downwards]
spaces: buy sell
stoploss: -1
roi: {"0": 10}
unclogger: off
epochs: 800
🏃‍♂️ HO Run 1
downwards-ho1
🏃‍♂️ HO Run 2
downwards-ho2
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-01 om 11 41 47

Sidetrend

Click to expand ⬇️ 
timerange = 20210518-20210610
mgm_trends = [sideways]
spaces: buy sell
stoploss: -1
roi: {"0": 10}
unclogger: off
epochs: 800
🏃‍♂️ HO Run 1
Schermafbeelding 2021-06-30 om 20 00 06
🏃‍♂️ HO Run 2
sideways-ho2
🏃‍♂️ HO Run 3
sideways-ho3

Uptrend

Click to expand ⬇️ 
timerange = 20210127-20210221
mgm_trends = [upwards]
spaces: buy sell
stoploss: -1
roi: {"0": 10}
unclogger: off
epochs: 800
🏃‍♂️ HO Run 1
Schermafbeelding 2021-06-30 om 15 15 29
🏃‍♂️ HO Run 2
Schermafbeelding 2021-06-30 om 15 57 31
🏃‍♂️ HO Run 3
Schermafbeelding 2021-06-30 om 16 29 33

Space: ROI

Downtrend
Click to expand ⬇️ 
# Downtrend
hurry hyperopt --timerange=20210509-20210524 --spaces=roi
🏃‍♂️ HO Run 1
Schermafbeelding 2021-07-02 om 09 00 37
🏃‍♂️ HO Run 2
Schermafbeelding 2021-07-02 om 09 20 05
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-02 om 09 35 15

Sidetrend

Click to expand ⬇️ 
# Sidetrend
hurry hyperopt --timerange=side --spaces=roi
🏃‍♂️ HO Run 1
Schermafbeelding 2021-07-02 om 09 56 45
🏃‍♂️ HO Run 2
Schermafbeelding 2021-07-02 om 10 10 55
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-02 om 10 36 08

Uptrend

Click to expand ⬇️ 
# Uptrend
hurry hyperopt --timerange=up --spaces=roi
🏃‍♂️ HO Run 1
Schermafbeelding 2021-07-02 om 11 09 49
🏃‍♂️ HO Run 2
Schermafbeelding 2021-07-02 om 11 33 17
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-02 om 16 37 18

Space: Stoploss

Downtrend

Click to expand ⬇️ 
# Downtrend
hurry hyperopt --timerange=down --spaces=stoploss
🏃‍♂️ HO Run 1

⚠️ --spaces=buy sell stoploss ⚠️

stoploss-downtrend-ho1
🏃‍♂️ HO Run 2

⚠️ --spaces=stoploss ⚠️

stoploss-downtrend-ho2
🏃‍♂️ HO Run 3

⚠️ --spaces=stoploss ⚠️

stoploss-downtrend-ho3

Sidetrend

Click to expand ⬇️ 
# Sidetrend
hurry hyperopt --timerange=side --spaces=stoploss
🏃‍♂️ HO Run 1

⚠️ --spaces=buy sell stoploss ⚠️

stoploss-sidetrend-ho1
🏃‍♂️ HO Run 2

⚠️ --spaces=stoploss ⚠️

stoploss-sidetrend-ho2
🏃‍♂️ HO Run 3

⚠️ --spaces=stoploss ⚠️

stoploss-sidetrend-ho3

Uptrend

Click to expand ⬇️ 
# Uptrend
hurry hyperopt --timerange=up --spaces=stoploss
🏃‍♂️ HO Run 1

⚠️ --spaces=buy sell stoploss ⚠️

stoploss-uptrend-ho1
🏃‍♂️ HO Run 2

⚠️ --spaces=stoploss ⚠️

stoploss-uptrend-ho2
🏃‍♂️ HO Run 3
stoploss-uptrend-ho3

Space: Trailing

Downtrend

Click to expand ⬇️ 
# Downtrend
hurry hyperopt --timerange=down --spaces=trailing
🏃‍♂️ HO Run 1
Schermafbeelding 2021-07-02 om 18 03 22
🏃‍♂️ HO Run 2
Schermafbeelding 2021-07-02 om 18 14 43
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-02 om 18 25 47

Sidetrend

Click to expand ⬇️ 
# Sidetrend
hurry hyperopt --timerange=side --spaces=buy sell trailing

⚠️ --spaces=buy sell trailing ⚠️

🏃‍♂️ HO Run 1
trailing-sidetrend-ho1
🏃‍♂️ HO Run 2
trailing-sidetrend-ho2
🏃‍♂️ HO Run 3
Schermafbeelding 2021-07-05 om 19 07 40

Uptrend

Click to expand ⬇️ 
# Uptrend
hurry hyperopt --timerange=up --spaces=trailing
🏃‍♂️ HO Run 1
trailing-uptrend-ho1
🏃‍♂️ HO Run 2
trailing-uptrend-ho2
🏃‍♂️ HO Run 3
trailing-uptrend-ho3

Space: Unclogger

Downtrend

Click to expand ⬇️ 
# Uptrend
hurry hyperopt --timerange=down --spaces=!FIXME
🏃‍♂️ HO Run 1

-- WIP

🏃‍♂️ HO Run 2

-- WIP

🏃‍♂️ HO Run 3

-- WIP

Sidetrend

Click to expand ⬇️ 
# Sidetrend
hurry hyperopt --timerange=side --spaces=!FIXME
🏃‍♂️ HO Run 1

-- WIP

🏃‍♂️ HO Run 2

-- WIP

🏃‍♂️ HO Run 3

-- WIP

Uptrend

Click to expand ⬇️ 
# Uptrend
hurry hyperopt --timerange=up --spaces=!FIXME
🏃‍♂️ HO Run 1

-- WIP

🏃‍♂️ HO Run 2

-- WIP

🏃‍♂️ HO Run 3

-- WIP

@Rikj000 Rikj000 removed the In Progress This is being worked on label Aug 25, 2021
@Rikj000 Rikj000 changed the title HyperOpt over 3 separate timeranges & individual spaces ✨ HyperOpt over 3 separate timeranges & individual spaces Sep 5, 2021
@Rikj000 Rikj000 moved this from In Progress to Planned in MoniGoMani - Global Development Progress Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@Rikj000 Rikj000

@3ylo 3ylo

@topscoder topscoder

Labels
Feature - New New feature or pull request
Projects
MoniGoMani - Global Development Progress
  
Planned
Milestone
No milestone
Development

No branches or pull requests
4 participants
@denvers @Rikj000 @3ylo @topscoder