Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dru-run mode profit falls below the stoploss a lot, and how to print all current pending limit orders? #9953

Closed
zhaohui-yang opened this issue Mar 15, 2024 · 4 comments
Labels
Question Questions - will be closed after some period of inactivity. Strategy assistance Help with a strategy

Comments

@zhaohui-yang
Copy link

zhaohui-yang commented Mar 15, 2024

Describe your environment

  • Operating system: Ubuntu 22.04.3
  • Python Version: _Python 3.11.5 (python -V)
  • CCXT version: ccxt==4.2.15_ (pip freeze | grep ccxt)
  • Freqtrade Version: freqtrade 2024.1-dev-b4fb13370 (freqtrade -V or docker compose run --rm freqtrade -V for Freqtrade running in docker)

Your question

Ask the question you have not been able to find an answer in the Documentation

Thank you very much for reading this issue. I have some questions about stop loss.

1. The actual stop loss price exceeds the limit price set by stoploss.

My config is set as below:

    "stoploss": -0.007,
    "trailing_stop": true,
    "trailing_only_offset_is_reached": false,
    "trailing_stop_positive_offset": 0.0002,
    "trailing_stop_positive": 0.0001,
    "use_custom_stoploss": true,

    "order_types": {
        "entry": "limit",
        "exit": "limit",
        "emergency_exit": "market",
        "stoploss": "limit",
        "stoploss_on_exchange": true,
        "stoploss_price_type": "last",
        "stoploss_on_exchange_interval": 10,
        "stoploss_on_exchange_limit_ratio": 0.99
    },
    "unfilledtimeout": {
        "entry": 20,
        "exit": 20,
        "exit_timeout_count": 0,
        "unit": "minutes"
    },
    "entry_pricing": {
        "price_side": "same",
        "use_order_book": true,
        "order_book_top": 1,
        "price_last_balance": 0.0,
        "check_depth_of_market": {
            "enabled": false,
            "bids_to_ask_delta": 1
        }
    },
    "exit_pricing":{
        "price_side": "same",
        "use_order_book": true,
        "order_book_top": 1
    },
    "exchange": {
        "name": "binance",

In dry-run mode, my trading pairs are DOT/USDT:USDT and etc. The leverage is set to 1.

The following logs are copied from telegram.

Close Date    Pair (ID)                    Profit (USDT)
------------  ---------------------------  ---------------------
2 hours ago   1000SATS/USDT:USDT (#6741)   0.42% (1.38425139)
2 hours ago   1000FLOKI/USDT:USDT (#6706)  3.60% (11.86468788)
2 hours ago   NEAR/USDT:USDT (#6682)       0.44% (1.442853)
2 hours ago   1000SATS/USDT:USDT (#6688)   0.15% (0.48604587)
3 hours ago   WLD/USDT:USDT (#6708)        1.95% (6.41469564)
3 hours ago   GALA/USDT:USDT (#6709)       1.92% (6.31830738)
3 hours ago   MATIC/USDT:USDT (#6692)      1.83% (6.01610022)
3 hours ago   1000SHIB/USDT:USDT (#6705)   1.50% (4.94921537)
3 hours ago   YGG/USDT:USDT (#6704)        1.93% (6.35652992)
3 hours ago   FIL/USDT:USDT (#6696)        1.42% (4.66424856)
3 hours ago   RNDR/USDT:USDT (#6701)       2.58% (8.48922121)
3 hours ago   DOGE/USDT:USDT (#6700)       1.60% (5.25794065)
3 hours ago   1000PEPE/USDT:USDT (#6693)   1.06% (3.5059336)
3 hours ago   AVAX/USDT:USDT (#6687)       0.69% (2.0181504)
3 hours ago   CELO/USDT:USDT (#6695)       1.01% (3.3345501)
3 hours ago   WLD/USDT:USDT (#6702)        1.17% (3.75180962)
3 hours ago   1000FLOKI/USDT:USDT (#6507)  0.37% (1.21415856)
4 hours ago   ETH/USDT:USDT (#6309)        0.05% (0.16221307)
5 hours ago   FOOTBALL/USDT:USDT (#5987)   0.32% (1.02666956)
6 hours ago   SOL/USDT:USDT (#5662)        0.57% (1.0110524)
8 hours ago   1000SHIB/USDT:USDT (#5274)   0.39% (1.28793095)
8 hours ago   BTC/USDT:USDT (#5265)        -0.06% (-0.15521792)
8 hours ago   GALA/USDT:USDT (#5276)       0.44% (1.43745559)
8 hours ago   AVAX/USDT:USDT (#5273)       3.44% (10.845546)
8 hours ago   ETH/USDT:USDT (#5267)        2.13% (6.94747492)
8 hours ago   ARB/USDT:USDT (#5277)        0.86% (2.81270565)
8 hours ago   XRP/USDT:USDT (#5268)        0.48% (1.5891542)
8 hours ago   DOGE/USDT:USDT (#5263)       -0.00% (-0.01259107)
8 hours ago   RNDR/USDT:USDT (#5287)       2.26% (7.42207022)
8 hours ago   1000PEPE/USDT:USDT (#5264)   0.41% (1.32953891)
8 hours ago   WLD/USDT:USDT (#5282)        1.30% (4.23469864)
8 hours ago   SOL/USDT:USDT (#5270)        1.33% (2.3052926)
8 hours ago   YGG/USDT:USDT (#5288)        0.09% (0.2924075)
8 hours ago   RUNE/USDT:USDT (#5289)       1.10% (3.578918)
8 hours ago   CELO/USDT:USDT (#5283)       0.50% (1.6276701)
8 hours ago   1000FLOKI/USDT:USDT (#5281)  0.45% (1.47800858)
8 hours ago   MATIC/USDT:USDT (#5272)      0.27% (0.8840302)
8 hours ago   ADA/USDT:USDT (#5269)        0.48% (1.58172838)
8 hours ago   1000SATS/USDT:USDT (#5290)   1.69% (5.54148809)
8 hours ago   DOT/USDT:USDT (#5280)        0.35% (1.1398762)
8 hours ago   NEAR/USDT:USDT (#5286)       -0.13% (-0.409976)
8 hours ago   LTC/USDT:USDT (#5275)        -0.15% (-0.50274655)
8 hours ago   LINK/USDT:USDT (#5278)       -0.02% (-0.06282443)
8 hours ago   FIL/USDT:USDT (#5284)        -0.87% (-2.8558617)
8 hours ago   1000SATS/USDT:USDT (#5285)   -0.13% (-0.41807299)
9 hours ago   LINK/USDT:USDT (#5133)       1.91% (6.26271228)
9 hours ago   DOT/USDT:USDT (#5049)        -2.49% (-8.188848)
9 hours ago   BTC/USDT:USDT (#5130)        1.22% (3.32426696)
9 hours ago   FIL/USDT:USDT (#5047)        -3.03% (-9.98278568)
9 hours ago   YGG/USDT:USDT (#5051)        -3.39% (-11.13401952)
9 hours ago   1000FLOKI/USDT:USDT (#5045)  -1.89% (-6.20461864)
9 hours ago   CELO/USDT:USDT (#5132)       0.71% (2.3290904)
9 hours ago   1000PEPE/USDT:USDT (#5131)   0.75% (2.45356566)
9 hours ago   MATIC/USDT:USDT (#5044)      -2.45% (-8.049701)
9 hours ago   XRP/USDT:USDT (#5043)        -2.22% (-7.32218016)
9 hours ago   FOOTBALL/USDT:USDT (#5072)   -0.31% (-1.020418)
9 hours ago   1000PEPE/USDT:USDT (#5036)   -0.18% (-0.57937194)
10 hours ago  SOL/USDT:USDT (#4746)        -1.79% (-3.2736208)
11 hours ago  ETH/USDT:USDT (#4533)        0.07% (0.22042356)
11 hours ago  TRX/USDT:USDT (#4464)        -0.00% (-0.00412752)

In my understanding, the limit stoploss order should be a limit price order. When the price exceeds this order, it will be executed immediately. Then the maximum loss of the transaction should be the handling fee at the time of transaction + a spread ( if market order) + the maximum stoploss cost of 70 bp. I would like to ask why there is a loss of, for example, 3%? In my opinion, trading fee, 4bp + spread + stoploss 70bp should not hit 300bp's lose. I wonder why this happened?

The trades with big losses happened 11 hours ago, which were the begining time of starting the strategy. Not sure if the strategy needs some warnup?

I find the messages of them:

Freqtrade bot, [2024/3/15 10:55]
❌ Binance (dry): Exited YGG/USDT:USDT (#5051)
Candle OHLC: 0.8691, 0.8752, 0.8669, 0.8727
Profit: -3.39% (loss: -11.134 USDT / -11.156 USD)
Exit Reason: trailing_stop_loss
Direction: Long
Amount: 362
Open Rate: 0.908 USDT
Exit Rate: 0.877 USDT
Duration: 0:32:08 (32.1 min)

Freqtrade bot, [2024/3/15 10:56]
⚠ Binance (dry): Exited FIL/USDT:USDT (#5047)
Candle OHLC: 9.858, 9.913, 9.857, 9.908
Profit: -3.03% (loss: -9.983 USDT / -10.003 USD)
Exit Reason: stop_loss
Direction: Long
Amount: 32.2
Open Rate: 10.214 USDT
Exit Rate: 9.908 USDT
Duration: 0:32:55 (32.9 min)

For the FIL/USDT:USDT, the open rate is 10.214, if I use the 70bp stoploss, then the stoploss should be around 10.14, rather than the exit rate of 9.908. As I read the document, the stoploss should be smaller than the currently used by default in the freqtrade system. So I wonder why the exit reason is 'stop_loss' and the exit profit is -3.03%.

As for the YGG/USDT:USDT order, it triggered the trailing stop offset, In this time, I set the trailing stoploss of 1bp, am I missing the trailing stoploss order placed on the market?

Therefore, as for the dry-run and production mode, is the profit result calculated related to the bar data or the latest quote data? or a smaller timeframe, e.g., 5s bar data?

In the backtest mode, the stoploss result seems like below, which is a little larger than the 70bp I set.

======================================================= EXIT REASON STATS ========================================================
|        Exit Reason |   Exits |   Win  Draws  Loss  Win% |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |
|--------------------+---------+--------------------------+----------------+----------------+-------------------+----------------|
| trailing_stop_loss |     106 |     90     0    16  84.9 |           0.09 |           9.98 |         103241    |           9.98 |
|          stop_loss |       1 |      0     0     1     0 |          -0.74 |          -0.74 |          -7870.63 |          -0.74 |

2. How to scan and print all pending orders of the current bot?

I want to send all the open orders running by the bot to the telegram, how can I fetch all the pending open orders?

3. Is this order right?

Trade ID: 7657 (since 13 minutes ago)
Current Pair: SOL/USDT:USDT
Direction: Short  (1.0x)
Amount: 1.0 (175.814 USDT)
Total invested: 175.814 USDT
Exit Reason: trailing_stop_loss
Number of Entries: 1
Number of Exits: 1
Open Rate: 175.814
Open Date: 2024-03-15 13:02:23
 
Current Rate: 175.761
Unrealized Profit: -0.01% (-0.017 USDT)
 
Initial Stoploss: 177.04400000 (-0.70%)
Stoploss: 175.459 (-0.01%)
Stoploss distance: -0.302 (-0.17%)
Open Order: (limit buy rem=1.00000000)

The SOL/USDT:USDT is a short order, however, the stoploss is 175.459, which is lower than the current rate 175.761 and open rate 175.814. Shouldn't the stoploss larger than the current rate for the short order?

4. How to cancel enter long limit order and place enter short limit order

For example, I placed an enter long limit order and it still in an 'open' status, not filled, after analyzing the latest candle, I want to enter short. Then, how can I cancel the enter long limit price order and place a enter short limit price order on the market?

5. Sync orders?

In the production mode running, If I manually send a buy order from the exchange's APP, and it is filled. Will it be synced by the freqtrade bot and set the stoploss order for the manually inserted order?

Many thanks!

@zhaohui-yang zhaohui-yang added the Question Questions - will be closed after some period of inactivity. label Mar 15, 2024
@froggleston
Copy link
Contributor

  1. A global stoploss of -0.007 is 0.7% relative to trade open. It is a percentage, not a price or "point"-relative value. Similarly, your trailing stoploss percentage values are far too tight for backtesting.
  2. Check the telegram command documentation: https://www.freqtrade.io/en/stable/telegram-usage/#status
  3. This is likely because you have trailing_stop set to True. Please read the documentation as to what this option means.
  4. I'd have to do some checking, but I'm not sure if you can cancel an open limit order through telegram (perhaps forceexit works? I've never tried it with an unfilled order)
  5. No.

@xmatthias
Copy link
Member

xmatthias commented Mar 15, 2024

There's no "startup time" for a bot (not unless you do something in your strategy requiring this).

Now i find your stoploss is WAY too tight at 0.7%.
In my experience, anything below 1% is unreliable at very short timeframes - at anything >= 5m - you'll want to work with 2-3% minimum.

The minimum asset tick size will also play a part in this - especially at values this low (you can't have a stop lower than the tick size - it'll always round up (the alternative would be an automatic exit at entry rate).

Your stop processing is then further constraint by the frequency of the bot checking on each pair (this is by default 5s - which is however a minimum. Ech additional open trade adds to this time - if you have open orders, the time will probably be lengthened (it's active calls against the exchange).
Essentially, this means that the price for each pair might be checked every 5 seconds - or more - so if we're at 20s - then the bot will be blind for all movements in between.

that's however not always relevant - as on this pair - 1 order of 1000$ can move price by several ticks - so even one second can make a difference - which is why i don't think a stoploss of <1.5-2% is realistic on such low-volume/value pairs

That particular pair also seems to have a fairly light orderbook (an order of a few $ can push price by mutiple orderbook steps - this is also something that's considered for market orders).

In summary: a stop will try to protect your loss - but there's no guarantee that it fills (if it's a limit order) - or at what price it fills (if it's a stop order).

  1. you can use /status to get all open trades.

  2. looks correct - it's a trailing stop - it started trailing - hence the "stop" price is above (or in case of a short below) the entry price (the order also exited at a win).
    With these trailing settings though - you'll have most trades exit at 0.01% profit (so nothing) - as a move of 0.01% to the downside from it's high will trigger stop.
    Stoploss on exchange in such a situations will fail probably 50% of the time (fail to be placed as "current price" is below the expected trigger price, resulting in an emergency exit).

  3. use the check_*_timeout() helpers

  4. no. Orders not opened by freqtrade will not be considered. Also for futures, we assume that freqtrade "owns" the whole (sub-)account - so manually opening and closing orders nearby can mess with some calculations (mainly for stake size) - and should be considered a non-supported setup.

edit i assume bp assumes to [Basis point](https://www.investopedia.com/ask/answers/what-basis-point-bps/#:~:text=a%20financial%20instrument.-,One%20basis%20point%20is%20equivalent%20to%200.01%25%20(1%2F100th,basis%20points%20number%20by%20100.) - which is 0.01% - and would align with your settings.

@xmatthias xmatthias added the Strategy assistance Help with a strategy label Mar 15, 2024
@zhaohui-yang
Copy link
Author

zhaohui-yang commented Mar 17, 2024

Thanks for the detailed comments! @xmatthias @froggleston . Let me check the documentation again! I am still confused about the prices of some stop-loss orders.

Also, is there any relevant configuration that can make the open rate in telegram display more decimal point precision? For some trading pairs, the decimal does not seem to be sufficient. For example,

✓ Binance (dry): New Trade filled (#4398)
Pair: 1000PEPE/USDT:USDT
Candle OHLC: 0.0072913, 0.007325, 0.0072744, 0.0073153
Amount: 226035
Direction: Short (5x)
Open Rate: 0.007 USDT
Total: 331.752 USDT / 331.405 USD

And, yes, bp is basis point, 1 bp = 0.01%, 20 bps = 0.2%.

In the process of optimizing some hyperparameters, the decimal point precision control method is usually used. I would like to ask if there are some control methods similar to step size, such as stoploss = DecimalParameter(0.01, 0.05, step=0.0020) and it will enumerate something like 0.0100, 0.0120, 0.0140 ... 0.0480, 0.0500. In this way, the step size is 20bps for the stoploss optimization. Below is the standard optimization parameter definition in the documentation.

class MyAwesomeStrategy(IStrategy):
    buy_adx = DecimalParameter(20, 40, decimals=1, default=30.1, space="buy")
    buy_rsi = IntParameter(20, 40, default=30, space="buy")
    buy_adx_enabled = BooleanParameter(default=True, space="buy")
    buy_rsi_enabled = CategoricalParameter([True, False], default=False, space="buy")
    buy_trigger = CategoricalParameter(["bb_lower", "macd_cross_signal"], default="bb_lower", space="buy")

@xmatthias
Copy link
Member

In the process of optimizing some hyperparameters, the decimal point precision control method is usually used. I would like to ask if there are some control methods similar to step size

you can implement this logic quite easily by doing the math in the strategy.

Also, is there any relevant configuration that can make the open rate in telegram display more decimal point precision? For some trading pairs, the decimal does not seem to be sufficient.

No - shown decimals are taken from the price precision given for each pair - or in case of quote currency values - from static values defined for that asset (3 decimals for USDT, 8 for BTC, ...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Question Questions - will be closed after some period of inactivity. Strategy assistance Help with a strategy
Projects
None yet
Development

No branches or pull requests

3 participants