whatIfOrder sometimes returns incorrect data

[mdelvaux]

I was writing code that uses the whatIf order to check margin requirements. Surprisingly that code was working on Thursday, returning reasonable values, and broken on Friday, returning a status with fields at infinity.

The clue was an other difference, the failing cases had a non empty warningText, an example is shown below. My theory was then something like this

• code places the whatIf message
• TWS (not the IB server) detects the potential issue and immediately return an orderStatus with the warning
• The arrival of the orderStatus terminates the library call, even if the IB server has not responded itself.
• The returned orderState is then the default orderState, i.e. with all fields still at their default value

I did some tracing and it seems the interpretation is at least roughly correct, I won't say I understand all the nuances of ib_insync, but I could trace the issue to method openOrder in wrapper.py. The failed cases enter twice that method, the first call has the nonsensical orderState with the warning text, the second one is the one with the correct values.

I made a terrible hack that captures the orderState passed to openOrder before openOrder calls self._endReq(order.orderId, orderState). Then the code polls on the field initMarginBefore of the captured value with a delay of ib.sleep(0.01) and reports the number of loop elapsed and any new observed value. I get this trace, that shows that the correct value appears about 20 ms after the warning text. The orderState shown is the orderState returned by whatIfOrder(), i.e. with the non sensical values.

0 1.7976931348623157e+308
2 306.07
OrderState(status='PreSubmitted', initMarginBefore='1.7976931348623157E308', maintMarginBefore='1.7976931348623157E308', equityWithLoanBefore='1.7976931348623157E308', initMarginChange='1.7976931348623157E308', maintMarginChange='1.7976931348623157E308', equityWithLoanChange='1.7976931348623157E308', initMarginAfter='1.7976931348623157E308', maintMarginAfter='1.7976931348623157E308', equityWithLoanAfter='1.7976931348623157E308', commission=1.7976931348623157e+308, minCommission=1.7976931348623157e+308, maxCommission=1.7976931348623157e+308, commissionCurrency='', warningText='If your order is not immediately executable, our systems may, depending on market conditions, reject your order if its limit price is more than the allowed amount away from the reference price at that time. If this happens, you will not receive a fill. This is a control designed to ensure that we comply with our regulatory obligations to avoid submitting disruptive orders to the marketplace.', completedTime='', completedStatus='')

I captured the API messages as shown by the IB Gateway while the corresponding code was running, showing the two different messages.

20:15:47:019 <- 3-8-265598-AAPL-STK--0.0---SMART-NASDAQ-USD-AAPL-NMS---BUY-100-LMT-126-----O-0--1-0-0-0-0-0-0-0--0.0
--------0---1-0---0---0-0--0------0-----0-----------0---0-0---1--0-0-0-0-------0---------0-0-0-0-
20:15:47:095 -> --�á5-8-265598-AAPL-STK--0-?--SMART-USD-AAPL-NMS-BUY-100-LMT-126.0-0.0-DAY--DU155368--0--0-1282568749
-0-0-0--1282568749.0/DU155368/100----------0---1-0-------0-0-0--3-0-0--0-0--0-None--0----?-0-0--0-0
------0-0-0-----0--IB-0-0--0-1-PreSubmitted-1.7976931348623157E308-1.7976931348623157E308-
1.7976931348623157E308-1.7976931348623157E308-1.7976931348623157E308-1.7976931348623157E308-
1.7976931348623157E308-1.7976931348623157E308-1.7976931348623157E308-----
If your order is not immediately executable, our systems may, depending on market conditions,
reject your order if its limit price is more than the allowed amount away from the reference price
at that time. If this happens, you will not receive a fill. This is a control designed to ensure that
we comply with our regulatory obligations to avoid submitting disruptive orders to the marketplace
.-0-0-0-None-1.7976931348623157E308-127.0-1.7976931348623157E308-1.7976931348623157E308
-1.7976931348623157E308-1.7976931348623157E308-0----0-1-0-0-0-
20:15:47:119 -> --�Þ5-8-265598-AAPL-STK--0-?--SMART-USD-AAPL-NMS-BUY-100-LMT-126.0-0.0-DAY--DU155368--0--0-1282568749
-0-0-0--1282568749.0/DU155368/100----------0---1-0-------0-0-0--3-0-0--0-0--0-None--0----?-0-0--0-0
------0-0-0-----0--IB-0-0--0-1-PreSubmitted-306.07-251.2-235204.8-4155.6900000000005-3777.9--
8.0-4461.76-4029.1-235196.8--0.9-1.0-USD--0-0-0-None-
1.7976931348623157E308-127.0-1.7976931348623157E308-1.7976931348623157E308-1.7976931348623157E308
-1.7976931348623157E308-0----0-1-0-0-0-

I am afraid I am not familiar enough to propose a solution, except a hacky one that would look at some fields inside orderState and somehow stop processing that message, waiting for a subsequent one. The problem is to be sure that there will be a subsequent one.

Thank you for making ib_insync available, I would not dream coding against the raw API as provided by IB itself.

[mdelvaux]

I hacked the code as shown below, with the hack active I get correct results, in how far that approach is robust is a different question

[mdelvaux]

Looking carefully, there is a more obvious difference, orderState.commissionCurrency is either empty (for the first, incorrect message) or a valid currency. The following code also works and may be preferable to the first one. I will now also check for changes in the order itself to see if anything explicit is present.

[erdewit]

Thanks for your report and investigation of the issue. I think testing for the commissionCurrency is a good solution and will add that to the code.

[stopdesign]

Sometimes (paper trading outside RTH?) I get e+308 commission value even with valid commissionCurrency:

commissionCurrency: USD
commission: 1.7976931348623157e+308
minCommission: 0.32
maxCommission: 0.39

OrderState(status='PreSubmitted', initMarginBefore='15841.63', maintMarginBefore='15841.63', equityWithLoanBefore='110154.78', initMarginChange='114.0', maintMarginChange='114.0', equityWithLoanChange='-2.789999999993597', initMarginAfter='15955.63', maintMarginAfter='15955.63', equityWithLoanAfter='110151.99', commission=1.7976931348623157e+308, minCommission=0.323373, maxCommission=0.385373, commissionCurrency='USD', warningText='', completedTime='', completedStatus='')

[markmark1]

nvm found an example here ***@***.*** | Checking the margin with whatIfOrder in onBarUpdate def onBarUpdate(bars, hasNewBar): if (hasNewBar and long_signal): # Check for enough margin. test_order = LimitOrder(action='BUY', totalQuantity=100000, lmtPrice=entry_price) test_order_state = ib.whatIfOrder(bars.contract, test_order) if float(test_order_state.initMarginAfter) < trade_account_margin_max: # Place the real trade using a bracket order else: print("Insufficient margin for trade")

…

On Mon, Aug 2, 2021 at 3:50 PM Gregory Zhizhilkin ***@***.***> wrote: Sometimes (paper trading outside RTH?) I get e+308 commission value even with valid commissionCurrency: commissionCurrency: USD commission: 1.7976931348623157e+308 minCommission: 0.32 maxCommission: 0.39 OrderState(status='PreSubmitted', initMarginBefore='15841.63', maintMarginBefore='15841.63', equityWithLoanBefore='110154.78', initMarginChange='114.0', maintMarginChange='114.0', equityWithLoanChange='-2.789999999993597', initMarginAfter='15955.63', maintMarginAfter='15955.63', equityWithLoanAfter='110151.99', commission=1.7976931348623157e+308, minCommission=0.323373, maxCommission=0.385373, commissionCurrency='USD', warningText='', completedTime='', completedStatus='') — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.