Missing data for trade details

[Flippo24]

When I am connected to the TWS, the individual prices of the single legs are missing as shown on your picture in the readme. Each line has its own P/L, which I would like to see. I am also interested in the current option price. Maybe you can display it in a column.

[PaulSquires]

There was a reason I removed it in this new version..... I just can't remember 100% why at the moment. I think it was related to the leg market price being correct but if the leg had been rolled previously then the leg cost basis was wrong. I will check into this again and put it back if it works without causing confusion for the user.

[Flippo24]

At this point you can view the calculation of the positions again. The amount on a leg is already the amount of the entire position. The individual position value is incorrect here.

[PaulSquires]

I am taking another look at this issue and will add back the leg cost, market value, difference, and percentage. Much like the older IB-Tracker functionality. Once I implement the changes then we should discuss it again to be sure that it meets our needs. I am hoping to have this change done today.

[PaulSquires]

Just finished adding the new functionality. Source code updated in repository. Looks pretty good. Here is a screenshot:

[Flippo24]

It looks good visually, but please check the calculation again. How can the values in both rows be the same in this case? Please describe how the values are calculated.
I assume that the red values on the right should signal a loss. Actually, it is still in profit.

[PaulSquires]

For the option legs:

• First column is the average cost of the leg (as calculated and retrieved from TWS). I don't know how IBKR calculates this amount because this is the reason that I had originally dropped it from the new TradeTracker.
• Second column. The current market value of the option and how much you would have to pay to close them.
• Third column. A simple difference between the first two columns.
• Fourth column. Percentage for value left in the leg. Even though the first column can vary, knowing the percentage of value left in the leg is a good approximation for deciding whether to close the leg (say, if your 50% profit goal has been reached).
  I am not sure why in the first column one of your legs is green and the other is red. That is a display bug. Both should be a neutral gray colour. I'll fix that.

[PaulSquires]

Actually, it is still in profit.

Looks like the current market value of your Put legs would cost you $2851.02 each to close (total $5702.04). Your current income received on the trade is $3721.30, so it appears you are not currently profitable for this trade(?)

[Flippo24]

Last close is 7.00$
4 * 100 *7 = 2800.
so I need 2800 to close the whole position. Not just one leg.

[PaulSquires]

Ah, okay, I see what the problem is now. Both option legs are exactly the same.

The data retrieved from TWS is getting the full 4 positions whereas in TradeTracker you have them split into two different transactions and when I loop the grid, I (incorrectly) display the TWS 4 contract value for both of your legs.

I need to fix it so that duplicate option legs are combined into one.

[Flippo24]

Please use the values entered in the transaction as the starting point. Please do not divide the value of the TWS by the number of legs.

[PaulSquires]

I will combine both of the legs so that it shows in the grid as -4 Feb 16 42d 65 P. That way it will match what shows in TWS and also match the value that TWS sends to TradeTracker.

[Flippo24]

No, please don't do that. Because you can look at both positions separately. Each has its own PnL.
You also do this separately in your strategy.

[Flippo24]

Take the value from the transaction for the first column. In my opinion, it is not necessary to take the value from the TWS.
If you were to combine it, then the same would appear in the line as in the line with the underlying. It would make even less sense to see the same thing twice.

[PaulSquires]

Maybe I am not understanding correctly. Maybe more talk about this may help.

The first column is calculated from incoming TWS data:
position_cost = (pd.average_cost * ld->leg->open_quantity); // pd.average comes from TWS and is cost per 1 PUT

Instead of using that data, I could use the two values that show in the Trade History for your legs (the values that you had manually entered: 1920.70 and 1800.60). I like that idea.

It is the second column that I was mostly referring to previously. That amount is not calculated by me, and is simply the amount that comes in from TWS. The problem is that the amount is the total of all 4 of your Puts and not just 2. To be correct, I would have to take the 2851.02, divide it by 2, and display 1425.51 for each leg in column 2.

Does that sound right?

[Flippo24]

For column one:
The value of the transaction history is completely ok.
The background is this:
Sell 1 put for 10$ premium if you want to buy it back for 50% it would be 5$.
If you sell the same put again during the term, say for 6$ and want to buy it back at 50%, that would be 3$. So both separately.
If you combine them now, 50% would be $4.

For the second column:
Just calculate it yourself.
I prefer to use the TWS model price in my spreadsheet. You certainly won't get it.
You could calculate the last close * position. You also have the multiplier in the transaction.
If you want to make it even more precise, then take the ask price for puts/calls sold and the bid price for puts/calls bought. This would have the advantage that a price is available even if there is little trading activity.

[PaulSquires]

Added new code to calculate the cost basis of individual Option legs. Needs to be tested on new data because the calculation can not be applied to transactions created before this patch. I will see how well this code works this week in a live setting when connected to TWS. Actually, older existing Trades may display correct data UNLESS they contain rolled transactions. In that case, the calculation will be incorrect.

[Flippo24]

The value for column 1 is now correct. Great so far.
But I don't understand why you always calculate the other columns from the total position and not row by row?
The value for column 2 is again the value for the entire position.
Why is it not the current price * the number in this row times the multiplier?
The value in column 3 also seems to come from the total position. Please switch to line-by-line calculation here as well.
The profit in column 4 is also clearly wrong. It is somewhere between 21-26%. It depends on which price you take from the TWS for the calculation.

[PaulSquires]

Yes, I see the problem and I have a solution for it. I just started working on the fix but I only have it about 50% finished. I will finish it today and post again here when the code is done. Thanks!

[PaulSquires]

I uploaded new code for the live data grid. I changed where some column data is displayed as well as use better data for the individual line calculations. I have not tested it a 100% yet but hopefully I will have time today to test.

[Flippo24]

That's what I wanted to see. Thanks a lot.

However, I still have one problem. But this has nothing to do with your current change. Can you have a look at why no values are displayed for these items?

[Flippo24]

The width of the fourth column could be slightly larger. Maybe an automatic adjustment of the width across all values of the column makes sense, in combination with a minimum width.
What about a space between the value and the %? Does that make sense? Could be my eyes are not the best anymore.
Do the brackets make sense at all or can you leave them out?

[Flippo24]

If the brackets only show that the position is in loss, then I would prefer a "-" in front of the value. I think the red color is right.

[PaulSquires]

The width of the fourth column could be slightly larger. Maybe an automatic adjustment of the width across all values of the column makes sense, in combination with a minimum width. What about a space between the value and the %? Does that make sense? Could be my eyes are not the best anymore. Do the brackets make sense at all or can you leave them out?

Thanks, I will look at this tomorrow. There is code that should automatically resize the columns contents based on min/max widths. I will check to ensure that code is being applied to that fourth column. I should be able to change from "( )" to negative signs.

I haven't had much time to day to test or code anything so hopefully tonight and tomorrow I can get things correct.

[Flippo24]

I don't expect you to solve things for me immediately. I'm already quite happy with your work.

Speaking of work. I have something else for you...
When I pull data from Yahoo, it only shows the price of the underlying and that the position is ITM. But all other values are missing. No values for the legs and the total PnL.

[PaulSquires]

Here is what it looks like with the minus sign and space before the %.

Here is what it looks like with ZERO decimal places instead of TWO.

Do you have a preference??? (I just committed the source using MINUS sign, SPACE before %, and ZERO decimal places if you
want to see it for yourself.

I have a few other areas in the program that use parenthesis instead of minus. I should just change everything to minus, right?

[Flippo24]

My favorite would be the second one, without the decimal points and with the space. It looks cleaner.

Your TradeTracker is getting better every day!

[PaulSquires]

My favorite would be the second one, without the decimal points and with the space. It looks cleaner.

I think that I like that version also. 😀

Your TradeTracker is getting better every day!

Thanks! I am very happy with all of the suggestions you've provided to make this program better. It has come a long way in just a week or so. 👍

[Flippo24]

You should have asked for my help earlier. 😆

[PaulSquires]

I have to go out for an hour! Damn.
I will dig deeper into your posts as soon as I get back. Hang in there, we'll figure it all out.

[Flippo24]

Lol. You seem to be there with heart and soul.

[PaulSquires]

I don't know why IBKR is not correcting the TMF1 -20 PUT 2024-01-19 $6 ---> TMF -2 PUT 2024-01-29 $60 when it sends you the data. Likewise the -40 TMF1 and -2 TMF 2024-02-16 $60 are not being combined. Granted, they are separate symbols in IBKR's eyes (TMF1 and TMF).

I wish I could explain why IBKR handles the data the way that they do for this scenario.

I wonder if you enter the Trades as follows would they then receive the TWS data:

TMF1 -40 PUT 2024-02-16 @ $6
TMF1 -20 PUT 2024-01-19 @ $6

TMF -2 PUT 2024-02-16 @ $60

I have done some online searches to see if this kind of situation has been explained but I have not found anything relevant.

[Flippo24]

That doesn't work either.

I once entered a TSLA position which I do not have. I don't get any data there either.

The TSLA positon is not displayed in the console. It looks as if only the tickers from the IB portfolio appear in the console, not those from TradeTracker.

[Flippo24]

Could it be that you only subscribe to the tickers that are in the IB portfolio and not those from TradeTracker?

[PaulSquires]

TradeTracker can get the Ticker data for whatever symbol that you set up for in the Trade.

However, TradeTracker will only get the portfolio/leg/option data for positions that you actually own in IB. This is because your entire Portfolio in IB sent to TradeTracker when the program starts and TradeTracker matches the incoming ContractID's from the TWS data to Trades that you have set up manually locally. Once the match has been made then TradeTracker knows which Contract ID's to capture whenever subsequent TWS data related to your open positions arrives.

This is part of the matching code logic that TradeTracker uses internally to match IBKR data to your local Trade data:

// ========================================================================================
// Test if IBKR and LOCAL position are equal
// ========================================================================================
bool Reconcile_ArePositionsEqual(positionStruct ibkr, positionStruct local) {
	if (ibkr.underlying == L"OPT" ||
		ibkr.underlying == L"FOP") {
		if (ibkr.strike_price == local.strike_price &&
			ibkr.open_quantity == local.open_quantity &&
			ibkr.ticker_symbol == local.ticker_symbol &&
			ibkr.expiry_date == local.expiry_date &&
			ibkr.put_call == local.put_call &&
			ibkr.underlying == local.underlying) {
			return true;
		}
	}
	if (ibkr.underlying == L"STK" ||
		ibkr.underlying == L"FUT") {
		if (ibkr.open_quantity == local.open_quantity &&
			ibkr.ticker_symbol == local.ticker_symbol &&
			ibkr.underlying == local.underlying) {
			return true;
		}
	}

	return false;
}

[PaulSquires]

That doesn't work either.

I am thinking that "TMF1" is a ticker that will no longer generate data because it is superseded by the data in "TMF". I honestly do not know for sure. This whole stock split and how IBKR deals with the handling of data related to pre and post split is all new to me.

[Flippo24]

However, TradeTracker will only get the portfolio/leg/option data for positions that you actually own in IB.

This must be changed.
In the current case, IB is the master with its portfolio as the database.
I think it is better if TradeTracker is the master.
Query the ContractID for all positions in TradeTracker and then subscribe to the data. No matter what is in the portfolio.

[PaulSquires]

I understand what you mean and I am pretty sure that I can query IB for contracts based on stock, option, futures, attributes. I would just have to test to see what streaming data gets received by TradeTracker for all of those different types of underlyings. Actually, since we changed the way we calculate the position cost and display it ActiveTrades grid, the data that we really need on a real time basis from IB would be the current market price.

I could then continue to use the Reconcile function to retrieve IB based Portfolio data and compare to local data to ensure that you have not entered local data incorrectly causing a mismatch to what positions you actually own.

I guess my only question is: Why would you want to set up a Trade locally within TradeTracker that you do not actually own based on the data in IB???? I am trying to think of a scenario where that would be useful?

I am certainly open to changing the way TradeTracker requests information but I would have to do some testing first to make sure that we are not losing any functionality that we currently have.

[PaulSquires]

I once entered a TSLA position which I do not have. I don't get any data there either.

The TSLA positon is not displayed in the console. It looks as if only the tickers from the IB portfolio appear in the console, not those from TradeTracker.

This is exactly what I was trying to ask in my last post. Why would you set up the TSLA trade in TradeTracker and want data for the Jan 250 Put if you do not own the position? It seems confusing to me to be tracking trades for positions that I don't own. Hopefully you have a use case reason that you can share! Thanks!

[PaulSquires]

I have tried your new approach but (so far) I am unable to retrieve option data for stocks or futures. It could be because the market is closed but I will try again tomorrow during regular trading hours. (I am using the reqMktData api).

[Flippo24]

I have tried your new approach but (so far) I am unable to retrieve option data for stocks or futures. It could be because the market is closed but I will try again tomorrow during regular trading hours. (I am using the reqMktData api).

Before you subscribe to the ticker, you could retrieve the last price once. Even if no trading takes place, you will then have the last valid price.

I don't know many cases where you can use this. You saw one in my TMF case. You could also create fictitious positions and delete them again so that they don't appear in the report if someone wants to compare two positions. One that he has traded and one that he had as a second choice. Maybe someone has a strategy where they only enter a trade when an option they are watching has fallen 50% to open a real position.

[Flippo24]

I'm wondering a few things here.

Why are the two values marked in green the same if they are different strike prices? I saw this in your pictures and not just for one trade.

The 756.82 marked in red should be negative. Since you get money when you sell the position. If it goes according to the money flow.
I don't understand the calculation of the profit. You paid 2450.71$ for this put. This is still worth 756.82$. Then the profit should be -1693.89$ and -69.11%.

[Flippo24]

I find the way you use the income and expenditure with the signs very difficult to interpret. It may be that the way you do it is right, but I would always do it from the point of view of my account.
When I sell something, I get money for it. Positive.
If I sell a put, I get a positive premium. If I have to buy it back, that would be negative because I have to spend money.
The statistics under Closed Trade are also positive for profits and not negative. Think about it.

[PaulSquires]

Good morning, the first column is always the money you have received for the position. The second column is the current market value of the position if you sold it right now. The 3rd and 4th columns are the difference and percentage.

It looks like that screenshot you posted is from 3 days ago and there have been fixes to the calculations since then. Here is what that exact same position looks like now:

The first column is calculated by assigning the total amount of cash received for the trade ($756.82) among the 3 different legs based on their percentage of contracts. That may not be the best way to do it but I am not sure what the alternative would be? I agree that I should not be playing with the negatives and I will fix that.

Here is what the costs of that Trade is showing right now:

And here is the market value of the Trade:

It has been the calculation and display of the individual leg positions that has caused us problems over the past week. I wonder if it would be an idea to receive the TWS reported costs of the 3 legs, total them, and then use each of their proportions to multiply against the amount received. That might be okay? Using that approach I would not have had to gone the path of tracking individual leg costs from transaction to transaction.

I think more thinking and head scratching might be needed here. 😁

[Flippo24]

I'm really sorry if I keep coming back to the point. But actually I expect exactly the same values in the TradeTracker as in the TWS. The profit of 611% is simply misleading.

What would be wrong with changing the order management window so that you enter the buy price for each leg instead of a total price? That would be the most accurate. All values can be seen in the TWS after the order has been executed.

[Flippo24]

Another total field for price and fee below the input fields. But only as a display.

[PaulSquires]

I like your ability to photoshop a new screen layout design 👍

[PaulSquires]

How about something like this. This would make it a lot easier when calculating the cost basis based on incoming TWS data:

[Flippo24]

I don't know what sense a ratio should make at this point. If a leg costs a certain price, then it costs exactly that price and not an adjusted one via a ratio.

[Flippo24]

Why isn't it easier this way?


You can also find the values from the TWS here!

[PaulSquires]

In the previous exampe there were no Rolls. This example, using a simple /ES Strangle has several rolls and as you can see, how do we equate the TWS reported cost values (-7869) to the ongoing profit/loss that we have manually recorded (3007.96)? That's why I thought that using a ratio would work.

[Flippo24]

I find this very difficult and complicated.
I don't know exactly how much you paid for rolling to buy back your positions.

I think the cost basis is 7869 - 3007.96.
But again, you can always look at both legs separately, as both legs were rolled individually. Which again means that you don't need a ratio.
Take the transactions and look at the whole thing as if the rolling is the closing of the trade and afterwards you open a new trade. What kind of position do you end up with? I think the one shown in the TWS. In the end it's a philosophy question, is the roll part of the trade or are there two trades.

[PaulSquires]

I find this very difficult and complicated.

I do also. 😁

I think the cost basis is 7869 - 3007.96.

I would think that the cost basis would be 3007.96 because that is how much money we received by selling the Puts and subsequent incremental income from rolling. When I am evaluating my position, I want to see how much the position is worth at the current market value, versus how much income I have currently collected. It is the allocation of the costs to the legs that is problematic.

In the end it's a philosophy question, is the roll part of the trade or are there two trades.

I agree that it is two trades and IB treats them that way (as do all brokers) because when you roll you'll see the realized gain/loss immediately. Of course the problem is that users want to treat all the rolls as part of a single trade.

I think I need more thinking.... maybe I will work on something else and let this sit in my subconscious for a while and wait for that "Eureka Moment" to hit me. 😀

[PaulSquires]

Okay, I've done more thinking about this. The fundamental problem is getting an accurate starting cost for the legs. Once we have that value then all subsequent transactions involving adding or reducing costing to the legs is easy, and that code is already part of TradeTracker.

A few days ago, to calculate that starting point, I included programming logic to simply use the ratio of contracts of the leg to the total contracts of the transaction and multiply that by the income received for the entire transaction. Of course, that is not accurate as some of the long and short legs and put/call skew would value the legs differently.

An option would be that when the initial transaction is created, set the legs cost basis to zero, and respond to the first incoming TWS message for that Trade. That message would have the true initial costs of the legs. I would simply test to see that the leg's cost is currently zero and if it is, then assign the incoming TWS cost data.

The problem with the above is that it assumes that the user has only entered that first transaction and no other in the same trade. If the user is NOT connected to TWS and enters several transactions for the trade and then later connects to TWS, then I would never get that initial cost data (I would only get the data for most recent transaction). This is the same problem that we are facing right now. I have trades with many transactions and no way to get the initial starting cost.

I do not want to go down the path of making the user try to find this cost data for themselves. Maybe there is a IB report that can be run that shows the evolution of the costs but once I force users to have to generate reports just to enter data, then I bet 99% of them will give up and abandon TradeTracker because it will become a hassle. The whole idea of this program from the very beginning was to make things easy. 😁

So, once again, I come back to the ratio approach...... or, should I say, "relative cost base" of the leg. Every time I receive a TWS data message about the Trade, I can calculate the relative costing of the all the legs compared to the overall income received to date for the Trade.

Is it perfect? Maybe not.

Is it close to what the actual costs would be? Maybe.

Another benefit of using this relative cost base approach is that it is easy to implement and willl easily work with existing data and the data of previous TradeTracker (IB-Tracker) databases. Previous users would simply need to start using TradeTracker and everything would work seamlessly under the hood.

Anyway, I will continue to do more thinking.....

[Flippo24]

I can't quite share your approach of extracting as much data as possible from the TWS and doing "too much" work for the user. As long as everything works, it's ok. But if other numbers appear and the user does not understand transparently where they come from or what is behind them, it becomes difficult. Maybe I'm the wrong person to ask and it would be interesting to hear other opinions.

[PaulSquires]

I have introduced a GPF when closing a Trade when Live Data connected to TWS. I hope to have this fixed ASAP.

[PaulSquires]

I have introduced a GPF when closing a Trade when Live Data connected to TWS. I hope to have this fixed ASAP.

Okay, should be fixed now.

[PaulSquires]

Finally, I have have the live Option Leg portfolio data matching the streaming data from TWS. Turned out it was a simple and straight forward implementation. Too bad I wasted so much time on this issue running around in circles. Code is now in the repository and an updated testing EXE posted to https://github.com/PaulSquires/TradeTracker/releases