[Logic] Store trade and transfer history for further use

[MarkoPaasila]

To prepare for profit calculation (#71), other displayed statistics, and trade history export (#80); store all trades and transfers.

I can imagine it might be a small challenge to get data of transactions that occurred during offline, but that is as important.

[lsji07]

This would be useful to format as required by Cointracking.info for automatic csv file importing to speed up analysis and increase bot adoption.

I understand their CSV is currently formatted as follows for importing.

"Type","Buy","Cur.","Sell","Cur.","Fee","Cur.","Exchange","Group","Comment","Date"

This would only be to review and identify bot trades as other historical trades made manually on the dex for the account would not be easily accessible from most dex nodes due to RAM limitations.

[MarkoPaasila]

Are there other similar services / software that analyze trade history? If so, it would be best to do some research and see if there is an industry standard format, and use that. Otherwise this seems like a good suggestion.

[the-scott-davis]

The problem is there is no standard. Cointracking.info (mentioned above) has a lot of users and allows for custom CSV import. However, there are 10 other sites that have the same functionality (all with different date format requirements).

However, starting with support for Cointracking.info, would be a great start. I find myself exporting my trade history from this site, which supports exporting to Cointracking.info format: http://www.cryptobridgestats.com/trade-history

[lsji07]

I have used the same site to get a csv downloaded for my trades. I would like the dexbot to provide the trade information directly. The link is limited to download 10000 trades due to RAM server limitations. The dexbot can save trades for a worker for far longer.

Other sites are out there but not free. Cointracking is good as it is not limited in terms of trade analysis uploaded etc to my knowledge. The only time a paid account becomes "required" is with their tax reporting reports for the US etc.

Not found anything else comparable use after a fairly thorough search.

I am all for alternatives though so I think the format of the CSV export should be selectable for additional formats in future. Possibly a dropdown selection etc to select how you want the information exported or saved in the CSV when a strategy is configured. Ideally by worker to make analysis easier.

[the-scott-davis]

Maybe we just need a CLI tool, or a separate script, to convert the history.csv to a cointracking.info friendly csv format to be run as needed. I haven’t checked to see if all needed info is in the history file, but if it is, I’ll make a util to handle this.

[MarkoPaasila]

DEXBot doesn't record the fee's in the trades.csv, but they are already deducted from the amounts.

Changing the format shouldn't be too hard, but it's low priority. Until someone does it we can simply do the conversion in a spreadsheet. I'll make an issue for changing the format, and someone will pick it up some beautiful day.

[bitphage]

Trying to record trade history without accessing account history may produce wrong results. You cannot say for sure was order filled or expired. Also imagine you're resetting orders on partial fill. There are some time between you checking order, writing a trade log entry and cancelling order to make a new one. During this small gap your order may be filled further and your log entry will be incorrect. This may be a rare condition, but when it will happen it will screw all tracking. This is named a "race condition".

How to obtain accurate results? Write an account history analyzer.

How to track profit/loss simply and accurate? Just store account balances changes! This is pretty easy and accurate.

[MarkoPaasila]

The main purpose of DEXBot is to make markets, hopefully for a profit. Trade analysis isn't the purpose, but it's still necessary and crucial for understanding strategy performance. Now it looks like we must build some kind of a reasoner/analyzer to get accurate information, and even with that we would have limited information from offline-time. The proper solution seems to be coming with elasticsearch, but it might take a while before we can access it easily using our python libraries.

Until that, it seems that profit estimation should be based on the asset total balances/values. And if trade history is inaccurate, I don't know what it's good for.

[bitphage]

To provide best user experience, I would like to propose implementing built-in trading and transfer history collector and analyzer. It's data can be used for creating profit/loss reports or even to use in strategy logic.

I already did a separate tool https://github.com/bitfag/bitshares-tradehistory-analyzer , so it can be implemented as built-in and be more specific.

[bitphage]

Agreed with @PermieBTS to implement this feature during this week.