Feature Request: Forex exchange implementation using Meta Trader 4

[notadamking]

Funding:
1.75 ETH (~$450 USD)

Description:
Create an implementation of Exchange capable of trading any forex pair on the MetaTrader 4 (MT4) platform.

Requirements:

• The exchange implementation must be completely interoperable with MT4, including spinning up any necessary ZeroMQ infrastructure.
• The exchange must fully implement the Exchange class, including the ability to list balance, net worth, asset valuations, trade history, etc.
• The exchange must be capable of returning observations from OHLCV, indicators, signals, and other data from the MT4 platform.
• The exchange must be capable of executing live trades on the MT4 platform, with either live or demo accounts.
• Must include an example with an agent trading using a demo account.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 1.0 ETH (184.42 USD @ $184.42/ETH) attached to it as part of the tensortrade-org fund.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $127,235.38 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 1 month, 1 week ago.
Please review their action plans below:

1) adivyas99 has been approved to start work.

I have good experience in machine learning, Python and Reinforcement Learning. Pls, allow me to work on this project. Thanks.

Learn more on the Gitcoin Issue Details page.

[gitcoinbot]

⚡️ A tip worth 0.05000 ETH (9.47 USD @ $189.39/ETH) has been granted to @CoeJoder for this issue from @notadamking. ⚡️

Nice work @CoeJoder! Your tip has automatically been deposited in the ETH address we have on file.

• $128628.08 in Funded OSS Work Available at: https://gitcoin.co/explorer
• Incentivize contributions to your repo: Send a Tip or Fund a PR
• No Email? Get help on the Gitcoin Slack

[sleekmike]

Hello any progress here?

[CoeJoder]

I've gone through the docs and the tutorial to familiarize myself with the application and requisite concepts, and am beginning my implementation of this Exchange class now.

[CoeJoder]

@notadamking The requirement is for MT4 compatibility, however the company no longer provides an installer for MT4. Support threads on this topic suggest that you can only obtain MT4 through a broker, eg:
https://www.mql5.com/en/forum/216552

Do you have an official installer for MT4 you could provide, or can this requirement be changed for MT5 compatibility?
EDIT: I was able to get a copy of MT4 by signing up for a demo account through forex.com.

[CoeJoder]

Update: I have the basic features working. It took longer than expected because I had to write additional MQL4 code than what was provided by the 3rd party connector library. Still working on it and will provide updates on progress this week as I have time.

[sleekmike]

ok this is great!!! keep the good work mate, I am really looking forward to the update on the progress.

[CoeJoder]

@notadamking Almost done. I've created a nice API for all the data pulled from MT4, and have methods implemented which pull the data and makes trades as required. However there's a conceptual gap between crypto and forex trading, so I need clarification on what to return for a few Exchange properties:

Exchange::portfolio()
A forex account is always denominated in the account's currency. You get exposure to a symbol during the lifetime of a trade, but the balance is always finalized in the account currency when the trade is closed. My understanding is that a forex portfolio should include the asset exposure of all open trades, but the usage of portfolio() by ActionScheme components suggests that only the free margin (the amount available for making new trades) of the account balance should be included. Awaiting your input on this one.

Exchange::trades()
Given the forex trade lifecycle, which kinds of trades should be included?

• pending (no margin committed yet)
• open (buy/sell limit met, margin committed)
• closed (stop-loss/take-profit met, or trade expired)

Exchange::balance()
I'm currently returning the account's free margin, based on how this property is being used throughout the application.

Exchange::net_worth()
I'm currently returning the account equity (the account balance plus or minus the floating profit/loss from any open trades).

[notadamking]

Hi @CoeJoder,

Thanks for your contribution! I have been asked to pass some information along:

[CoeJoder]

Thank you for the response. I might be misinterpreting it, but I don't think it answered my question regarding the contract of the Exchange class. It looks like an update to the requirement. That is, instead of delivering an implementation of the Exchange class, there will be three classes: Trades, Balance and Margin which will act as proxies for the MetaTrader terminal, providing all of the interactivity as described in the screenshot (minus the last paragraph as this is out of scope) and the original bounty requirements. Please let me know if this is correct.

Other than figuring out how you want this packaged, it's pretty much done.

[sleekmike]

Hello CoeJoder,
have you any progress or update on the issue that you are working on so we can try it?
also why are you not on the discord group?

[CoeJoder]

@sleekmike Hi Mike. I have joined the discord group now. Currently I have a Python API for creating new market & pending orders, closing/deleting orders, checking all account properties (balance, margin, etc), checking all symbol properties (latest tick, fetching OHLCV bars), calling any of the built-in indicator functions, and fetching data from all available signals. These are demonstrated with a few dozen unit tests. I also have a partial implementation of Exchange but ran into the problem described in my previous messages. I can push this code to a forked branch if you are interested.

The last reply I received 8 days ago from @notadamking seems to suggest I abandon the Exchange implementation in favor of a MT4-driven "block/allow" paradigm. That is, a standalone Python class rather than an Exchange. If someone can confirm this is the new requirement to satisfy the bounty, I will have it ready soon.

[sleekmike]

Ok great! what's your discord id or name? I want us to collaborate on this issue.

[CoeJoder]

Sounds good. Name is same as here, CoeJoder.

[TheSnowGuru]

Hey CoeJoder,
I am sorry if I confused you with my request. let's make things simpler . this is important that users will be able to easily understand what is inside a class. and it has to make sense.
Find my request inline with your comments.
Moreover,
We have to divide the work with mt4 into 2, inputs & outputs.
in regards to the classes you mentioned:

Inputs to TT from from the MT4 :

We get a lot of data , we need to have it all inside some kind of array that sums it all up, this should include. but some should go to state and some to the environment in TT. correct me if I am wrong.

Exchange::portfolio() - **should return: balance, equity, margins, free margins, , margin level , which are related to the account exposures and appears in the trade tab in mt4- the name here is Fine.
those should go mapped into state in TT.... correct me if I am wrong.

Exchange::trades() - ** Should include all the columns in the trade tab, which includes: order id, time, type, size, symbol, price, sl, tp, price, commission, swap, profit, comment.

• pending (no margin committed yet) - OK.
• open (buy/sell limit met, margin committed) - OK.
• closed (stop-loss/take-profit met, or trade expired) - this should not be included here, we need
  Another class called TradeHistory which will include all historical trades.

Exchange::balance() - I think it's not necessary as we have it in portfolio
So can be deleted. what do you think?

Exchange::net_worth() I think it's not necessary as we have it in portfolio
I would change this to Exchange::exposure () http://prntscr.com/qcfc74 and give the agent an over look of it's overall account exposures this way: asset, volume, rate, usd

Sending & TMF - Trade Management functions.** Trades to MT4:

This was not discussed throughout the issue, and I am sorry I wasn't so clear about it, but it's crucial to the success of the implementation.

When sending trades to MT4, we have few types of new trades:
buy/sell market, buy/sell stop, buy/sell limit, - are they all supported?

When we have open trades, we have few options that should be supported with TT & MT4. and all this relates to TMF - Trade Management functions.
FullOrder Close, & Partial Close - can we support that inside the actions space?

TMF should return values back to agent TT and the agent should interact with MT4 in the following way: will be able to modify an open orders/open trades:
0 - check the trade's Entry, Stop, TakeProfit, and Trail parameters.
1 - Update an openOrder or openTrade with exitNow , CancelOrder , Trail, TP ,SL

Hope that's clearer now.

Cheers,

[TheSnowGuru]

Small note, most FX brokers nowdays supports more assets like CFD's , e.g gold, silver, crude oil, indices and more, it's important that the agent will be able to engage with those as well, and not just fx pairs.

these are the parameters and characteristics these assets have:
Name,Price,Spread,RollLong,RollShort,PIP,PIPCost,MarginCost,Leverage,LotAmount,Commission,Symbol
thx

[CoeJoder]

@TheSnowGuru Thank you for the very detailed response. The Exchange methods pass their return values to other components in the TradingEnvironment. Since these return values will be of a different type than the usual, it will not be compatible with any of the existing ActionScheme implementations (DiscreteActions, ContinuousActions, MultiDiscreteActions). I think this is as far as the incompatibility extends; all other components should be reusable as they operate using standard data frames. I will provide a very simple ActionScheme implementation to work with this Exchange so that it can perform an agent run.

Regarding output/TMFs, yes all of the features you describe are already implemented, except for modification of sl/tp & trail (but I will add this now) and will be accessible in the ActionScheme.

Regarding exotic symbols, this should be fine. However I'm not seeing the same parameters and characteristics that you've listed (see below). Where are you seeing those fields? The values returned by the MQL4 programming language are standard for all symbols so I am unaware of those fields.

[CoeJoder]

@notadamking @TheSnowGuru @sleekmike I'd like to discuss increasing the bounty prize on this to 2 ETH. This has been a lot of work! ETH also dropped in price by a lot since this started.
Some stats:

• 1300 lines of MQL4 code
• 2000+ lines of Python including tests

[sleekmike]

Ok @CoeJoder I will get back to you on this soon!

[CoeJoder]

I was notified directly by the bounty sponsor that he and his partner are no longer interested in this feature, therefore I will stop work on it.

[sleekmike]

Hello CoeJoder, your bounty will be paid shortly, we are still interested in the feature so I suggest you continue working on it. Thanks

[sleekmike]

Hello CoeJoder!

[sleekmike]

Hello @CoeJoder,
The issue has been funded with additional 1 ETH making it 2ETH just like you demanded. Please provide updates and commits on the issue.
Thanks

[CoeJoder]

@sleekmike No, the bounty still shows 1ETH: https://gitcoin.co/issue/notadamking/tensortrade/97/3678
Also, I'm not demanding anything; it's an open bounty. 2 ETH is a very generous bid by me. For comparison, I made 12.5 ETH on my previous bounty which was much easier and smaller to do. I'm only making $5/hour on this ticket, and honestly, that's not worth it either. I provided a detailed explanation of why this bounty has ballooned into somewhat of a beast. I'm already working on other things now, so you would have to pay me more competitively to work on it. Otherwise, good luck in attracting another bounty hunter for such a low reward.

Please read the gitcoin guidelines on how to price a bounty effectively: https://gitcoin.co/blog/tutorial-how-to-price-work-on-gitcoin/

[sleekmike]

Hello CoeJoder,
It really took you time to respond or reply, I was really worried. The bounty has been funded, please be patient you will see it soon, I was billed on the 6th of this month but there is a little technical issue that would be resolved soon and it will reflect. Also are you keeping abreast of the recent update on TensorTrade? Also you should join the discord group so that you will be more informed.
Best regards.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of this issue was increased to 1.75 ETH (446.81 USD @ $255.32/ETH) .

• If you want to claim the bounty you can do so here
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $82,470.99 more funded OSS Work available on the Gitcoin Issue Explorer

[sleekmike]

Hello Cojoeder

[delaji-san]

I've done years worth of work on MT4 previously and could take a look at this if:

• You get a working model of this software. Yes I have spent a while trying to get this working and tried the previous releases.
• There's correct documentation on how it all works. Yes I have looked through the docs and examples.

Here's the caveats:

• You won't be able to pass back indicators from MT4 other than the standard ones which are already available in TA. Most Custom Indicators do not allow reading.
• You will have to factor in swaps and using leverage with lots instead of just amounts.
• The responses you are looking for in CCXT will not be similar to MT4 so you will need a rewrite of the back end.

What I can do:

• Buy/Sell orders
• Stream market data into Python
• Check available balances/equity
• Check orders

Could take a look at this soon if you can get me a working version with documentation. It will probably take a fair few days to implement and test.

[sleekmike]

Hello @delaji-san, thanks for showing interests on this issue and would be really glad to if you could implement the Buy/Sell orders, check orders and other things that you mentioned. I can give you a working version tensortrade framework that I use to playground with and develop, I can also help you with documentation and guidance in setting this framework on your PC. Thanks

[TheSnowGuru]

If it would be easier for you to implement using MT5 and not MT4 , than go for it. buy sell is 6 types: but limit sell limit buy market sell market sell stop and lastly buy stop thanks

…

On Sun, Apr 5, 2020 at 11:23 AM Mike Ohanu ***@***.***> wrote: Hello @delaji-san <https://github.com/delaji-san>, thanks for showing interests on this issue and would be really glad to if you could implement the Buy/Sell orders, check orders and other things that you mentioned. I can give you a working version tensortrade framework that I use to playground with and develop, I can also help you with documentation and guidance in setting this framework on your PC. Thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#97 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIRHQZDGHI2C76NT7IWD7TRLA5ZVANCNFSM4JLKLMYQ> .

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This Bounty has been completed.

Additional Tips for this Bounty:

• notadamking tipped 0.0500 ETH worth 11.91 USD to coejoder.

---

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Chat
• $326,435.49 more funded OSS Work available on the Gitcoin Issue Explorer