Python-based client for the new socket-based API #225

owocki · 2017-10-30T21:38:06Z

Requirements:

Write a Python-based reference client for the new socket-based API
Must have feature parity with https://github.com/etherdelta/etherdelta.github.io/tree/master/bots

gitcoinbot · 2017-10-31T12:45:43Z

This issue now has a funding of 0.55 ETH (169.56 USDT) attached to it. To view or claim this funding, click here.

owocki · 2017-11-01T15:18:15Z

I see that an AnonymousUser claimed this issue this morning. To whomever you are, please submit your code within the next several days (lets say, by Sunday). If we don't hear back from you, I'll assume you're not serious about turning around the issue and will reject the claim.

bennyprofane · 2017-11-02T00:37:22Z

I appreciate you posting a bounty and I hope someone is able to come up with something worthy of claiming it. Right now for me it's just driving me crazy that I can't get this api to work in python. Here's the best I've been able to come up with:

import websocket

def on_message(ws, message):
    print(message)

def on_error(ws, error):
    print(error)

def on_close(ws):
    print("### closed ###")


if __name__ == "__main__":
    websocket.enableTrace(True)
    ws = websocket.WebSocketApp(
        "wss://socket.etherdelta.com/socket.io/?transport=websocket",
                              on_message = on_message,
                              on_error = on_error,
                              on_close = on_close)
    ws.run_forever()

Note that I'm running this in python 3.5.3.

The above code makes a connection to some kind of data stream but it's not what I want. What I want is to be able to send the getMarket message like the api documentation says you can and then receive the market response. With the above code you should in theory be able to send the message with ws.send() but in practice it doesn't work when I try it. I'm hoping this code might help someone else find a proper solution.

Also note that the api says to use https://socket.etherdelta.com but I have not been able to make that work. The only thing that has worked at all for me is the wss://socket.etherdelta.com/socket.io/?transport=websocket address which I found listed in other issues.

EDIT: I removed import json and import _thread from the above code as they are not needed for this simple connection example and were imports I was using in my failed attempts.

EDIT: Also I forgot to add that I'm using websocket-client version 0.44.0. Install it with pip install websocket-client. This is a link to the github for it https://github.com/websocket-client/websocket-client

owocki · 2017-11-02T03:08:55Z

I’m mobile right now so I can’t test. What do you observe when you run the above code? An exception? Does it just hang? Can take a more solid look after devcon

On Wed, Nov 1, 2017 at 7:37 PM benny profane ***@***.***> wrote: I appreciate you posting a bounty and I hope someone is able to come up with something worthy of claiming it. Right now for me it's just driving me crazy that I can't get this api to work in python. Here's the best I've been able to come up with: `import websocket import _thread import json def on_message(ws, message): print(message) def on_error(ws, error): print(error) def on_close(ws): print("### closed ###") if *name* == "*main*": websocket.enableTrace(True) ws = websocket.WebSocketApp( "wss://socket.etherdelta.com/socket.io/?transport=websocket", on_message = on_message, on_error = on_error, on_close = on_close) ws.run_forever()` Note that I'm running this in python 3.5.3. The above code makes a connection to some kind of data stream but it's not what I want. What I want is to be able to send the getMarket message like the api documentation says you can and then receive the market response. With the above code you should in theory be able to send the message with ws.send() but in practice it doesn't work when I try it. I'm hoping this code might help someone else find a proper solution. Also note that the api says to use https://socket.etherdelta.com but I have not been able to make that work. The only thing that has worked at all for me is the wss://socket.etherdelta.com/socket.io/?transport=websocket address which I found listed in other issues. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#225 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAfXiW_SdAXt1w5TxIbRg8ez40c7jAH0ks5syQ7EgaJpZM4QL5FX> .

-- - sent from an intelligent telephone

ghost · 2017-11-03T08:15:57Z

Since etherdelta.com runs behind CloudFlare, which AWS region would you suggest to run a test API client on?

Thanks!

zackcoburn · 2017-11-03T13:31:31Z

Us-east-1

On Fri, Nov 3, 2017 at 3:15 AM minibbjd ***@***.***> wrote: Since etherdelta.com runs behind CloudFlare, which AWS region would you suggest to run a test API client on? us-east-1 | US East (N. Virginia) us-east-2 | US East (Ohio) us-west-1 | US West (N. California) us-west-2 | US West (Oregon) ca-central-1 | Canada (Central) eu-west-1 | EU (Ireland) eu-central-1 | EU (Frankfurt) eu-west-2 | EU (London) ap-northeast-1 | Asia Pacific (Tokyo) ap-northeast-2 | Asia Pacific (Seoul) ap-southeast-1 | Asia Pacific (Singapore) ap-southeast-2 | Asia Pacific (Sydney) ap-south-1 | Asia Pacific (Mumbai) sa-east-1 | South America (São Paulo) Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#225 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APEqPZd4HN6sn1c5nINJHTe3ODEj2cIIks5sysu_gaJpZM4QL5FX> .

-- Zack Coburn NOTICE: This e-mail transmission (and/or the attachments accompanying it) may contain confidential information belonging to the sender which is protected by law. The information is intended only for the use of the intended recipient. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this information is strictly prohibited. Any unauthorized interception of this transmission is illegal. If you have received this transmission in error, please promptly notify the sender by reply e-mail, and then destroy all copies of the transmission.

owocki · 2017-11-05T16:55:00Z

@minibbjd @bennyprofane is one of you the user who posted the claim? i'm trying to decide whether to reject the claim or not, since it was done without context of what the intended solution was or who was doing the claim.

ghost · 2017-11-05T16:57:22Z

It wasn't me.

ghost · 2017-11-05T16:57:43Z

@owocki bennys solution is copy and paste from websocket client docs, and doesn't work as intended. the proper solution for a python API should implement the python socketIO client so that one can emit a message, and receive the correct response

bennyprofane · 2017-11-06T04:42:58Z

@owocki wasn't me either I am not trying to claim anything other than the fact that I can't get it to work the way I want it to. and @armehanh what I posted wasn't intended to be a solution. Perhaps you misunderstood my post. I only posted the code above to try to be helpful for someone else who might be able to figure out how to make it work properly. So if you have a working solution please post it otherwise your response is not helpful. I also tried socketIO but was unable to get it to work at all. With the code I posted above I was at least able to get something connected and get some kind of response from the server.

zheli · 2017-11-07T17:19:34Z

Is there a preference for the Python version? (2.x or 3.x)

zackcoburn · 2017-11-07T17:47:08Z

3.x preferred.

On Tue, Nov 7, 2017 at 12:19 PM Zhe Li ***@***.***> wrote: Is there a preference for which Python version you would like it to be in? (2.x or 3.x) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#225 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APEqPTtcWEgf9lUdkpUwm-pDr6G5g_YAks5s0JEngaJpZM4QL5FX> .

-- Zack Coburn NOTICE: This e-mail transmission (and/or the attachments accompanying it) may contain confidential information belonging to the sender which is protected by law. The information is intended only for the use of the intended recipient. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this information is strictly prohibited. Any unauthorized interception of this transmission is illegal. If you have received this transmission in error, please promptly notify the sender by reply e-mail, and then destroy all copies of the transmission.

mdevalck · 2017-11-16T14:13:17Z

Running into exactly the same issues using Python. Able to connect to a stream of data through wss://socket.etherdelta.com/socket.io/?transport=websocket but unable to push any 'get' commands. Indeed, the websocket-client module cannot connect to https://socket.etherdelta.com since it doesn't support https.

tomvanbraeckel · 2017-11-17T22:40:26Z

Done! I've taken up this issue as a little project.

I added lots of documentation, instructions and comments in the code and tried to keep it very easy and readable. Feedback welcome!

I'd be happy to send a merge request. Shall I add it in the /bots/ folder, as taker.py, next to taker.js?

zheli · 2017-11-18T08:57:18Z

What a pity! I was half way finished :(

ghost · 2017-11-18T09:07:45Z

Looks good.

Line 104: "new buy orders" should read "new sell orders"

If you comment he "while len(orders_sells) == 0" trading part, to have a forever updated order book, the socket will always disconnect after a minute or two.

Is there some ping/pong answering logic missing to keep the socket alive?

Also, an automatically reconnect, like in the node.js sample would be nice.

tomvanbraeckel · 2017-11-18T13:38:45Z

@zheli Sorry about that... I hope it was still a useful exercise. Feel free to post your code, regardless!

@minibbjd Thanks for your feedback! Ping/pong and reconnects are indeed not implemented because I wanted to have the same one-shot buy behavior that taker.js has. It might be as simple as changing the very last line to "ws.run_forever(ping_interval=15)" though :-)

tomvanbraeckel · 2017-11-18T20:23:29Z

Version 2.0 is out!

Same functionality as before (feature parity with the taker.js bot) but after doing the trade, it stays connected, with ping/pong and automatic reconnection. The client will keep listening for data and updating the order book and trade history as updates come in.

It goes into the /bots/ folder so that it finds the necessary contracts.

bellerophons-pegasus · 2017-11-18T20:28:26Z

Hi @tomvanbraeckel how did you figure out how the message to be sent should look like?
I was stuck trying all kinds of combinations as strings or in json format derived from the API documentation...

tomvanbraeckel · 2017-11-18T20:37:04Z

@bellerophons-pegasus Yeah, it also took me a while. But then I pressed F12 in Google Chrome to open the debugger, visited EtherDelta.com and in the Networking tab I could see the WebSocket requests passing by. WebSocket traffic only seemed to show up if I had the debugger open before loading the page though so that had me scratching my head at first!

bellerophons-pegasus · 2017-11-19T11:17:48Z

@tomvanbraeckel thanks! I was seeing the 42, but never had the idea it should also be sent.
Good work :-)

owocki · 2017-11-20T14:44:24Z

What a pity! I was half way finished :(

@zheli sorry about this. bad CX. i'm open to any feedback about how to make it clearer who is going to 'claim' a project to minimize double spending of time among participants.

@tomvanbraeckel thanks for owning this :) let me know when youre getting close to done and i'll ask zack if we're good to pay this out.

zackcoburn · 2017-11-20T20:07:32Z

Very nice. How about maker.py? :)

zheli · 2017-11-20T20:33:07Z

@owocki yeah, I am a bit worried when I pick new task now. Sometimes it can take longer time since I only do this in spare time. Plus I was actually making a Pythin client library for script to call so maybe it's a bit off scope :P

Maybe it's possible to have a stats showing how many people have started working on an issue with the start timestamp? Then I can make better decision next time :)

But I really like this bounty for github issue idea, I hope more open source projects start using it!

owocki · 2017-11-20T22:30:59Z

@zheli thanks for your thoughtful feedback. one standard i've seen folks adopt is leaving a comment to the effect of "hey all, i was thinking of claiming this bounty. let me know if someone is already working on it in the next 12 hours; otherwise i'll assume it's open"

i'm looking into a way of making the software faciliattate this on a go-forward basis.

zheli · 2017-11-21T09:28:18Z

@owocki ok, so I can just "claim" it when I start working on it? I was under the impression that you can only claim it after the work is almost finished. Thanks for the clarification!

tomvanbraeckel · 2017-11-21T15:55:20Z

@zackcoburn I'm reworking the whole example into a library now (similar to service.js) and adding the new maker.py :-)

zackcoburn · 2017-11-21T16:01:34Z

Awesome!

anlamak · 2017-11-24T21:33:14Z

Please can you help. I have tried V2.0 in multiple environments, using 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25), inc fresh Ubuntu 16.04 VM. Failing each time with:

`Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1596, in
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 974, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/Users/paulmattei/googledrive/PycharmProjects/node/main.py", line 352, in
balance = contractToken.call().balanceOf(userAccount)
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/contract.py", line 832, in call_contract_function
transaction=transaction,
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/utils/decorators.py", line 13, in _wrapper
return self.method(obj, *args, **kwargs)
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/contract.py", line 692, in _prepare_transaction
fn_kwargs,
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/eth_utils/string.py", line 85, in inner
return force_obj_to_text(fn(*args, **kwargs))
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/contract.py", line 731, in _encode_transaction_data
fn_name, args, kwargs,
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/contract.py", line 660, in _get_function_info
fn_abi = cls._find_matching_fn_abi(fn_name, args, kwargs)
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/contract.py", line 613, in _find_matching_fn_abi
function_candidates = filter_by_type('function', cls.abi)
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/utils/abi.py", line 43, in filter_by_type
return [abi for abi in contract_abi if abi['type'] == _type]
File "/Users/paulmattei/.virtualenvs/node/lib/python3.5/site-packages/web3/utils/abi.py", line 43, in
return [abi for abi in contract_abi if abi['type'] == _type]
TypeError: string indices must be integers

Process finished with exit code 1`

When calling balance = contractToken.call().balanceOf(userAccount)

What am I missing? Many thanks in advance!!

UPDATED 25/11: I have this working now. I was using an incorrect token.json. Please could you advise where the token.json ABI in /bots/contracts comes from?
Thanks!

frosty00 · 2017-11-25T10:21:10Z

Where is the python code that uses the new websocket API?

The bug bounty says it has been claimed, but there is no link to any working python code.

https://gitcoin.co/funding/details?url=https://github.com/etherdelta/etherdelta.github.io/issues/225

anlamak · 2017-11-25T11:01:11Z

@frosty00 See post above from @tomvanbraeckel

https://github.com/etherdelta/etherdelta.github.io/files/1484991/taker_v2.py.txt

def websocket_connect(): ws = websocket.WebSocketApp( "wss://socket.etherdelta.com/socket.io/?transport=websocket", on_message=on_message, on_ping=on_ping, on_pong=on_pong, on_error=on_error, on_close=on_close) ws.on_open = on_open # The API seems to close the connection, even when we send a periodic ping ws.run_forever(ping_interval=10)

frosty00 · 2017-11-25T11:48:13Z

@tomvanbraeckel looking at the code now.

Can you update the repo https://github.com/tomvanbraeckel/etherdeltaclientapi.py so I that I can work on it too?

I've opened an issue with some of the stuff I would like to have in a Python Etherdelta API.

P.S. I'm more than happy to work on this free on charge ;)

PhpRu · 2017-11-25T20:27:46Z

How to get address for each token smart-contract? I only found etherdelta global smart-contract which is constant

zackcoburn · 2017-11-25T20:33:14Z

Etherscan or https://github.com/etherdelta/etherdelta.github.io/blob/master/config/main.json Zack Coburn NOTICE: This e-mail transmission (and/or the attachments accompanying it) may contain confidential information belonging to the sender which is protected by law. The information is intended only for the use of the intended recipient. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or the taking of any action in reliance on the contents of this information is strictly prohibited. Any unauthorized interception of this transmission is illegal. If you have received this transmission in error, please promptly notify the sender by reply e-mail, and then destroy all copies of the transmission.

…

On Sat, Nov 25, 2017 at 8:27 PM, PhpRu ***@***.***> wrote: How to get address for each token smart-contract? I only found etherdelta global smart-contract which is constant — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#225 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APEqPcMZmVgdErJHq-1b7ls1qyMn1q8oks5s6HhEgaJpZM4QL5FX> .

tomvanbraeckel · 2017-11-25T21:09:47Z

I've finished the rewrite of the code, will upload today.

tomvanbraeckel · 2017-11-25T22:06:39Z

Done!

Major rewrite, I couldn't live with copy-pasting taker.py code in maker.py code so here she is; the all-new etherdeltaclientservice.py along with taker.py and maker.py!

I've added initial Unit Tests for good measure. More and better tests are of course always good. I've refactored the code a few times but your improvements are more than welcome.

In hindsight it took some time because web3.py lacks the proper signing algorithm for maker.py - it has SoliditySha3 but not something like SoliditySha256. And as I was debugging these signature errors, I wished that the API wouldn't always respond "order quantity too low" but something about the signature being incorrect :-)

I also had lots of failing orders with "Error encountered during contract execution [Bad jump destination]" on some markets but not usually on others, which took time to debug. In the end I guess my gas price of 1 Gwei is just too low for these high volume markets, as the orders get traded before my transaction gets mined...

I think I wasted quite some Ether on silly tests. Anyway, it was well worth it! EtherDelta for the win!

@zackcoburn taker.py is done! Would you like me to send a merge request? Or maybe you could merge it as you know best where to put it in etherdelta.github.io... I'm not sure whether we should create a substructure for python bots to keep python and javascript code clearly separated, or place it all in the top-level bots directory, since it's only a few files, or...? Anyway, the code is MIT just like etherdelta.github.io so feel free to do as you see fit :-)

@frosty00 Done! Your contributions, feedback and improvements are very welcome indeed :-)

owocki · 2017-11-27T14:47:45Z

@zackcoburn let me know if we're close to being able to pay out this bounty?

zackcoburn · 2017-11-28T04:41:14Z

Great work @tomvanbraeckel, ready for payout @owocki! I've moved the bots into their own repository, https://github.com/etherdelta/bots. (The only modification I made to the python bot was moving the contracts path to be one level up so all bots can share the same contracts.)

gitcoinbot · 2017-11-28T14:19:15Z

The funding of 0.55 ETH attached to this issue has been approved & issued.

Learn more at: https://gitcoin.co/funding/details?url=https://github.com/etherdelta/etherdelta.github.io/issues/225

tomvanbraeckel · 2017-11-29T08:07:41Z

Perfect, thanks a lot @zackcoburn and @owocki!

tomvanbraeckel mentioned this issue Dec 17, 2017

Buy orders are successful, but sell orders aren't working for me etherdelta/bots#13

Open

Python-based client for the new socket-based API #225

Python-based client for the new socket-based API #225

Comments

owocki commented Oct 30, 2017 • edited Loading

gitcoinbot commented Oct 31, 2017

owocki commented Nov 1, 2017

bennyprofane commented Nov 2, 2017 • edited Loading

owocki commented Nov 2, 2017 via email

ghost commented Nov 3, 2017

zackcoburn commented Nov 3, 2017 via email

owocki commented Nov 5, 2017

ghost commented Nov 5, 2017

ghost commented Nov 5, 2017

bennyprofane commented Nov 6, 2017 • edited Loading

zheli commented Nov 7, 2017 • edited Loading

zackcoburn commented Nov 7, 2017 via email

mdevalck commented Nov 16, 2017

tomvanbraeckel commented Nov 17, 2017 • edited Loading

zheli commented Nov 18, 2017

ghost commented Nov 18, 2017

tomvanbraeckel commented Nov 18, 2017 • edited Loading

tomvanbraeckel commented Nov 18, 2017 • edited Loading

bellerophons-pegasus commented Nov 18, 2017

tomvanbraeckel commented Nov 18, 2017 • edited Loading

bellerophons-pegasus commented Nov 19, 2017

owocki commented Nov 20, 2017

zackcoburn commented Nov 20, 2017

zheli commented Nov 20, 2017

owocki commented Nov 20, 2017

zheli commented Nov 21, 2017

tomvanbraeckel commented Nov 21, 2017 • edited Loading

zackcoburn commented Nov 21, 2017 via email

anlamak commented Nov 24, 2017 • edited Loading

frosty00 commented Nov 25, 2017

anlamak commented Nov 25, 2017

frosty00 commented Nov 25, 2017 • edited Loading

PhpRu commented Nov 25, 2017

zackcoburn commented Nov 25, 2017 via email

tomvanbraeckel commented Nov 25, 2017

tomvanbraeckel commented Nov 25, 2017 • edited Loading

owocki commented Nov 27, 2017

zackcoburn commented Nov 28, 2017 via email

gitcoinbot commented Nov 28, 2017

tomvanbraeckel commented Nov 29, 2017

owocki commented Oct 30, 2017 •

edited

Loading

bennyprofane commented Nov 2, 2017 •

edited

Loading

bennyprofane commented Nov 6, 2017 •

edited

Loading

zheli commented Nov 7, 2017 •

edited

Loading

tomvanbraeckel commented Nov 17, 2017 •

edited

Loading

tomvanbraeckel commented Nov 18, 2017 •

edited

Loading

tomvanbraeckel commented Nov 18, 2017 •

edited

Loading

tomvanbraeckel commented Nov 18, 2017 •

edited

Loading

tomvanbraeckel commented Nov 21, 2017 •

edited

Loading

anlamak commented Nov 24, 2017 •

edited

Loading

frosty00 commented Nov 25, 2017 •

edited

Loading

tomvanbraeckel commented Nov 25, 2017 •

edited

Loading