retry and default timeout

[jencijanos]

i use default timeout value, 1 sec. ping to poloniex.com <2ms. place some sell order. sleep time between order places is 2 sec. coach set to 1 call/1sec. some time i found sell order duplicates in order list.

{'orderNumber': '76671857444', 'type': 'sell', 'total': '0.00231183', 'date': '2017-05-06 08:39:18', 'margin': 0, 'amount': '0.10391764', 'startingAmount': '0.10391764', 'rate': '0.02224676'}, {'orderNumber': '76671858443', 'type': 'sell', 'total': '0.00231183', 'date': '2017-05-06 08:39:18', 'margin': 0, 'amount': '0.10391764', 'startingAmount': '0.10391764', 'rate': '0.02224676'}

conclusion (for me) at this time for poloniex 1 second is not perfect timeout value!

[s4w3d0ff]

I have been noticing lots of new errors recently.

I saw something similar to what you got I was trying to cancel a loan order and it 'failed' with a connection error, but poloniex actually received the request and retry tried the call again.

[11:24:00]Checking Open Loan Offers:----------------
[11:24:02]-- delaying for 0s
[11:24:03]2017-05-08 18:13:47|DASH:2.60511986-[rate:0.0044%]
[11:24:03]Canceling DASH offer 434067565.0
[11:24:05]-- delaying for 0s
[11:24:07]-- delaying for 2s
[11:24:11]Error canceling loan order, or you are not the person who placed it.
Traceback (most recent call last):
  File "loanbot.py", line 124, in run
    self.cancelOldOffers()
  File "loanbot.py", line 91, in cancelOldOffers
    logger.debug(self.api.cancelLoanOffer(offer['id']))
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 533, in cancelLoanOffer
    'cancelLoanOffer', {'orderNumber': str(orderNumber)})
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/retry.py", line 15, in wrapped
    return function(*args, **kwargs)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 182, in __call__
    raise PoloniexError(jsonout['error'])
PoloniexError: Error canceling loan order, or you are not the person who placed it.

And again when trying to create a new loan offer:

[09:40:38]Checking for coins to lend:---------------
[09:40:39]DASH:2.60511986
[09:40:40]Creating 2.60511986 DASH loan offer at 0.005%
[09:40:42]-- delaying for 0s
[09:40:44]Not enough DASH available to offer.
Traceback (most recent call last):
  File "loanbot.py", line 126, in run
    self.createLoanOffers()
  File "loanbot.py", line 116, in createLoanOffers
    coin, amount, price, autoRenew=0))
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 527, in createLoanOffer
    'lendingRate': str(lendingRate)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/retry.py", line 15, in wrapped
    return function(*args, **kwargs)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 182, in __call__
    raise PoloniexError(jsonout['error'])
PoloniexError: Not enough DASH available to offer.

I think poloniex has been having some network issues. I was running my loanbot all last night, and was getting errors almost the whole time. I have been getting the following errors (even after setting timeout=3):

[12:33:09]Checking for coins to lend:---------------
[12:33:09]DASH:2.60511986
[12:33:10]Creating 2.60511986 DASH loan offer at 0.006%
[12:33:13]-- delaying for 0s
[12:33:16]-- delaying for 2s
[12:33:22]-- delaying for 5s
[12:33:30]-- delaying for 30s
[12:34:03][ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)",),), ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)",),), ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)",),), ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)",),), ReadTimeout(ReadTimeoutError("HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)",),)]
[12:34:03]HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)
Traceback (most recent call last):
  File "loanbot.py", line 126, in run
    self.createLoanOffers()
  File "loanbot.py", line 116, in createLoanOffers
    coin, amount, price, autoRenew=0))
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 527, in createLoanOffer
    'lendingRate': str(lendingRate)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/retry.py", line 15, in wrapped
    return function(*args, **kwargs)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 172, in __call__
    timeout=self.timeout)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 107, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 449, in send
    raise ReadTimeout(e, request=request)
ReadTimeout: HTTPSConnectionPool(host='poloniex.com', port=443): Read timed out. (read timeout=3)

And this one is new for me:

[12:31:21]Checking Open Loan Offers:----------------
[12:31:26]No JSON object could be decoded
Traceback (most recent call last):
  File "loanbot.py", line 124, in run
    self.cancelOldOffers()
  File "loanbot.py", line 76, in cancelOldOffers
    offers = self.api.returnOpenLoanOffers()
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 537, in returnOpenLoanOffers
    return self.__call__('returnOpenLoanOffers')
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/retry.py", line 15, in wrapped
    return function(*args, **kwargs)
  File "/home/s4w3d0ff/.local/lib/python2.7/site-packages/poloniex/__init__.py", line 179, in __call__
    parse_int=self.jsonNums)
  File "/usr/lib/python2.7/json/__init__.py", line 352, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

[s4w3d0ff]

It looks like Poloniex is working on the problem, I got this new error recently:

[13:13:29]Checking for coins to lend:---------------
[13:13:30]ETH:0.53054985
[13:13:30]Too many requests. Please try again in a few minutes.
Traceback (most recent call last):
  File "loaner.py", line 79, in run
    self.createLoanOffers()
  File "loaner.py", line 63, in createLoanOffers
    orders = self.api.returnLoanOrders(coin)['offers']
  File "/media/s4w3d0ff/EXT4Space/Projects/marconibot/marconi/tools/poloniex/__init__.py", line 281, in returnLoanOrders
    'currency': str(currency).upper()})
  File "/media/s4w3d0ff/EXT4Space/Projects/marconibot/marconi/tools/poloniex/retry.py", line 17, in wrapped
    return function(*args, **kwargs)
  File "/media/s4w3d0ff/EXT4Space/Projects/marconibot/marconi/tools/poloniex/__init__.py", line 199, in __call__
    raise PoloniexError(jsonout['error'])
tools.poloniex.PoloniexError: Too many requests. Please try again in a few minutes.

[jencijanos]

after last DDoS on polo, for test, i set timeout 120sec. today many time i get error message "Connection timed out. Please try again." after 30 sec of "wait". polo programmers working on API?

[jknielse]

Yeah, I was running into this issue too. Even 10s resulted in some erroneous retries for me. Poloniex did just tweet about their insane user growth and DDoS attacks in the past little while. I guess that's the cause of all these issues.

[cyzanfar]

Issue should be fixed since last commit b7410bd

CC: @s4w3d0ff

[s4w3d0ff]

OK, so I submitted a ticket with polo support and here is the response:

You may be temporarily blocked by Cloudflare if your API calls create many client errors (any HTTP 4xx code such as invalid nonce, insufficient balance, etc). Did you see any such errors before you were blocked?

Please note that while re-trying a timed out call is fine, if you re-try an unanswered API call after less than 80 seconds with the same API key, the earlier call may still go through however its nonce value collides with the second call, causing an error 422. If your client already dropped the request, you would not actually see the error, but nonetheless it occurred. If you cause too many of these errors Cloudflare may temporarily block you.

In terms of a solution this depends on your setup but I could suggest using a set of rotating API keys to avoid nonce errors, or use a time-out of 80 seconds or higher.

So I am going to set the default timeout to 80 sec, and maybe work on an api key rotator...

NOTE: nonces should not collide because the nonce is re-calculated before retrying (but this is probably what causes the duplicate requests when timeout is set low)

[ekerstein]

@s4w3d0ff Are you going to archive a new version for the readme? New people installing the latest release will still get the timeout=1 which could cause issues for them.

[s4w3d0ff]

ya, I had done a few other small changes and wanted to make sure it functioned. Ill do a release now.

[s4w3d0ff]

OK v0.4.3 has been archived with 80 second timeout

The above errors should be minimal now unless you are sending lots of requests that result in errors.

[skyl]

I got this from Polo, after experiencing much the same thing. They suggested 130 second timeout.

Due to ongoing DDOS there are currently rather strict limits on 499 (client closed connection) responses, in order to avoid an issue please use a timeout of 130 seconds or higher. If you request API calls and then drop them before the server had a chance to reply, you may be temporarily blocked by Cloudflare.

[s4w3d0ff]

You can also set timeout=None and it will never timeout. This is what I am going to set it to for the next release.
http://docs.python-requests.org/en/master/user/advanced/#timeouts