Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for hasFetchOHLCV tag, so we work correctly with exchanges like Kraken that use the newer format #5

Closed
wants to merge 18 commits into from

Conversation

moogman
Copy link

@moogman moogman commented Jan 24, 2018

As title; Tested OK against Kraken and Binance.

bartosh and others added 18 commits November 27, 2017 16:11
This is a draft implementation - to be continued

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Instead of using last_id added new property _last_ts to
track last data timestamp.

This should fix a bug where no data was added when
using a time based frame.
Check if exchange supports OHLCV fetching in _fetch_ohlcv method.

This should prevent code duplication because this was not handled
in the "start" method.
Fetched data since the last timestamp instead of fetching the
whole data feed which made it more slower.

This makes it more efficient to feed the live data, so for
each iteration it will fetch only the newer data.
2017-12-04:

just pip installed the package and this error popped up so i came here to check it out and open a PR if yall want it! Chances are you might already know about it
Fetched OHLCV data in chunks of ohlcv_limit bars.
Do it while there is new data available.

This should fix hangups and crashes when a lot of history
data requested.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
We'll need to handle request timeouts and other things,
so it makes sense to move all ccxt-related functionality
to the separate module to be able to call it from broker
and feed classes.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Retry exchange queries when network error[s] occurs.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Checked if requested timeframe/compression is supported
by the exchange.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
_load method of the feed should returned False when price
data is not yet available. This caused cerebro to stop.
It should return None in this case to indicate timeout
getting the data.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Don't process ohlcv bars that contain None prices

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Default order of output historical bars is not the same
for different exchanges.
Explicitly specifying it with params={"reverse": False}
seems not having any effect.

Had to sort the output in the feed code to solve this.

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Used public order properties 'side' and 'amount' instead of
original unparsed properties ['info']['side'] and ['info']['original_amount'].
The latter are exchange-specific and may not be provided by
all exchanges.

This should fix crashes like this:
Traceback (most recent call last):
  File "machine_engin.py", line 161, in <module>
    cerebro.run()
  File "backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "backtrader/cerebro.py", line 1295, in runstrategies
    self._runnext(runstrats)
  File "backtrader/cerebro.py", line 1626, in _runnext
    strat._next()
  File "backtrader/strategy.py", line 325, in _next
    super(Strategy, self)._next()
  File "backtrader/lineiterator.py", line 266, in _next
    self.next()
  File "machine_engin_prod.py", line 138, in next
    price=self.data0.close[0])
  File "backtrader/strategy.py", line 945, in sell
    **kwargs)
  File "backtrader/brokers/ccxtbroker.py", line 96, in sell
    return self._submit(owner, data, exectype, 'sell', size, price, kwargs)
  File "backtrader/brokers/ccxtbroker.py", line 82, in _submit
    order = CCXTOrder(owner, data, _order)
  File "backtrader/metabase.py", line 88, in __call__
    _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
  File "backtrader/metabase.py", line 78, in doinit
    _obj.__init__(*args, **kwargs)
  File "backtrader/brokers/ccxtbroker.py", line 34, in __init__
    self.size = float(ccxt_order['info']['original_amount'])
KeyError: 'original_amount'

Signed-off-by: Ed Bartosh <bartosh@gmail.com>
Copy link
Owner

@bartosh bartosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach doesn't work for bitmex:
In [1]: import ccxt

In [2]: exchange = ccxt.bitmex()

In [3]: exchange.has['fetchOHLCV']
Out[3]: False

In [4]: exchange.hasFetchOHLCV
Out[4]: True

We need to come up with something that works for all exchanges.

@bartosh
Copy link
Owner

bartosh commented Jan 24, 2018

BTW, I can't reproduce the issue with kraken:
In [5]: exchange = ccxt.kraken()

In [6]: exchange.hasFetchOHLCV
Out[6]: True

In [7]: exchange.has['fetchOHLCV']
Out[7]: True

@moogman
Copy link
Author

moogman commented Jan 24, 2018

Are you testing against a latest ccxt? I pulled from git just a few days ago, and saw the behaviour. I'll pop a change in to cover for both cases, as I'm sure it will bite others with and without this fix as-is

@bartosh
Copy link
Owner

bartosh commented Jan 24, 2018

Good point. This is what I've got with the latest ccxt master:

In [1]: import ccxt

In [2]: exchange = ccxt.kraken()

In [3]: exchange.hasFetchOHLCV
Out[3]: False

In [4]: exchange.has['fetchOHLCV']
Out[4]: True

In [5]: exchange = ccxt.bitmex()

In [6]: exchange.hasFetchOHLCV
Out[6]: False

In [7]: exchange.has['fetchOHLCV']
Out[7]: True

I'm just wondering would it be better to fix this in ccxt?
hasFetchOHLCV seems to be a standard method of checking this capability. Unlike has['fetchOHLCV'] it's mentioned in the ccxt manual.

@bartosh
Copy link
Owner

bartosh commented Jan 24, 2018

Looking at their commit log I'd say they're changing interface to has[] approach. However, this is not the reason to break old interface. How about creating bug in their issue tracker or at least looking if similar bug exists?

Copy link
Owner

@bartosh bartosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving as it looks like they're not going to fix it in ccxt.

@bartosh
Copy link
Owner

bartosh commented Feb 3, 2018

merged into ccxt branch

@bartosh bartosh closed this Feb 3, 2018
athon-millane pushed a commit to athon-millane/backtrader that referenced this pull request Oct 7, 2020
* bug fix: bartosh#5 fixing writer.py after 1.9.75.123 pull

* Renaming _start() to _start_output()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants