How do I start paper trade from the recent time?

[booboothefool]

I was under the impression that if you start paper trading and set historical=False with no date it should start getting data from the most recent time. When I try with other broker such as backtrader with Oanda, it does not seem necessary to set fromdate.

    paper = True
    live = True    

    store = alpaca_backtrader_api.AlpacaStore(
        key_id=key_id,
        secret_key=secret_key,
        paper=paper,
    )

    DataFactory = store.getdata
    data0 = DataFactory(dataname='SHOP',
            timeframe=bt.TimeFrame.TFrame("Minutes"),
            historical=False,
    )

    if live:
        broker = store.getbroker(use_positions=True)
        cerebro.setbroker(broker)
    else:
        cerebro.broker.setcash(100000.0)
       
    data0.addfilter(bt.filters.Renko, size=0.5)
    cerebro.adddata(data0)

and then logging out the time in next():

        txt = list()
        txt.append('Data0')
        txt.append('%04d' % len(self.data0))
        dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
        txt.append('%s' % self.data.datetime.datetime(0).strftime(dtfmt))
        txt.append('C {:2f}'.format(self.data.close[0]))
        txt.append('O {:2f}'.format(self.data.open[0]))
        print(', '.join(txt))

Today is currently 5/12 18:05, but the data will feeding from 5/11 13:56.

Any suggestions? Also in general, I can't seem to get any sort of Renko chart to operate correctly e.g. it always looks and acts differently from what I'd see on TradingView and fires multiple signals for the same bar if anyone has any experience with that. Any help is greatly appreciated.

[shlomiku]

try the new version and let me know if you still get this error: https://github.com/alpacahq/alpaca-backtrader-api/releases/tag/v0.8.0

[booboothefool]

I am still getting the error where it starts from yesterday. It doesn't seem to start at the right live time unless I include fromdate=pd.Timestamp(2020,5,14,14) up to the most recent hour.

I am also getting an error where it doesn't detect that I have a position, when I clearly do and can see it on the Alpaca dashboard. I have seen this work before in the past so not sure if it's just my latest code which uses multiple data feeds, understanding that it should default to data0.

print(self.position) # returns 0 size

I am also getting this error on the first order, but it doesn't seem to break anything.

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/a/opt/anaconda3/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/Users/a/opt/anaconda3/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 328, in _t_streaming_listener
    self._transaction(trans)
  File "/usr/local/lib/python3.7/site-packages/alpaca_backtrader_api/alpacastore.py", line 650, in _transaction
    oid = trans['id']
TypeError: 'Entity' object is not subscriptable
TypeError: 'Entity' object is not subscriptable
on the first order

Sometimes I don't get the Completed status:

2020-05-14: Order ref: 2 / Type Buy / Status Submitted
2020-05-14: Order ref: 2 / Type Buy / Status Accepted

Sometimes I get duplicate status:

2020-05-14: Order ref: 2 / Type Buy / Status Accepted
2020-05-14: Order ref: 2 / Type Buy / Status Accepted

The orders don't seem to break though - they seem to go through and complete ok, regardless of the notifications.

Is there a way to print out why it is rejected? (Usually it's just cause of buying power.)

2020-05-14: Order ref: 3 / Type Buy / Status Rejected

Here is my full code. Update with your keys and you can run like:

python3 mystrat.py --plot --broker=alpaca --candle=heikin --symbol=SHOP

I am having quite a bit of trouble getting just a simple "buy when green and sell when red" strat to execute properly. When using Oanda it seems to execute as expected and I have gotten the raw alpaca-trade-api to work well, but I could use some help here with the errors. Thanks!

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime
import pandas as pd

import backtrader as bt

import alpaca_backtrader_api
import backtrader.analyzers as btanalyzers

import btoandav20
StoreCls = btoandav20.stores.OandaV20Store
DataCls = btoandav20.feeds.OandaV20Data
# BrokerCls = btoandav20.brokers.OandaV20Broker

from keys.keys import key_id, secret_key, base_url, token, account

class St(bt.Strategy):
    params = (
        ('candle', None),
        ('test', True),
        ('compression', 1)
    )
    
    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
        # if status == data.LIVE:
            # self.counttostop = self.p.stopafter
            # self.datastatus = 1
            
    def notify_order(self, order):
        print('{}: Order ref: {} / Type {} / Status {}'.format(
            self.data.datetime.date(0),
            order.ref, 'Buy' * order.isbuy() or 'Sell',
            order.getstatusname()))
        
    def notify_trade(self, trade):
        if not trade.isclosed:
            return
        symbol = ''
        if trade.pnl > 0: symbol = '💸 +'
        elif trade.pnl < 0: symbol = '🔻 '
        else: symbol = '◀ '
        if trade.pnl != 0:
            print(f'(TRD)\t\t\t{symbol}%.6f' % trade.pnl)
        
    def stop(self):
        print('==================================================')
        print('Starting Value - %.2f' % self.broker.startingcash)
        print('Ending   Value - %.2f' % self.broker.getvalue())
        print('==================================================')


    def __init__(self):
        for i, d in enumerate(self.datas):
            print(i, d._name)
            if d._name == 'Real':
                self.real = d
            elif d._name == 'Heikin':
                self.hk = d
            elif d._name == 'Renko':
                self.renko = d
            
        # self.had = bt.ind.haDelta(self.hk)
        # self.hadx = bt.ind.CrossOver(self.had.lines.smoothed, 0)
        
        self.green_streak = 0
        self.red_streak = 0
        
        self.lastlen = 0
        self.lasttime = 0
        

    def next(self):        
        # skip the same time data lines (RENKO causes this)
        if self.params.candle == 'renko':
            if self.lastlen and len(self.data0) == self.lastlen: return
            self.lastlen = len(self.data0)
                
        logicdata = None
        orderdata = None
        if self.params.candle == 'heikin':
            logicdata = self.hk
            orderdata = self.real
        elif self.params.candle == 'renko':
            logicdata = self.renko
            orderdata = self.renko
        else:
            logicdata = self.real
            orderdata = self.real
        
        color = ''
        diff = 0
        # boxsize = 1
        if logicdata.close[0] > logicdata.open[0]:
            color = '🟢'
            diff = logicdata[0] - logicdata[-1]
            self.green_streak += diff
        elif logicdata.close[0] < logicdata.open[0]:
            color = '🔴'
            diff = -(logicdata[-1] - logicdata[0])
            self.red_streak += diff
        elif logicdata.close[0] == logicdata.open[0]:
            color = '⚪️'
            diff = 0.0
            # self.green_streak = self.red_streak = 0
                        
        compression = self.params.compression
        secs_in_units = 60
        recent_data = (datetime.datetime.utcnow() - self.data.datetime.datetime(0)).total_seconds() / secs_in_units <= compression + 1
        
        txt = list()
        txt.append(f'{"LIVE" if recent_data else "BACK"}')
        txt.append(f'{logicdata.LIVE}')
        txt.append('Data0')
        txt.append('%04d' % len(self.data0))
        dtfmt = '%m-%d %H:%M:%S'
        txt.append('%s' % self.data.datetime.datetime(0).strftime(dtfmt))
        txt.append('O {:2f}'.format(logicdata.open[0]))
        txt.append('C {:2f}'.format(logicdata.close[0]))
        # # txt.append(f'{self.hadelta.lines.smoothed[0]} {self.hdx[0]}')
        txt.append(f'{"+" if diff > 0 else ""}{diff} {color}')
        print(', '.join(txt))
        
        # LIVE ONLY
        # a new piece of data we're using shouldn't be older than the time we're using (to skip backfill)
        if not self.params.test:
            if not recent_data:
                return False
        
        print(f'\t\t\t\t\t\t\t\t\t\t\t🟢 {self.green_streak} 🔴 {self.red_streak}')
        # required_streak = 3
        
        # Green Heikin
        long = logicdata.close[0] > logicdata.open[0]
        # long = logicdata.close[0] > logicdata.open[0] and logicdata.close[-1] > logicdata.open[-1]
        # long = self.green_streak >= required_streak
        # long = self.hadx == 1
        
        # Red Heikin
        short = logicdata.close[0] < logicdata.open[0]
        # short = logicdata.close[0] < logicdata.open[0] and logicdata.close[-1] < logicdata.open[-1]
        # short = self.red_streak <= -required_streak
        # short = self.hadx == -1
        
        print(self.position) # why is this blank when I have a position?
        
        if not self.position:
            if long:
                self.buy()
                print(f'BUY @ {orderdata.close[0]}')
                
                self.green_streak = self.red_streak = 0
            elif short:
                self.sell()
                print(f'SELL @ {orderdata.close[0]}')
                
                self.green_streak = self.red_streak = 0

        else:
            if self.position.size > 0 and short:
                self.close() 
                print(f'BUY CLOSE (SELL) @ {orderdata.close[0]}')
                
                self.green_streak = self.red_streak = 0
            elif self.position.size < 0 and long:
                self.close()
                print(f'SELL CLOSE (BUY) @ {orderdata.close[0]}')
                
                self.green_streak = self.red_streak = 0

                    
def measure(cerebro):
    cerebro.addanalyzer(btanalyzers.AnnualReturn, _name='myannualreturn')
    cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mysharpe', timeframe=bt.TimeFrame.Minutes, compression=1)
    cerebro.addanalyzer(btanalyzers.SQN, _name='mysqn')
    cerebro.addanalyzer(btanalyzers.TradeAnalyzer, _name='mytradeanalyzer')
    
    thestrats = cerebro.run()
    
    thestrat = thestrats[0]
    print('Annual Return:', thestrat.analyzers.myannualreturn.get_analysis())
    print('Sharpe Ratio:', thestrat.analyzers.mysharpe.get_analysis())
    print('SQN:', thestrat.analyzers.mysqn.get_analysis())
    ta = thestrat.analyzers.mytradeanalyzer.get_analysis()
    print('Trade Analyzer:')
    if 'streak' in ta:
        print('longest win / lose streak:\t', f"{ta['streak']['won']['longest']} / {ta['streak']['lost']['longest']}")
        print('win / lose:               \t', f"{ta['won']['total']} W / {ta['lost']['total']} L")
        print('win rate %:               \t', f"{ta['won']['total']/(ta['won']['total']+ta['lost']['total'])*100}")
        

def runstrat(args=None):
    args = parse_args(args)

    cerebro = bt.Cerebro()

    # use the lowest timeframe and compression available, resample after
    if args.broker == 'alpaca':
        store = alpaca_backtrader_api.AlpacaStore(
            key_id=key_id,
            secret_key=secret_key,
            paper=True,
            usePolygon=True,
        )
        DataFactory = store.getdata
        data0 = DataFactory(dataname=args.symbol,
                # timeframe=bt.TimeFrame.Days,
                timeframe=bt.TimeFrame.Minutes,
                compression=1,
                fromdate=pd.Timestamp(2020,5,14,14),
                # sessionstart=datetime.time(14, 0),
                historical=args.test,
        )
    elif args.broker == 'oanda':
        store = btoandav20.stores.OandaV20Store(
                token=token,
                account=account,
                practice=True,
        )
        DataFactory = store.getdata
        data0 = DataFactory(dataname=args.symbol,
                # timeframe=bt.TimeFrame.Minutes,
                timeframe=bt.TimeFrame.Seconds,
                compression=5,
                fromdate=pd.Timestamp(2020,5,13),
                # todate=pd.Timestamp(2020,5,7),
                historical=args.test,
                # backfill_start=False,
        )

    fkwargs = dict()
    fkwargs.update(**eval('dict(' + args.renko + ')'))

    compression, timeframe = 1, bt.TimeFrame.Minutes

    if args.candle:
        if args.candle == 'heikin':
            # REAL
            cerebro.resampledata(data0, name="Real", timeframe=timeframe, compression=compression)
            data0.plotinfo.plot = False
            
            # HEIKIN ASHI
            # have to resample it again it seems
            data1 = data0.clone()
            data1.addfilter(bt.filters.HeikinAshi)
            cerebro.resampledata(data1, name="Heikin", timeframe=timeframe, compression=compression)
            
        if args.candle == 'renko':
            # RENKO ONLY
            # can't seem to use clone with Renko, so can only do the 1 data stream
            equitysize = 1
            forexsize = 0.01
            boxsize = 0
            if args.broker == 'alpaca':
                boxsize = equitysize
            elif args.broker == 'oanda':
                boxsize = forexsize
            data0.addfilter(bt.filters.Renko, size=boxsize)
            cerebro.resampledata(data0, name="Renko", timeframe=timeframe, compression=compression)
    else:
        cerebro.resampledata(data0, name="Real", timeframe=timeframe, compression=compression)

    if args.test:
        cerebro.broker.setcash(100000)
    else:
        broker = store.getbroker()
        cerebro.setbroker(broker)
        
    cerebro.addsizer(bt.sizers.FixedSize, stake=1)
    
    cerebro.addstrategy(St, candle=args.candle, test=args.test, compression=compression)
    
    measure(cerebro)

    cerebro.plot(style='candle', barup='green', bardown='maroon')
        

def parse_args(pargs=None):
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        description=(
            'Renko bricks sample'
        )
    )
    
    parser.add_argument('--symbol', required=True, default='',
                        metavar='kwargs', help='kwargs in key=value format')
    
    parser.add_argument('--candle', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')
    
    parser.add_argument('--test', default='', nargs='?', const='{}',metavar='kwargs',
                        required=False, help='Data to read in')

    parser.add_argument('--data0', default='data/TSLA.csv',
                        required=False, help='Data to read in')

    # Defaults for dates
    parser.add_argument('--fromdate', required=False, default='',
                        help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')

    parser.add_argument('--todate', required=False, default='',
                        help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')

    parser.add_argument('--cerebro', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--broker', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--sizer', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--strat', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--plot', required=False, default='',
                        nargs='?', const='{}',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--renko', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--dual', required=False, action='store_true',
                        help='put the filter on a second version of the data')

    return parser.parse_args(pargs)


if __name__ == '__main__':
    runstrat()

[shlomiku]

so, executing a complexed algorithm introduces complexed debugging environment.
it is not a good way to debug.
executing the simple example code sma_crossover_strategy.py and adding the date prints you pointed out gives a result that seems correct:

and the result:

which seems as expected.
try to run the example code, and let me know if you get the same result

this is the first sample I get. running it now (14:57 UTC)
and I am using the alpaca data stream

[ic-gcp]

Hey @shlomikushchi, so I'm facing the same issue where if I'm trying to get live data for paper trading on the 16th of June it gives me data starting from the 15th of June. Also if I reset the API key the data request URL shows that the API is being asked for data starting from the 13th of June. Now since 13th and 14th are weekends it starts giving data from the 15th.

This is the URL that I mentioned above:

[https://api.polygon.io/v2/aggs/ticker/NVDA/range/1/minute/2020-06-13/2020-06-16?unadjusted=False&apiKey=XXXXXXXXXXXXXX]

This is the sample code that I'm using:

`import backtrader as bt
import alpaca_backtrader_api
import pandas as pd

class SmaCross(bt.SignalStrategy):
def init(self):
sma1, sma2 = bt.ind.SMA(period=10), bt.ind.SMA(period=30)
crossover = bt.ind.CrossOver(sma1, sma2)
self.signal_add(bt.SIGNAL_LONG, crossover)
def next(self):
print(self.datas[0].datetime.datetime(0))

is_live = True

cerebro = bt.Cerebro()
cerebro.addstrategy(SmaCross)

store = alpaca_backtrader_api.AlpacaStore(key_id=ALPACA_API_KEY,
secret_key= ALPACA_SECRET_KEY,
paper=ALPACA_PAPER)

if is_live:
broker = store.getbroker() # or just alpaca_backtrader_api.AlpacaBroker()
cerebro.setbroker(broker)
else:
cerebro.broker.setcash(100000)
cerebro.broker.setcommission(commission=0.0)
cerebro.addsizer(bt.sizers.PercentSizer, percents=20)

DataFactory = store.getdata # or use alpaca_backtrader_api.AlpacaData
if is_live:
data0 = DataFactory(
dataname='NVDA',
timeframe=bt.TimeFrame.TFrame("Minutes"),
)
else:
data0 = DataFactory(
dataname='NVDA',
timeframe=bt.TimeFrame.TFrame("Minutes"),
fromdate=pd.Timestamp('2020-06-15'),
todate=pd.Timestamp('2020-06-16'),
historical=True)
cerebro.adddata(data0)

cerebro.run()
print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
cerebro.plot()`

[shlomiku]

Hi, data from the past is probably required for your indicators.
you should use this:

    def notify_data(self, data, status, *args, **kwargs):
        super().notify_data(data, status, *args, **kwargs)
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)

and when status == 'LIVE' (all statuses defined in _NOTIFNAMES)
set it to self, and only then do something inside next()

[shlomiku]

closing due to lack of activity