Arctic datastore as Zipline datasource

[femtotrader]

Hello,

this datastore may also be worth to be add to zipline
https://github.com/manahl/arctic
at least it should be discussed here

Kind regards

See for arctic side man-group/arctic#19

[twiecki]

I wonder if perhaps this should be linked to odo instead where it will be available to zipline if #659 gets implemented.

[femtotrader]

to my understanding that's quite different because with Blaze (and odo) you won't have a Numpy arrays like in this datastore but you are yieding values using a generator but I also understand your pov.

[twiecki]

I think odo handles streaming data sources too. @mrocklin?

[llllllllll]

odo supports streaming data sources with chunks

[femtotrader]

see blaze/blaze#1185

[llllllllll]

Sorry, what am I supposed to be seeing in that issue?

[mrocklin]

Joe is correct that the right approach here is chunks. The brief chunks documentation is here: http://odo.pydata.org/en/latest/type-modifiers.html#chunks

[femtotrader]

@llllllllll you are supposed to see your own post

for i, chunk in enumerate(odo(dat, chunks(pd.DataFrame), chunksize=10)):
    print("chunk %d" % (i + 1))
    for j, row in chunk.iterrows():
        print(row)

and you are right Blaze wasn't used in this simple example but I wonder if it will be the case if you want to align data.

[llllllllll]

Oh, were you just showing the iterable? I wasn't sure if you were responding to my comment there