Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix tardis support #172

Merged
merged 81 commits into from
Jan 10, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
0289cfb
Merge pull request #94 from nautechsystems/develop
cjdsellers Oct 25, 2020
9b0aa33
Merge pull request #95 from nautechsystems/develop
cjdsellers Oct 27, 2020
1a5cf5f
Merge pull request #96 from nautechsystems/develop
cjdsellers Oct 27, 2020
152e1ad
Merge pull request #97 from nautechsystems/develop
cjdsellers Oct 28, 2020
e52580c
Merge pull request #98 from nautechsystems/develop
cjdsellers Oct 28, 2020
76a871f
Merge pull request #99 from nautechsystems/develop
cjdsellers Oct 28, 2020
c489ab1
Merge pull request #100 from nautechsystems/develop
cjdsellers Oct 28, 2020
313f78a
Merge pull request #101 from nautechsystems/develop
cjdsellers Oct 29, 2020
b9c6a09
Merge pull request #102 from nautechsystems/develop
cjdsellers Oct 29, 2020
e3ffa4b
Merge pull request #103 from nautechsystems/develop
cjdsellers Oct 30, 2020
152f927
Merge pull request #104 from nautechsystems/develop
cjdsellers Nov 1, 2020
a3de5d7
Merge pull request #105 from nautechsystems/develop
cjdsellers Nov 2, 2020
be02c55
Merge pull request #107 from nautechsystems/develop
cjdsellers Nov 2, 2020
bc60d48
Merge pull request #108 from nautechsystems/develop
cjdsellers Nov 2, 2020
19b66e1
Merge pull request #109 from nautechsystems/develop
cjdsellers Nov 3, 2020
b90fc0c
Merge pull request #110 from nautechsystems/develop
cjdsellers Nov 4, 2020
d7e644c
Merge pull request #111 from nautechsystems/develop
cjdsellers Nov 4, 2020
4b3508e
Merge pull request #112 from nautechsystems/develop
cjdsellers Nov 5, 2020
8e07fd6
Merge pull request #113 from nautechsystems/develop
cjdsellers Nov 5, 2020
494e8f3
Merge pull request #114 from nautechsystems/develop
cjdsellers Nov 5, 2020
d9a0b87
Merge pull request #115 from nautechsystems/develop
cjdsellers Nov 5, 2020
6b91848
Merge pull request #116 from nautechsystems/develop
cjdsellers Nov 7, 2020
0c410dd
Merge pull request #117 from nautechsystems/develop
cjdsellers Nov 8, 2020
2e56c9c
Merge pull request #120 from nautechsystems/develop
cjdsellers Nov 9, 2020
7cfdfa7
Merge pull request #121 from nautechsystems/develop
cjdsellers Nov 11, 2020
b445b7d
Merge pull request #125 from nautechsystems/develop
cjdsellers Nov 13, 2020
9abc794
Merge pull request #128 from nautechsystems/develop
cjdsellers Nov 14, 2020
56a10d3
Merge pull request #131 from nautechsystems/develop
cjdsellers Nov 17, 2020
399006d
Merge pull request #132 from nautechsystems/develop
cjdsellers Nov 18, 2020
a9708d7
Merge pull request #133 from nautechsystems/develop
cjdsellers Nov 19, 2020
8596555
Merge pull request #134 from nautechsystems/develop
cjdsellers Nov 19, 2020
02ca2df
Merge pull request #135 from nautechsystems/develop
cjdsellers Nov 21, 2020
4db1103
Merge pull request #136 from nautechsystems/develop
cjdsellers Nov 22, 2020
fc287cb
Merge pull request #137 from nautechsystems/develop
cjdsellers Nov 24, 2020
485f813
Merge pull request #138 from nautechsystems/develop
cjdsellers Nov 24, 2020
f07fe40
Merge pull request #139 from nautechsystems/develop
cjdsellers Nov 26, 2020
b4091c6
Merge pull request #140 from nautechsystems/develop
cjdsellers Nov 27, 2020
7eb25c4
Merge pull request #141 from nautechsystems/develop
cjdsellers Nov 29, 2020
cd330ef
Merge pull request #142 from nautechsystems/develop
cjdsellers Nov 30, 2020
b44ec25
Merge pull request #143 from nautechsystems/develop
cjdsellers Dec 2, 2020
756f08a
Merge pull request #144 from nautechsystems/develop
cjdsellers Dec 4, 2020
b8ddd89
Merge pull request #145 from nautechsystems/develop
cjdsellers Dec 4, 2020
cc60295
Merge pull request #146 from nautechsystems/develop
cjdsellers Dec 7, 2020
cbfd1ec
Merge pull request #147 from nautechsystems/develop
cjdsellers Dec 7, 2020
f1516c6
Merge pull request #148 from nautechsystems/develop
cjdsellers Dec 8, 2020
98c35b6
Merge pull request #149 from nautechsystems/develop
cjdsellers Dec 10, 2020
0964117
Merge pull request #150 from nautechsystems/develop
cjdsellers Dec 14, 2020
29a89b2
Merge pull request #151 from nautechsystems/develop
cjdsellers Dec 15, 2020
7d2bce3
Merge pull request #152 from nautechsystems/develop
cjdsellers Dec 16, 2020
5c3c55f
Merge pull request #153 from nautechsystems/develop
cjdsellers Dec 16, 2020
59a01bc
Merge pull request #154 from nautechsystems/develop
cjdsellers Dec 18, 2020
1043a56
Merge pull request #155 from nautechsystems/develop
cjdsellers Dec 21, 2020
4bc3d73
Merge pull request #156 from nautechsystems/develop
cjdsellers Dec 23, 2020
38aa711
Merge pull request #157 from nautechsystems/develop
cjdsellers Dec 29, 2020
817171a
Merge pull request #158 from nautechsystems/develop
cjdsellers Dec 31, 2020
67f2522
Merge pull request #159 from nautechsystems/develop
cjdsellers Jan 1, 2021
31f74fc
Merge pull request #160 from nautechsystems/develop
cjdsellers Jan 1, 2021
5dcb915
Merge pull request #161 from nautechsystems/develop
cjdsellers Jan 5, 2021
b053dc4
Add strategy_id.tag to trader.generate_positions_report() for analize…
jpmediadev Jan 6, 2021
a8c6f27
Revert "Add strategy_id.tag to trader.generate_positions_report() for…
jpmediadev Jan 6, 2021
c872bc7
Add strategy_id.tag to trader.generate_positions_report()
Jan 6, 2021
ca215b1
Merge pull request #163 from nautechsystems/develop
cjdsellers Jan 7, 2021
d358495
Merge remote-tracking branch 'upstream/develop' into develop
Jan 7, 2021
eea5f73
Merge pull request #164 from nautechsystems/develop
cjdsellers Jan 7, 2021
a8d4664
Merge remote-tracking branch 'upstream/develop' into develop
Jan 7, 2021
dcc225d
Merge pull request #2 from nautechsystems/develop
jpmediadev Jan 7, 2021
c0582c4
Merge pull request #166 from nautechsystems/develop
cjdsellers Jan 8, 2021
c2256b4
Merge remote-tracking branch 'upstream/develop' into develop
Jan 8, 2021
345e983
Merge pull request #167 from nautechsystems/develop
cjdsellers Jan 9, 2021
815301c
Merge remote-tracking branch 'upstream/develop' into develop
Jan 9, 2021
f7bb51c
Merge pull request #7 from nautechsystems/develop
jpmediadev Jan 9, 2021
2e2f624
add support for Tardis.dev [quote, trade] CSV file format
Jan 9, 2021
f3cda50
Merge branch 'develop' of https://github.com/jpmediadev/nautilus_trad…
Jan 9, 2021
1d0eb94
New line
jpmediadev Jan 9, 2021
855cca3
Merge pull request #170 from nautechsystems/develop
cjdsellers Jan 10, 2021
a0cdce7
Merge pull request #171 from nautechsystems/develop
cjdsellers Jan 10, 2021
c8a697f
* pre-commit fix new line
jpmediadev Jan 10, 2021
6d27267
Merge branch 'master' of https://github.com/nautechsystems/nautilus_t…
jpmediadev Jan 10, 2021
93e05c1
* re-upload broken .gz files
jpmediadev Jan 10, 2021
5742b83
* re-upload broken .gz files
jpmediadev Jan 10, 2021
dd20df9
add uncompressed & truncated csv files
jpmediadev Jan 10, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions nautilus_trader/backtest/loaders.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,11 @@ cdef class CSVTickDataLoader:

cdef class CSVBarDataLoader:
pass


cdef class TardisTradeDataLoader:
pass


cdef class TardisQuoteDataLoader:
pass
62 changes: 62 additions & 0 deletions nautilus_trader/backtest/loaders.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@
import pandas as pd

from nautilus_trader.core.correctness cimport Condition
from cpython.datetime cimport datetime


cdef datetime _ts_parser(str time_in_secs):
return datetime.utcfromtimestamp(int(time_in_secs) / 1_000_000.0)


cdef class CSVTickDataLoader:
Expand Down Expand Up @@ -74,3 +79,60 @@ cdef class CSVBarDataLoader:
index_col="timestamp",
parse_dates=True,
)


cdef class TardisTradeDataLoader:
"""
Provides a means of loading trade data pandas DataFrames from Tardis CSV files.
"""

@staticmethod
def load(str file_path) -> pd.DataFrame:
"""
Return the trade pandas.DataFrame loaded from the given csv file.

Parameters
----------
file_path : str
The absolute path to the CSV file.

Returns
-------
pd.DataFrame

"""
Condition.not_none(file_path, "file_path")

df = pd.read_csv(file_path, index_col='local_timestamp', date_parser=_ts_parser, parse_dates=True)
df.rename(columns={'id': 'trade_id', 'amount': 'quantity'}, inplace=True)
df['side'] = df.side.str.upper()
df = df[['symbol', 'trade_id', 'price', 'quantity', 'side']]
return df


cdef class TardisQuoteDataLoader:
"""
Provides a means of loading quote data pandas DataFrames from Tardis CSV files.
"""

@staticmethod
def load(str file_path) -> pd.DataFrame:
"""
Return the quote pandas.DataFrame loaded from the given csv file.

Parameters
----------
file_path : str
The absolute path to the CSV file.

Returns
-------
pd.DataFrame

"""
Condition.not_none(file_path, "file_path")
df = pd.read_csv(file_path, index_col='local_timestamp', date_parser=_ts_parser, parse_dates=True)
df.rename(columns={'ask_amount': 'ask_size', 'ask_price': 'ask', 'bid_price': 'bid', 'bid_amount': 'bid_size'},
inplace=True)
df = df[['symbol','ask_size','ask','bid_size','bid']]
return df
10 changes: 8 additions & 2 deletions nautilus_trader/data/wrangling.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -300,12 +300,18 @@ cdef class TradeTickDataWrangler:
processed_trades = pd.DataFrame(index=self._data_trades.index)
processed_trades["price"] = self._data_trades["price"].apply(lambda x: f'{x:.{self.instrument.price_precision}f}')
processed_trades["quantity"] = self._data_trades["quantity"].apply(lambda x: f'{x:.{self.instrument.size_precision}f}')
processed_trades["side"] = self._data_trades["buyer_maker"].apply(lambda x: "SELL" if x is True else "BUY")
processed_trades["side"] = self._create_side_if_not_exist()
processed_trades["match_id"] = self._data_trades["trade_id"].apply(str)
processed_trades["symbol"] = symbol_indexer

self.processed_data = processed_trades


def _create_side_if_not_exist(self):
if 'side' in self._data_trades.columns:
return self._data_trades["side"]
else:
return self._data_trades["buyer_maker"].apply(lambda x: "SELL" if x is True else "BUY")

cpdef list build_ticks(self):
"""
Build ticks from all data.
Expand Down
Loading