Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance tweaks #7

Open
8 tasks
untoreh opened this issue Jan 27, 2023 · 1 comment
Open
8 tasks

Performance tweaks #7

untoreh opened this issue Jan 27, 2023 · 1 comment

Comments

@untoreh
Copy link
Collaborator

untoreh commented Jan 27, 2023

backtest

  • There are many repeated function calls like openat,closeat,etc... that could be memoized
  • Consider either removing getproperty for asset instances and strategies or replace all dot accessors within SimMode with some getfield wrapper function. Because the getproperty accessor is slower than getfield if it is customized
  • an alternative implementation for orders/trades would use arrays instead of structs to be more allocation friendly

watchers

  • StructArrays or TypedTables could be used when fetching (streaming) ohlcv data feeds
  • OHLCV watcher implementations parse the whole python dicts into named tuples (CcxtTrade, CcxtTicker), in cases where full parsing is not required, the python data should be parsed more lazily

other

  • Scour the codebase for lazy use of @view/s on assignments, that should be replaced with copyto! (starting from Data)
  • search/replace Period with FixedPeriod to see if union splitting improves performance
  • Full specialization: many functions signatures are defined over generic types (non fully parametrized), benchmarks are needed to assert if full specialization is worth the extra compilation time / bin size
@panbonker
Copy link
Collaborator

Backtester is fast, but produces a lot of garbage, which forces short, but not negligible hiccups during multiple backtesting runs, so here is a possible solution for a garbage minimize events structure:

  • Replace trades/orders structs with either smaller structs, or even just tuples
  • the tuples only hold 2 elements: a ref to journal (structarray) the index of a row in such journal
  • the row of the journal would be the actual trade/order
  • for trades the table should be append-only (but that can still be empty!ed, just no operations like pop! or insert!)
  • for orders with might use one append-only table for closed orders and one fully writeable for pending orders.
  • Creating an order/trade then requires providing the table as argument, the strategy should hold the journals (tables) for both trades and orders. We loose the fine-grained per asset order history, but a filter function over the table achieves the same and should be fast.
  • logs should also just be a table.
  • declare function to access fields for all existing structure fields (and implement getproperty that forwards correctly) to avoid breaking anything that relies on the structure fields
  • During backtesting use sizehint! to correctly size the journals, which should avoid constant de/allocations for trades/orders, or at least minimize them since the trades/orders structures would be ephemereal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants