MarketData <> prepare_data

[quant5]

Seems there is a deprecation in progress:

cvxportfolio/cvxportfolio/simulator.py

Line 206 in df69302

class MarketData:

is being deprecated in favor of

cvxportfolio/cvxportfolio/simulator.py

Line 555 in df69302

def prepare_data(self):

Is this intended to be a full deprecation? If so I would note the below (understand some of these are WIP):

• some methods (e.g., add_cash_column and remove_missing_recent are not carried over - should they be?
• the methods serve_data_policy and serve_data_simulator would be deprecated - looks like this would impact on how the various CostSimulator's would work - as compute_cost relies on current_prices, current_and_past_volumes, current_and_past_returns right?
• these tests will no longer be relevant / need to be updated: https://github.com/cvxgrp/cvxportfolio/blob/df6930289748dfa3d8c855175d0b29d57d891fc4/cvxportfolio/tests/test_simulator.py

Apologies if I keep bumping onto these things because I'm pulling from the "bleeding edge" but as a general note, I'd say there are a fair amount of WIP elements in the master branch such that it becomes a bit hard to collaborate / stably use the latest pull. Obviously I'm glomming onto the repo unannounced, but I'd be an advocate for higher standards for pushing to master (more than happy to collaborate on what they'd be).

[enzbus]

No, is the other way round, right now only the methods inside marketsimulator are being used and I'm building outside of it their replacements with some improvements. Everything is tested (both old and new). I didn't touch the old pieces as I am building the new so no disruption there. The new pieces (marketdata etc.) are 100% tested but not currently plugged in the main api. Git branches are an overkill for this project at the moment.

[enzbus]

Also, sorry I didn't mean to sound dismissive of your comment, you're right that it's not great to have code duplication in the master branch (I wrote the new stuff on Monday and I thought to merge earlier but will probably have to wait for the weekend). I can share what I have in mind. MarketSimulator.__init__ takes a list of cost functions (currently I wrote classes that implement a single method but will be simple callbacks instead) and **kwargs that are passed to them (e.g., linear_transaction_cost, cash_borrowing_spread, ...). MarketData holds and serve data to both the simulator and and the policy and is also initialized by the simulator with the optional **kwargs. MarketSimulator becomes a very thin class almost stateless that implements the simulate and backtest methods, also BacktestResult will change and take on a few things that are currently done inside MarketSimulator. Main issue is about multiprocessing, it's very tricky to pass classes with lots of internal state through the interprocess communication system (they all get pickled and unpickled on the other end) so I need to make sure that classes have as little state as possible. In fact MarketSimulator.backtest will only implement logic to initialize a backtest (and break it up into smaller backtests as the universe changes with IPOs or bankruptcies, cfr. #85) and then call a separate _backtest function, which is the one that is sent to multiprocessing. Again, the goal is to have a final product that is both easy to use (expose to the user a clear object model, not a bunch of functions), easy to extend (user-defined extensions should be easy to write) and easy to read and debug. Also, since everything goes through multiprocessing, at the lower level it should ideally call functions or near-stateless classes' methods).

[enzbus]

BTW, the default UX will be like what is in the examples now, i.e.

results = MarketSimulator(['AAPL', 'GOOG', 'AMZN']).backtest([strategy1, strategy2], '2000-01-01')
print(results)
results.plot()

(This is not the syntax now.)

[quant5]

Ah, it's my bad then, I didn't realize that MarketData was new and should have checked the history not just the current code.
It's just that things like MarketData being unused confused me.
I think the proposed class structure (MarketData servicing both the policy and the simulator) makes a lot of sense. And can perhaps solve the problem with nan assets (#85) in the same go.
I will continue to get up to speed with the codebase.

[quant5]

Follow-up question on multiprocessing - can you help me understand how you envision a fully multiprocessing compatible backtest, as it was my understanding that backtesting on time t requires state knowledge of holdings for t-1?

[enzbus]

Multiprocessing is used when doing multiple backtests (which will be largely automated once we make the hyperparameter optimization as part of the simulator logic). This was already implemented in old cvxportfolio (branch 0.0.X) and used in the example notebooks. It gets trickier with thicker classes, which is the reason why I'm refactoring the simulator logic now. Also we used to use an external multiprocess library, instead I moved to the standard library multiprocessing module which is less forgiving (but with better support). The problem with multiprocessing is that is hard to debug, hence code needs to be crystal clear. You can't really parallelize a single backtest because it is path-dependent (even without tcosts). All cvx* stack is mostly single-threaded (apart from a few BLAS level3 / Lapack calls which are very rare) so multiprocessing is a great fit. Also I'm not closing some old issues because they contain stuff that needs to be processed (e.g., in this one there is a lot of text that needs to go in the docs).

[enzbus]

BTW, the above is true for optimization-based strategies, for toy strategies easier parallelization might work. Everything in cvxportfolio maps the content of the book, the dynamics is explained in Chapter 2. The dynamics and parallelization are the same as they have always been in cvxportfolio since the initial 2016 release.