Bayessian analysis doesn't seem to work

[mmisup]

Hello,

I'm following the Pyfolio tutorials from Quantopian website, I was able to run the Single Stock Analysis but when I try to run the Bayessian Analysis I always get errors and not graphic is shown (I'm using exactly the examples from the tutorials, except that I read the stock data from a csv file because the tutorial instruction for getting the data crashes).

I'm using Python 3.5 on Ubuntu 18.04 with zipline and pyfolio installed in a conda environment. When I've installed pyfolio[bayessian] I got a warning about "arviz 0.7.0 has requirement pandas>=0.23, but you'll have pandas 0.22.0 which is incompatible" but zipline doesn't work with pandas > 0.22!

This is the error I get when I run the bayessian analysis example:

Running T model
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (2 chains in 2 jobs)
NUTS: [nu_minus_two, volatility, mean returns]
Sampling 2 chains, 0 divergences: 100%|██████████| 5000/5000 [00:08<00:00, 576.24draws/s] 
The acceptance probability does not match the target. It is 0.9426219264131023, but should be close to 0.8. Try to increase the number of tuning steps.
The acceptance probability does not match the target. It is 0.8911002054428593, but should be close to 0.8. Try to increase the number of tuning steps.
100%|██████████| 2000/2000 [00:06<00:00, 318.30it/s]

Finished T model (required 22.12 seconds).

Running BEST model
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (2 chains in 2 jobs)
NUTS: [nu_minus_two, group2_std, group1_std, group2_mean, group1_mean]
Sampling 2 chains, 0 divergences: 100%|██████████| 5000/5000 [00:14<00:00, 353.76draws/s]
The acceptance probability does not match the target. It is 0.9047599837651139, but should be close to 0.8. Try to increase the number of tuning steps.
The acceptance probability does not match the target. It is 0.9022099136930037, but should be close to 0.8. Try to increase the number of tuning steps.

Finished BEST model (required 20.64 seconds).
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-f779f7a8ad16> in <module>
      2 
      3 out_of_sample = stock_rets.index[-10]
----> 4 pf.create_bayesian_tear_sheet(stock_rets, live_start_date=out_of_sample)

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pyfolio/plotting.py in call_w_context(*args, **kwargs)
     50         if set_context:
     51             with plotting_context(), axes_style():
---> 52                 return func(*args, **kwargs)
     53         else:
     54             return func(*args, **kwargs)

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pyfolio/tears.py in create_bayesian_tear_sheet(returns, benchmark_rets, live_start_date, samples, return_fig, stoch_vol, progressbar)
   1156     # Plot Bayesian cone
   1157     ax_cone = plt.subplot(gs[row, :])
-> 1158     bayesian.plot_bayes_cone(df_train, df_test, ppc_t, ax=ax_cone)
   1159     previous_time = timer("plotting Bayesian cone", previous_time)
   1160 

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pyfolio/bayesian.py in plot_bayes_cone(returns_train, returns_test, ppc, plot_train_len, ax)
    624         ppc,
    625         plot_train_len=plot_train_len,
--> 626         ax=ax)
    627     ax.text(
    628         0.40,

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pyfolio/bayesian.py in _plot_bayes_cone(returns_train, returns_test, preds, plot_train_len, ax)
    496     perc = compute_bayes_cone(preds, starting_value=returns_train_cum.iloc[-1])
    497     # Add indices
--> 498     perc = {k: pd.Series(v, index=returns_test.index) for k, v in perc.items()}
    499 
    500     returns_test_cum_rel = returns_test_cum

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pyfolio/bayesian.py in <dictcomp>(.0)
    496     perc = compute_bayes_cone(preds, starting_value=returns_train_cum.iloc[-1])
    497     # Add indices
--> 498     perc = {k: pd.Series(v, index=returns_test.index) for k, v in perc.items()}
    499 
    500     returns_test_cum_rel = returns_test_cum

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    264                                        raise_cast_failure=True)
    265 
--> 266                 data = SingleBlockManager(data, index, fastpath=True)
    267 
    268         generic.NDFrame.__init__(self, data, fastpath=True)

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pandas/core/internals.py in __init__(self, block, axis, do_integrity_check, fastpath)
   4400         if not isinstance(block, Block):
   4401             block = make_block(block, placement=slice(0, len(axis)), ndim=1,
-> 4402                                fastpath=True)
   4403 
   4404         self.blocks = [block]

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pandas/core/internals.py in make_block(values, placement, klass, ndim, dtype, fastpath)
   2955                      placement=placement, dtype=dtype)
   2956 
-> 2957     return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
   2958 
   2959 # TODO: flexible with index=None and/or items=None

~/miniconda3/envs/env_pyfolio/lib/python3.5/site-packages/pandas/core/internals.py in __init__(self, values, placement, ndim, fastpath)
    118             raise ValueError('Wrong number of items passed %d, placement '
    119                              'implies %d' % (len(self.values),
--> 120                                              len(self.mgr_locs)))
    121 
    122     @property

ValueError: Wrong number of items passed 1900, placement implies 10

[ssanderson]

hey @mmisup. Sorry for the rough experience here. The bayesian module has been removed in the latest version of PyFolio because it was unmaintained. See #608 for more details.

Unfortunately the Quantopian tutorials haven't been updated to reflect that update. I've bumped an issue on our internal issue tracker for updating those tutorials.

[mmisup]

Thanks for the quick feedback.