ChapmanKolmogorovValidator loses 'has_errors' attribute upon saving/loading

[jeiros]

I'm doing some CK tests and saving them to disk as pickle files for visualization purposes.
I get the following error when I try to plot a CK test that has been previously saved to disk:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-22-86b859a92999> in <module>()
----> 1 plot_cktest(ck_loaded)

~/miniconda3/lib/python3.6/site-packages/pyemma/plots/markovtests.py in plot_cktest(cktest, figsize, diag, y01, layout, padding_between, padding_top, units, dt, **plot_kwargs)
    136             i = int(k/cktest.nsets)
    137             j = int(k%cktest.nsets)
--> 138             lest, lpred = _add_ck_subplot(cktest, ax, i, j, y01=y01, units=units, dt=dt, **plot_kwargs)
    139     # figure legend
    140     predlabel = 'predict'

~/miniconda3/lib/python3.6/site-packages/pyemma/plots/markovtests.py in _add_ck_subplot(cktest, ax, i, j, ipos, jpos, y01, units, dt, **plot_kwargs)
     33     lest = ax.plot(dt*cktest.lagtimes, cktest.estimates[:, i, j], color='black', **plot_kwargs)
     34     # plot error of estimates if available
---> 35     if cktest.has_errors and cktest.err_est:
     36         ax.fill_between(dt*cktest.lagtimes, cktest.estimates_conf[0][:, i, j], cktest.estimates_conf[1][:, i, j],
     37                         color='black', alpha=0.2)

AttributeError: 'ChapmanKolmogorovValidator' object has no attribute 'has_errors'

Here are the steps to reproduce:

from pyemma.msm import BayesianMSM, ChapmanKolmogorovValidator
from pyemma.plots import plot_cktest
lag_time = 25
n_macrostates = 5
mlags = 6

msm = BayesianMSM(lag=lag_time, dt_traj='0.2 ns')
msm.fit(list(ktrajs.values()))  # ktrajs is a dict of lists with integers

ck = msm.cktest(
    n_macrostates,
    err_est=True,
    n_jobs=-1,
    mlags=mlags
)
plot_cktest(ck) # This works
ck.save('ck_testing.pkl', model_name='model')

ck_loaded = ChapmanKolmogorovValidator.load('ck_testing.pkl', 'model')

plot_cktest(ck_loaded) # This gives the above error

ck.has_errors # This is True

ck_loaded.has_errors

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-28-78c530e7ce04> in <module>()
----> 1 ck_loaded.has_errors

AttributeError: 'ChapmanKolmogorovValidator' object has no attribute 'has_errors'

I'm on pyemma version 2.5.2, OS is macOS High Sierra v.10.13.3

here's the output of my conda list

[marscher]

Hi, thanks for your report! Sorry about that, you can fix your restored models in the mean time (we can provide a fix for this, which will also makes this work with future version) by doing this:

ck = pyemma.load('...')
ck.has_errors = issubclass(ck.test_model.__class__, SampledModel)

[jeiros]

Hi @marscher , thanks for your help. What is SampledModel though?

[marscher]

On 04/23/2018 04:47 PM, Juan Eiros wrote: Hi @marscher <https://github.com/marscher> , thanks for your help. What is |SampledModel| though?

sorry you need to import it from here like this to make the snippet work: from pyemma._base.model import SampledModel

[marscher]

please also assert that the confidence interval of the test_model is equal to the one in ck to avoid strange results. But is should hold - I guess.

[jeiros]

Ahhh, I see. Yeah, the plots look the same when doing it directly or saving/loading. Thanks for the help!