Error with Forecaster Attribute 'validation_metric_value'

[JPVercosa]

Hey, I've just started using Scalecast, and was trying to use the RandomForest Model. After initializing Forecaster and adding COVID19 Regressor and Time Trend, I got the error:

AttributeError: 'Forecaster' object has no attribute 'validation_metric_value'

Here is the code that I'm running:

def plot_test_export_summaries(f):
    """ exports the relevant statisitcal information and displays a plot of the test-set results for the last model run
    """
    f.plot_test_set(models=f.estimator,include_train=False)
    plt.title(f'{f.estimator} test-set results',size=16)
    plt.show()
    return f.export('model_summaries',determine_best_by='TestSetMAPE')[
        [
            'ModelNickname',
            'HyperParams',
            'TestSetMAPE',
            'InSampleMAPE',
        ]
    ]

icms = Forecaster(y=df.ICMS, current_dates=df.DATA, test_length=12, freq='MS')
icms.generate_future_dates(12)
icms.plot()

icms.add_covid19_regressor()
icms.add_time_trend()

icms.set_estimator('rf')
rf_grid = {
    'max_depth':[2,3,4,5],
    'n_estimators':[100,200,500],
    'max_features':['auto','sqrt','log2'],
    'max_samples':[.75,.9,1],
}
icms.ingest_grid(rf_grid)
icms.auto_forecast()
plot_test_export_summaries(icms)

Full error:

I'm trying to figure out what is the problem, if anyone have any suggestion, I would aprecciate.

I still want to use other models ('xgboost', 'LSTM' and 'prophet'), but I get the same error. Am I missing something?

[mikekeith52]

Hi,

Before calling icms.auto_forecast() and after running icms.ingest_grid(), allow the object to select the optimal hyperparameters by calling either icms.tune() or icms.cross_validate(). This should avoid the error.

Next scalecast update, I will make sure there is a more descriptive error message or warning when this happens.

Thanks for raising the issue!

[JPVercosa]

Thank you, it solved the problem.