Invalid MAE value

[shedar]

MAE value is half of the actual value.

E.g.

train_pool = Pool(x_train, y_train, cat_features=cat_indexes)
valid_pool = Pool(x_valid, y_valid, cat_features=cat_indexes)
test_pool = Pool(x_valid, cat_features=cat_indexes)

model = CatBoostRegressor(iterations=10, depth=6, learning_rate=0.03, loss_function='MAE',verbose=True)
model.fit(train_pool, eval_set=valid_pool, verbose=True)
y_pred = model.predict(test_pool)
sk_err = metrics.mean_absolute_error(y_valid, y_pred)
print("sklearn error:", sk_err)

Output is

bestTest = 0.0346194358
bestIteration = 9

sklearn error: 0.0692388715917

CatBoost version: 0.1.1.5
Python: 3.5.2
OS: OS X 10.12.5

[unnir]

but, the mse for valid data, yes?

[Donskov7]

Try to add parameter use_best_model=True to the model and check again.

[shedar]

@unnir, not 100% sure what you mean, please let me know if I answered a wrong question. RMSE calculated by catboost matches with a value I get from sklearn, but MAE - doesn't.

@Donskov7, here's result with use_best_model=True

bestTest = 0.03309215839
bestIteration = 9

Shrink model to first 10 iterations.
sklearn error: 0.0661843167294

[demikandr]

MAE loss in catboost is actually MAE/2.

It is so because internally it's a Quantile loss with alpha=0.5.

[annaveronika]

Issue MATRIXNET-758

[annaveronika]

This is fixed now, try updating to the last version.