Lots of warning when running example.

[r06922112]

I try to running the following example:

  X, y = sklearn.datasets.load_digits(return_X_y=True)
  X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
  automl = autosklearn.classification.AutoSklearnClassifier()
  automl.fit(X_train, y_train)
  y_hat = automl.predict(X_test)
  print("Accuracy score", sklearn.metrics.accuracy_score(y_test, y_hat))

It should be done in about one hour, but it didn't finish after one and half hours,
and keep printing the warning as following:

/home/<mydir>/miniconda3/lib/python3.6/site-packages/autosklearn/pipeline/create_searchspace_util.py:228:
FutureWarning:
Using a non-tuple sequence for multidimensional indexing is deprecated; 
use `arr[tuple(seq)]` instead of `arr[seq]`.
In the future this will be interpreted as an array index, 
`arr[np.array(seq)]`, which will result either in an error or a different result.

/home/<mydir>/miniconda3/lib/python3.6/site-packages/autosklearn/evaluation/train_evaluator.py:197
: RuntimeWarning: Mean of empty slice

My system details:
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

auto-sklearn              0.4.0 
scikit-learn              0.20.0 
scipy                     1.1.0 
xgboost                   0.7.post3 
numpy                     1.15.2 

[mfeurer]

Thank you for reporting this. The warning are an issue of numpy 1.15 and will be removed with the next release. However, Auto-sklearn should not work with scikit-learn 0.20. Please downgrade to 0.19 to use it and create a new issue if Auto-sklearn still runs over time.

[seancaodo]

Hi @mfeurer ,

I don't think I need to reopen this but I'm having below warnings from running the exact example given above:

/..../python3.6/site-packages/autosklearn/evaluation/train_evaluator.py:197: RuntimeWarning: Mean of empty slice
Y_train_pred = np.nanmean(Y_train_pred_full, axis=0)

Below is the module versions:

The warning suggests that the array of prediction values are all empty hence the numpy error. Does this then suggest that the model wasn't built therefore Y_train_pred_full couldn't be calculated hence the error? The example would still run to have a final model and came back with an accuracy rate reported.

Should I be worried about this RunTimeWarning or it's part of expected warnings in a autosklearn run?

Many thanks,

S Do

[seancaodo]

Looks like it's the same as #600

[mfeurer]

Indeed, it's the same as #600. Ab fix is underway in #679.