Different performance of the algorithm depending on input data type (np.ndarray or pd.DataFrame)

[lccol]

I encountered a performance issue (in terms of clustering results) depending on the type of input data.

In particular, if train_data is np.ndarray, the predicted cluster labels are incorrect.

Example to reproduce the issue (starting from 2D shapes datasets.ipynb):

• Working version (using pd.DataFrame as input data type for train_data):

data_path = 'datasets/denmune/shapes/'  
# datasets = {"aggregation": 6, "jain": 15, "flame": 8, "compound": 13, "varydensity": 23,
            # "unbalance": 8, "spiral": 6, "pathbased": 6, "mouse": 11}

datasets = {'aggregation': 6}

for dataset in datasets:
  data_file = data_path + dataset + '.csv'
  X_train = pd.read_csv(data_file, sep=',', header=None)
  y_train = X_train.iloc[:, -1]
  X_train = X_train.drop(X_train.columns[-1], axis=1)   
 
  print ("Dataset:", dataset)
  dm = DenMune(train_data=X_train,
              train_truth=y_train,
              k_nearest=datasets[dataset],
              rgn_tsne=False)

  labels, validity = dm.fit_predict(show_noise=True, show_analyzer=True)

• Version with performance issue (using np.ndarray as input data type for train_data):

data_path = 'datasets/denmune/shapes/'  
# datasets = {"aggregation": 6, "jain": 15, "flame": 8, "compound": 13, "varydensity": 23,
            # "unbalance": 8, "spiral": 6, "pathbased": 6, "mouse": 11}

datasets = {'aggregation': 6}

for dataset in datasets:
  data_file = data_path + dataset + '.csv'
  X_train = pd.read_csv(data_file, sep=',', header=None)
  y_train = X_train.iloc[:, -1]
  X_train = X_train.drop(X_train.columns[-1], axis=1)   
  X_train = X_train.values
 
  print ("Dataset:", dataset)
  dm = DenMune(train_data=X_train,
              train_truth=y_train,
              k_nearest=datasets[dataset],
              rgn_tsne=False)

  labels, validity = dm.fit_predict(show_noise=True, show_analyzer=True)

[egy1st]

I encountered a performance issue (in terms of clustering results) depending on the type of input data.

In particular, if train_data is np.ndarray, the predicted cluster labels are incorrect.

Example to reproduce the issue (starting from 2D shapes datasets.ipynb):

• Working version (using pd.DataFrame as input data type for train_data):

data_path = 'datasets/denmune/shapes/'  
# datasets = {"aggregation": 6, "jain": 15, "flame": 8, "compound": 13, "varydensity": 23,
            # "unbalance": 8, "spiral": 6, "pathbased": 6, "mouse": 11}

datasets = {'aggregation': 6}

for dataset in datasets:
  data_file = data_path + dataset + '.csv'
  X_train = pd.read_csv(data_file, sep=',', header=None)
  y_train = X_train.iloc[:, -1]
  X_train = X_train.drop(X_train.columns[-1], axis=1)   
 
  print ("Dataset:", dataset)
  dm = DenMune(train_data=X_train,
              train_truth=y_train,
              k_nearest=datasets[dataset],
              rgn_tsne=False)

  labels, validity = dm.fit_predict(show_noise=True, show_analyzer=True)

• Version with performance issue (using np.ndarray as input data type for train_data):

data_path = 'datasets/denmune/shapes/'

datasets = {"aggregation": 6, "jain": 15, "flame": 8, "compound": 13, "varydensity": 23,

        # "unbalance": 8, "spiral": 6, "pathbased": 6, "mouse": 11}

datasets = {'aggregation': 6}

for dataset in datasets:
data_file = data_path + dataset + '.csv'
X_train = pd.read_csv(data_file, sep=',', header=None)
y_train = X_train.iloc[:, -1]
X_train = X_train.drop(X_train.columns[-1], axis=1)
X_train = X_train.values

print ("Dataset:", dataset)
dm = DenMune(train_data=X_train,
train_truth=y_train,
k_nearest=datasets[dataset],
rgn_tsne=False)

labels, validity = dm.fit_predict(show_noise=True, show_analyzer=True)

I will check it once I come back on Friday

[egy1st]

@lccol Please note the X_train.values is an array of array while X_train is a 2-d array
to fix this issue just remove the the command:
X_train = X_train.values

[lccol]

I checked by adding the following print statements:

...
  X_train = X_train.values

  print(type(X_train))
  print(X_train.shape)
  print(X_train)
...

The result is the following:

<class 'numpy.ndarray'>
(788, 2)
[[15.55 28.65]
 [14.9  27.55]
 [14.45 28.35]
 ...
 [ 8.5   3.25]
 [ 8.1   3.55]
 [ 8.15  4.  ]]

As can be seen, the result of .values (which is the same as .to_numpy()) is the corresponding numpy representation of the DataFrame, i.e., a numpy array of shape (n_points, 2). The returned value is a single 2-d array as can be seen from X_train.shape.

The original X_train is instead a pandas DataFrame, for which I report here the same three resulting print statements.

<class 'pandas.core.frame.DataFrame'>
(788, 2)
         0      1
0    15.55  28.65
1    14.90  27.55
2    14.45  28.35
3    14.15  28.80
4    13.75  28.05
..     ...    ...
783   7.80   3.35
784   8.05   2.75
785   8.50   3.25
786   8.10   3.55
787   8.15   4.00

[788 rows x 2 columns]

I reported the issue since, being the methods similar to the sklearn's APIs (fit_predict), I suppose that the majority of users would interchangeably use DataFrames and numpy arrays (like I do, since sklearn-based classes accepts both).

I noticed degraded performances in my experiments using DenMune and by passing input data using numpy arrays instead of pandas DataFrame (adjusted rand scores < 0.1 over 20 different datasets).

[egy1st]

@lccol I see. That is due the fact of working with contiguus array vs faltten ones. I amnaged that issue in the latest update
please make sure to install latest version

pip install denmune==0.1.0