Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classification - add test-indices to summary #22

Closed
katharina-j-fau opened this issue Jan 14, 2022 · 1 comment
Closed

Classification - add test-indices to summary #22

katharina-j-fau opened this issue Jan 14, 2022 · 1 comment
Labels
feature-request New feature or request

Comments

@katharina-j-fau
Copy link
Collaborator

  1. In nested_cv.py nested_cv_param_search() add train and test indices to cols and results_dict:
cols = [
        "param_search",
        "cv_results",
        "best_estimator",
        "conf_matrix",
        "predicted_labels",
        "true_labels",
        "train_indices",      #add this line
        "test_indices"        #add this line
    ]
results_dict["train_indices"].append(train)     #add this line
results_dict["test_indices"].append(test)       #add this line
results_dict["predicted_labels"].append(cv_obj.predict(x_test))
results_dict["true_labels"].append(y_test)
results_dict["cv_results"].append(cv_obj.cv_results_)
results_dict["best_estimator"].append(cv_obj.best_estimator_)
results_dict["conf_matrix"].append(confusion_matrix(y_test, cv_obj.predict(x_test), normalize=None))
  1. in sklearn_pipeline_permuter.py metric_summary() get test indices and add it to df_metric:
for param_key, param_value in self.param_searches.items():
      ...
      test_indices = np.array(param_value["test_indices"], dtype="object").ravel()
      ...
      df_metric["test_indices"] = [test_indices]
      
      for key in param_values:
            if "test" in key:
                  if "test_indices" in key:
                        continue
  1. Optional: Find a more elegant way to exclude test_indices from metric calculation ;)
@katharina-j-fau katharina-j-fau added the feature-request New feature or request label Jan 14, 2022
@richrobe
Copy link
Member

richrobe commented Apr 21, 2022

Added in cc3cd86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants