Skip to content

Fix sweep to keep the best model and add best_score of the first model #371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 14, 2024
Merged

Fix sweep to keep the best model and add best_score of the first model #371

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1e178c1
Fix sweep to keep the best model and add best_score of the first model
ProgramadorArtificial Jan 13, 2024
8524970
Merge branch 'main' into bugfix/sweep_best_model
manujosephv Jan 14, 2024
42e95cb
Remove datamodule before deepcopy best_model
ProgramadorArtificial Jan 14, 2024
f13a0e3
Add sanity check
ProgramadorArtificial Jan 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 20 additions & 16 deletions src/pytorch_tabular/tabular_model_sweep.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,27 +359,31 @@ def _init_tabular_model(m):
res_dict["time_taken"] = time.time() - start_time
res_dict["time_taken_per_epoch"] = res_dict["time_taken"] / res_dict["epochs"]

if verbose:
logger.info(f"Finished Training {name}")
logger.info("Results:" f" {', '.join([f'{k}: {v}' for k,v in res_dict.items()])}")
res_dict["params"] = params
results.append(res_dict)
if best_model is None:
best_model = tabular_model
else:
if is_lower_better:
if res_dict[f"test_{rank_metric[0]}"] < best_score:
best_model = tabular_model
best_score = res_dict[f"test_{rank_metric[0]}"]
else:
if res_dict[f"test_{rank_metric[0]}"] > best_score:
best_model = tabular_model
if verbose:
logger.info(f"Finished Training {name}")
logger.info("Results:" f" {', '.join([f'{k}: {v}' for k,v in res_dict.items()])}")
res_dict["params"] = params
results.append(res_dict)
if return_best_model:
tabular_model.datamodule = None
if best_model is None:
best_model = copy.deepcopy(tabular_model)
best_score = res_dict[f"test_{rank_metric[0]}"]
else:
if is_lower_better:
if res_dict[f"test_{rank_metric[0]}"] < best_score:
best_model = copy.deepcopy(tabular_model)
best_score = res_dict[f"test_{rank_metric[0]}"]
else:
if res_dict[f"test_{rank_metric[0]}"] > best_score:
best_model = copy.deepcopy(tabular_model)
best_score = res_dict[f"test_{rank_metric[0]}"]
if verbose:
logger.info("Model Sweep Finished")
logger.info(f"Best Model: {best_model.name}")
results = pd.DataFrame(results).sort_values(by=f"test_{rank_metric[0]}", ascending=is_lower_better)
if return_best_model:
if return_best_model and best_model is not None:
best_model.datamodule = datamodule
return results, best_model
else:
return results