This issue was found by a Codex global repository scan of tracked non-test files at commit 8c93925cb10b401b2b83c738bd9263fd74474468.
Relevant code
|
result_df["type"] = result_df["traj"].apply(lambda x: x.split("_")[0]) |
|
result_df["error"] = result_df.apply( |
|
lambda x: np.abs(x["pred_Ea"] - x["Ea"]), axis=1 |
|
) |
|
type_percentages = ( |
|
result_df[result_df["error"] > ERROR_THRESHOLD].groupby("type").size() |
|
/ result_df.groupby("type").size() |
|
* 100 |
|
) |
|
type_percentages = type_percentages.round(2) |
|
# update key names |
|
type_percentages.rename( |
|
index={ |
|
"desorption": "ø_Desorption", |
|
"dissociation": "ø_Dissociation", |
|
"transfer": "ø_Transfer", |
|
}, |
|
inplace=True, |
|
) |
|
|
|
results = type_percentages.to_dict() |
|
result_df.dropna(inplace=True) |
|
results["success_rate"] = len(result_df) / NUM_RECORDS * 100 |
|
results["MAE_Ea"] = mean_absolute_error(result_df["Ea"], result_df["pred_Ea"]) |
|
results["MAE_dE"] = mean_absolute_error(result_df["dE"], result_df["pred_dE"]) |
Impact
The NEB failure percentage is computed from only the rows where error > ERROR_THRESHOLD. If a reaction type has zero failures, that type is absent from the numerator groupby, so the division can produce a missing value instead of an explicit 0.0 failure rate.
A perfect category should be represented as 0.0, not as a missing metric. Otherwise downstream JSON and leaderboard plots can treat a valid result as absent.
Suggested fix
Build the denominator counts first, reindex the failure counts against those types with fill_value=0, then divide. Add a regression case where one NEB type has no failures.
This issue was found by a Codex global repository scan of tracked non-test files at commit
8c93925cb10b401b2b83c738bd9263fd74474468.Relevant code
LAMBench/lambench/tasks/calculator/neb/neb.py
Lines 62 to 86 in 8c93925
Impact
The NEB failure percentage is computed from only the rows where
error > ERROR_THRESHOLD. If a reaction type has zero failures, that type is absent from the numerator groupby, so the division can produce a missing value instead of an explicit0.0failure rate.A perfect category should be represented as
0.0, not as a missing metric. Otherwise downstream JSON and leaderboard plots can treat a valid result as absent.Suggested fix
Build the denominator counts first, reindex the failure counts against those types with
fill_value=0, then divide. Add a regression case where one NEB type has no failures.