Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stress errors reported during fitting and at end should be consistent #442

Closed
bernstei opened this issue Jun 4, 2024 · 5 comments
Closed
Labels
bug Something isn't working

Comments

@bernstei
Copy link
Collaborator

bernstei commented Jun 4, 2024

The error report during fitting reports RMSE_stress_per_atom, and it clearly differs in value from the RMSE Stress (Virials) / meV / A (A^3) column of the final report table (see below). I think we should just stop ever reporting (by default, at least) single atom stress contributions, because they seem to be normalized very weirdly (i.e. divided by total volume, so they're sub-intensive - only the sum over all atoms is intensive). Also, calling them "per atom" is confusing to me, as I assume that means "divided by n_atoms", rather than from each atom.

2024-05-31 17:58:38.255 INFO: head: pbe_mp, Epoch 475: loss=0.0024, RMSE_E_per_atom=53.2 meV, RMSE_F=230.2 meV / A, RMSE_stress_per_atom=2.8 meV / A^3
2024-05-31 17:58:39.544 INFO: head: Default, Epoch 475: loss=0.0027, RMSE_E_per_atom=2.3 meV, RMSE_F=99.3 meV / A, RMSE_stress_per_atom=0.5 meV / A^3

vs.

2024-05-31 19:06:24.127 INFO: Loaded model from epoch 475
2024-05-31 19:06:24.127 INFO: Evaluating train ...
2024-05-31 19:06:34.452 INFO: Evaluating Default ...
2024-05-31 19:06:35.808 INFO: Evaluating pbe_mp ...
2024-05-31 19:06:36.454 INFO: 
+-------------+---------------------+------------------+-------------------+---------------------------------------+
| config_type | RMSE E / meV / atom | RMSE F / meV / A | relative F RMSE % | RMSE Stress (Virials) / meV / A (A^3) |
+-------------+---------------------+------------------+-------------------+---------------------------------------+
|    train    |         4.8         |       51.0       |        4.63       |                  6.3                  |
|   Default   |         2.3         |       99.3       |        9.66       |                  5.1                  |
|    pbe_mp   |         53.2        |      230.2       |       16.09       |                  14.5                 |
+-------------+---------------------+------------------+-------------------+---------------------------------------+
@gabor1
Copy link
Collaborator

gabor1 commented Jun 4, 2024

Is it just the name? That this is really virial/atom?

@bernstei
Copy link
Collaborator Author

bernstei commented Jun 4, 2024

Is it just the name? That this is really virial/atom?

If so then the printed units are wrong, so that needs to be fixed instead (using consistent units)

@bernstei
Copy link
Collaborator Author

bernstei commented Jun 4, 2024

Pretty sure it's

error_stress = eval_metrics["rmse_stress_per_atom"] * 1e3
and it appears that it really is per atom (based on what update() in the same file does in

mace/mace/tools/train.py

Lines 423 to 425 in dee204f

self.delta_stress_per_atom.append(
(batch.stress - output["stress"])
/ (batch.ptr[1:] - batch.ptr[:-1]).view(-1, 1, 1)
), not just the contribution from each atom (i.e. the site stress contribution).

But I think we'll need @ilyes319 to confirm. And I'd strongly urge never to store an actual total stress / n_atoms. That's just not meaningful.

@bernstei
Copy link
Collaborator Author

bernstei commented Jun 4, 2024

See #443

@ilyes319 ilyes319 closed this as completed Jun 6, 2024
@ilyes319 ilyes319 reopened this Jun 6, 2024
@ilyes319 ilyes319 added the bug Something isn't working label Jun 21, 2024
@ilyes319
Copy link
Contributor

ilyes319 commented Sep 4, 2024

this is fixed in dev

@ilyes319 ilyes319 closed this as completed Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants