Training NEP for mixed dataset

[artempi]

I am training NEP with a dataset that has both bulk structures, as well as some oxide structures. It seems that the training goes fine in the beginning, however, both F-train and F-test begin to increase close to the end of the training process.
I have uploaded my files into https://github.com/CUANTAM/NEP-Training

My guess is that I may need a larger basis size, maybe 15 15 instead of 12 12.
version 4
type 5 Hf O Si W Zr
cutoff 5 5 #
n_max 12 6 #
basis_size 12 12 #
l_max 4 #
neuron 40 #
lambda_1 0.05 #
lambda_2 0.05 #
population 50 #
batch 1000 #
generation 200000 #

[brucefan1983]

The major problem is that you have too weak regularization. I suggest you start from the default settings, just writing a single line: type 5 Hf O Si W Zr in nep.in Zheyong

…

On Wed, Dec 27, 2023 at 6:01 AM artempi ***@***.***> wrote: I am training NEP with a dataset that has both bulk structures, as well as some oxide structures. It seems that the training goes fine in the beginning, however, both F-train and F-test begin to increase close to the end of the training process. I have uploaded my files into https://github.com/CUANTAM/NEP-Training LossOUT.png (view on web) <https://github.com/brucefan1983/GPUMD/assets/14337432/2deddce4-9105-4459-bcff-f3093b4d466d> My guess is that I may need a larger basis size, maybe 15 15 instead of 12 12. version 4 type 5 Hf O Si W Zr cutoff 5 5 # n_max 12 6 # basis_size 12 12 # l_max 4 # neuron 40 # lambda_1 0.05 # lambda_2 0.05 # population 50 # batch 1000 # generation 200000 # — Reply to this email directly, view it on GitHub <#540>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF546OLKJOH5D6CTW2XBLZDYLNCLRAVCNFSM6AAAAABBDU4WIWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2TMNZTGE4DIOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[artempi]

@brucefan1983
I was able to get smaller F-test errors with the default settings.
Is there a way to improve further with custom settings and perhaps including ZBL?
Thank you

Input or default parameters:
(default) model_type = potential.
(default) calculation mode = train.
(default) use NEP version 4.
(input) number of atom types = 5.
(default) type 0 (Hf with Z = 72) has force weight of 1.
(default) type 1 (O with Z = 8) has force weight of 1.
(default) type 2 (Si with Z = 14) has force weight of 1.
(default) type 3 (W with Z = 74) has force weight of 1.
(default) type 4 (Zr with Z = 40) has force weight of 1.
(default) will not add the ZBL potential.
(default) radial cutoff = 8 A.
(default) angular cutoff = 4 A.
(default) n_max_radial = 4.
(default) n_max_angular = 4.
(default) basis_size_radial = 12.
(default) basis_size_angular = 12.
(default) l_max_3body = 4.
(default) l_max_4body = 2.
(default) l_max_5body = 0.
(default) number of neurons = 30.
(default) lambda_1 = -1.
(default) lambda_2 = -1.
(default) lambda_e = 1.
(default) lambda_f = 1.
(default) lambda_v = 0.1.
(default) lambda_shear = 1.
(default) force_delta = 0.
(default) batch size = 1000.
(default) population size = 50.
(default) maximum number of generations = 100000.
Some calculated parameters:
number of radial descriptor components = 5.
number of angular descriptor components = 25.
total number of descriptor components = 30.
NN architecture = 30-30-1.
number of NN parameters to be optimized = 4801.
number of descriptor parameters to be optimized = 3250.
total number of parameters to be optimized = 8051.

[brucefan1983]

if you do not study radiation damage, there is no need to add ZBL.
When you need to add ZBL, it is usually required to have some dimer structures to make the connection between NEP and ZBL fixed.

[brucefan1983]

The major parameters to tune are the cutoff radii, which are 8 A and 4 A in the default setting. You can try a few combinations:

1. your original ones 5 A, 5A
2. the default ones: 8 A, 4 A
3. perhap you can try another set: 7 A, 5 A

Then you can decide which to take based on accuarcy and speed

[brucefan1983]

Actually, the defult regularization might be too strong. I have revised the default regularization a few days ago (#541), and you can try to see if that gives better training and testing accuracy.

[brucefan1983]

I think there is no real issue here, so I will close it.