Some questions on preparing my own training dataset

[yuedawang]

Hi developers, after learning from the section of "Train MatterGen yourself" in README, I wanna to train MatterGen myself using my own dataset and now there are some questions as following:

1. For the number of structures, taking an example of mp_20 dataset, I notice that there are about 20k structures in the train.csv. For my own training dataset, how many structures are acceptable in the dataset? If I only have a small dataset like 200 structures, could the model be trained efficiently?

2. Could thoes disordered structures with partially occupied atoms be involved in the training dataset? For example, the following cif file with the Sr atoms partially occupied:

 _atom_site_type_symbol
 _atom_site_label
 _atom_site_symmetry_multiplicity
 _atom_site_fract_x
 _atom_site_fract_y
 _atom_site_fract_z
 _atom_site_occupancy
  Sr  Sr0  1  0.89006600  0.10993400  0.25000000  0.5
  Sr  Sr1  1  0.10993400  0.89006600  0.75000000  0.5

3. I notice that there are three files in the mp_20.zip of train.csv, test.csv and val.csv which might be the training, testing and validating dataset. After I collecting all the structures I needed, if necessary to divided the whole dataset into training, testing and validating dataset? In other words, if I only have the training dataset (train.csv), could the model be trained efficiently?

4. For the base model training, I notice that the properties such as "dft_band_gap" and their values were already specified in the train.csv file. So, the properties and their values need to be set first in the base model training?

5. There are many properties in the header of train.csv like ,material_id,formation_energy_per_atom,dft_band_gap,pretty_formula,e_above_hull,elements,cif,spacegroup_number,azure_bulk_modulus,larsen_score_2d,Si_100_mismatch,azure_band_gap,dft_bulk_modulus,dft_poisson_ratio,dft_mag_density
   . If I wanna to add my own property like "dft_density", how many properties are necessary and must be specified in the header of train.csv file. Could I just simply set the header as ",dft_density"?

[danielzuegner]

Hi @yuedawang,

Thanks for reaching out. I'll try to answer your questions one by one.

1. 200 structures is a very small dataset. In this case I would recommend fine-tuning the base model as opposed to training from scratch. Even then, it may be too small. For reference, the dataset which leads to best performance in the paper has around 600k samples.
2. Learning from and generating structures with compositional disorder is not supported by MatterGen.
3. I would highly recommend to have at least a validation set (test set is not necessary) in order to track whether the model is overfitting on the training set. Though it is not strictly required, i.e., in principle you can train the model without a validation set.
4. It is not necessary to add the properties to the base model. This can be done via fine-tuning post-hoc.
5. Yes, if you want to add your own property, add them as a new column to the CSV file. See our instructions.

[yuedawang]

@danielzuegner Thanks a lot for your reply.
As you said, if I only have a small dataset like 200 structures and I wanna train my own model, it is better to fine-tune on the base model, I would do this with the following steps:

1. Combining the mp_20 dataset (mp_20.zip) and my own 200 structures as the initial dataset to train the base model.
2. Fine-tuning the above base model with my own property. (In this step, could I just labeled data with the property value for my own 200 structures)
3. Evaluation as README with the reference dataset of reference_MP2020correction.gz
   Are these steps above reasonable?

[danielzuegner]

Sounds reasonable. Please note that when using reference_MP2020correction.gz as reference dataset, if the 200 new structures are not included in this reference set, then the novelty metric will be skewed. This is because the model could reproduce exact copies of the training set and the metrics evaluation would consider these structures 'novel'. By following our README you can also create your own reference dataset to add your new structures.

[yuedawang]

@danielzuegner Yes, thanks for your reminding and I will also combine the reference_MP2020correction.gz and my own 200 structures as the new reference dataset following the instructions in README to make my final results reliable. I think the steps above could be a general solution for small dataset training and hope this issue could be helpful for other users facing the similar situation like me.