Questions on the implementation of transfer learning.

[PeihuaZhu0516]

Dear lulu,

Thank you for your paper on "Extraction of mechanical properties... " and your open source codes. They provide me a novel approach to investigate mechanical issues.

I have run your code and feel confused about how the transfer learning works. Specifically, I cannot find how the first training influences the second training of multi-fidelity NN in the code of nn.py.

Thanks for your time and I am looking forward to your reply.

[lululxvi]

The code for transfer learning requires some dirty work. Here is the procedure to run the transfer learning:

1. Pretrain: Train the MFNN with FEM dataset, and save the trained network.

• In mfnn(), comment line 127, and uncomment line 128-131.
• In validation_exp_cross_transfer(), use the current code.
• After training, the best network model will be saved in the "model" folder, e.g., "model/model.ckpt-28000". Here 28000 is the step with the best model, and it may change in your case. You need to read the display information.

2. Transfer: fine-tune with experimental dataset.

• In mfnn(), set trainable_low_fidelity=False in line 121. Comment line 127-131, and uncomment line 132. Note that you need to change the filename of the model, i.e., "model/model.ckpt-28000".
• In validation_exp_cross_transfer(), comment line 317-326.

[PeihuaZhu0516]

I am so happy to receive your reply. Thank you a lot for giving me detailed instructions!

[mburaksayici]

Thanks for sharing the code which is rare thing. I wonder if you elaborate transfer learning procedures.

First, it seems to me that training is so long that it shouldn't take to long for for train_size in range(1, 10): in nn.py .

The second is, which .py file includes the model architecture that we fine tune? In nn.py, there is mfnn() as you've said, which should be the network. But, in model.py, there are models used for prediction. What's the difference?

Third, which datafile should I fill for transfer learning? In data, there are data files, but, for transfer learning, First I need to train with the data in data file, and after pretraining, I need to "fine-tune with experimental dataset". Where should I place the experimental dataset?

Fourth, there are Pi1 Pi2 .. .. functions in model.py, there numbers multiplied by variables. Don't they harm the fine tuning? Are they material specific or general constants for formulas?

I asked a lot, I know the answer of some question above but sth is not clear for me, my bad, wish to learn it in more detailed way. Thanks.

[lululxvi]

• The train should be fast, because the network is small and the dataset is small.
• All the code for network is in nn.py
• model.py includes the code for the models in Ref [5] and [6], and has nothing to do with networks.

datalow = FEMData(yname, [70])  # The low fidelity 2D FEM dataset
dataBerkovich = BerkovichData(yname)  # The high fidelity 3D FEM dataset for pretraining
dataexp = ExpData("../data/B3090.csv", yname)  # The high fidelity experimental dataset for fine tuning