finetuning code clarification

[arijitpaul-code]

@rajatsen91 Thanks for updating and sharing the finetuning code. While I was able to utilize it for my own data, I was struggling to understand the following, so would appreciate your response.

1. In the block "Loading TimesFM pretrained checkpoint" the horizon_len = 128, but in the data_loader step pred_len = 96. Is this intentional?
2. In the last block "Loading and evaluating the best (according to validation loss) finetuned checkpoint", tfm.forecast function call doesn't have normalize = True, whereas in the beginning "MAE on the test split for the pretrained TimesFM model" does have normalize = True. So is the before and after MAE comparison fair? Alternatively, is the finetuning taking care of the normalization, so that the inputs to forecast can be raw?
3. Also, curious why in the last block "Loading and evaluating the best (according to validation loss) finetuned checkpoint", take the median from the quantile forecast value for evaluation, and not the point forecast?

I know these are specific questions, but would really appreciate if you can clarify as these would help me understand and utilize the solution correctly.

Thanks!

[rajatsen91]

Hi @arijitpaul-code,

1. This is a good question. The model's architecture is fixed during finetuning so the output residual block still expects to output 128 points into the future. But you can choose to fine-tune it to predict < 128 time-steps if you want to. What would happen is only the first 96 out of 128 prediction heads will participate in the loss function.

2. Good catch. It should be normalize=True but in this dataset in my experience that would not really make much of a difference. Please feel free to try that out.

3. Since the default point forecasting model is "median, both should output the same.

[arijitpaul-code]

Thanks for the clear and prompt response!

[arijitpaul-code]

@rajatsen91 One follow-on question - Does the loss function (train/eval loss) during training calculate on a normalized basis, or is it not needed if the loss is calculated on the entire training/validation population?