some weight problem

[Hephaestusxg]

If I want to use the CIFAR-10 dataset, what should I set the memory size to? Also, during the initial training from 0-7, is there an incremental method involved in the training? Can I save the weights from 0-7 and then train on top of these weights later?

[zhoudw-zdw]

You can follow the settings in https://arxiv.org/abs/2302.03648 in this paper to set the memory size.

For the second question, I think you need to read the PyTorch tutorial and learn how to save and load a model. Unfortunately, we cannot do this for you.

[zhoudw-zdw]

See this page for the detailed tutorial

https://pytorch.org/tutorials/beginner/saving_loading_models.html

[Hephaestusxg]

You can follow the settings in https://arxiv.org/abs/2302.03648 in this paper to set the memory size.

For the second question, I think you need to read the PyTorch tutorial and learn how to save and load a model. Unfortunately, we cannot do this for you.

Sry My question is whether there is a method of incremental learning during the initial training phase. For example, if I am training 10 classes, with the initial classes being 0-7, my experimental results based on cifar10 and resnet32 show an accuracy of 93% for the training on classes 0-7. However, the accuracy should be higher than this if trained normally. Therefore, I would like to ask if there is an incremental approach involved in the initial training phase.

[NguyenQuangMinh0504]

@Hephaestusxg Which method do you use ? Can you explain this sentence "However, the accuracy should be higher than this if trained normally."

[Hephaestusxg]

@Hephaestusxg Which method do you use ? Can you explain this sentence "However, the accuracy should be higher than this if trained normally."

I used the der foster podnet. What I mean is, the accuracy of a normal cifar10 resnet30 should be around 95%, but when I use your code, the accuracy during the initial training phase from 0-7 is at 91%. Additionally, is this acc the average accuracy across the 7 classes from 0-7?

[NguyenQuangMinh0504]

Can you share the code you use to achieve 95% accuracy ? And 95% accuracy is calculated across classes, or just in the first 0-7 classes. I'm not an contributor to this project so I'm not entirely sure, but from my code reading here's my opinion:

1. Is there an incremental approach involved in the initial training phase ?

• No, but I think there is an error in the implementation in the code. The code have a step called weight_align, but it happened before after the training of the model and before the evaluation: Train_on_task_1 -> Weight_align -> Eval_on_task_1 -> Train_on_task_2. I encountered slight drop when using image dataset (3% iirc and in when I tested in another dataset in dropped quiet significant). But I don't think that happen in the first task.

2. is this acc the average accuracy across the 8 classes from 0-7 ?

• No, it's just normal accuracy. There are 2 ways to calculate: NME (nearest means of exemplar, mention in paper ICARL) and normal (max probability of output).

[Hephaestusxg]

Can you share the code you use to achieve 95% accuracy ? And 95% accuracy is calculated across classes, or just in the first 0-7 classes. I'm not an contributor to this project so I'm not entirely sure, but from my code reading here's my opinion:

1. Is there an incremental approach involved in the initial training phase ?

• No, but I think there is an error in the implementation in the code. The code have a step called weight_align, but it happened before after the training of the model and before the evaluation: Train_on_task_1 -> Weight_align -> Eval_on_task_1 -> Train_on_task_2. I encountered slight drop when using image dataset (3% iirc and in when I tested in another dataset in dropped quiet significant). But I don't think that happen in the first task.

2. is this acc the average accuracy across the 8 classes from 0-7 ?

• No, it's just normal accuracy. There are 2 ways to calculate: NME (nearest means of exemplar, mention in paper ICARL) and normal (max probability of output).

I set init_cls to 7 and changed the accuracy calculation in the code. The print display shows the accuracy rate for classes 0 to 6, displayed as: 0-0, 1-1, and so on. I want to know if it is possible to save the weights for the first 7 classes and then perform incremental learning training based on these weights, training for the 3 classes

[zhoudw-zdw]

Actually, if you need to load your own model, you need to learn how to write and implement PyTorch. Modifying the source code can quickly answer questions like " Is there an incremental method involved in the training?"