Some problems about baseline

[ZzBros]

When I was training the baseline network (useing exp_cifar10_wrn2810_1net_standard_bar1.yaml) , I want to know if I understand the following correctly?

1. mixmo uses random sampling (learning every sample once on one epoch)
2. mixmo uses DADataset
3. mixmo uses wideRsenet28-10
4. mixmo with warmup lr in first epoch and reduce to 0.1 * lr in [101,201,226] epoch
5. mixmo with L2 regular for net params
6. mixmo uses the specified initialization

I look forward to your comment if I have missed anything !!!
Thanks!!!

[alexrame]

Yes, all this is true!
Note that config/cifar10/exp_cifar10_wrn2810_1net_standard_bar1.yaml is a configuration file for a standard baseline network without data augmentation nor ensembling, so should not be called "mixmo" but rather "vanilla".
In detail, the critical code sections for the different points are:

1. vanilla uses random sampling (learning every sample once on one epoch)
   

   mixmo-pytorch/mixmo/loaders/abstract_loader.py

   Line 62 in 5a2a1a0

   replacement=False,

2. vanilla uses MSDADataset with msda_mix_method == None, which is strictly equivalent to DADataset
   

   mixmo-pytorch/mixmo/loaders/abstract_loader.py

   Line 47 in 5a2a1a0

   class_dataset_wrapper = dataset_wrapper.MSDADataset

3. vanilla uses wide Resnet28-10
   

   mixmo-pytorch/mixmo/networks/wrn.py

   Line 54 in 5a2a1a0

   class WideResNet(PreActResNet):

4. vanilla with warmup lr in first epoch (=782 steps) and reduce to 0.1 * lr in [101,201,226] epoch
   

   mixmo-pytorch/config/cifar10/exp_cifar10_wrn2810_1net_standard_bar1.yaml

   Line 69 in 5a2a1a0

   warmup_period: 782

   
   

   mixmo-pytorch/mixmo/learners/model_wrapper.py

   Line 80 in 5a2a1a0

   def _init_scheduler(self, epoch):

5. vanilla with L2 regular for net params
   

   mixmo-pytorch/config/cifar10/exp_cifar10_wrn2810_1net_standard_bar1.yaml

   Line 52 in 5a2a1a0

   l2_reg: 0.0003

   
   

   mixmo-pytorch/mixmo/core/loss.py

   Line 176 in 5a2a1a0

   if self.config_loss.get("l2_reg"):

6. vanilla uses the specified initialization
   

   mixmo-pytorch/mixmo/networks/resnet.py

   Line 167 in 5a2a1a0

   def _init_weights_resnet(self):

If you have any questions please let me know.
Best regards