How to use l2 regularization and dropout to prevent overfitting?

[jaried]

• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

tianshou:0.4.6.post1, torch:1.10.2+cu113, numpy:1.21.5, sys:3.7.0 (default, Jun 28 2018, 08:04:48) [MSC v.1912 64 bit (AMD64)], platform:win32

I solved the CartPole problem with FQF in legacy mode, and with your help, I have been able to solve it.
Other questions, how to add l2 regularization or dropout to prevent overfitting?

1. I see that there is norm_layer in the MLP of net. Is l2 regularization added here?
2. How to add dropout?

.\tianshou\utils\net\common.py
line 51-60
    def __init__(
        self,
        input_dim: int,
        output_dim: int = 0,
        hidden_sizes: Sequence[int] = (),
        norm_layer: Optional[Union[ModuleType, Sequence[ModuleType]]] = None,
        activation: Optional[Union[ModuleType, Sequence[ModuleType]]] = nn.ReLU,
        device: Optional[Union[str, int, torch.device]] = None,
        linear_layer: Type[nn.Linear] = nn.Linear,
    ) -> None:

[nuance1979]

1. I see that there is norm_layer in the MLP of net. Is l2 regularization added here?

No. In PyTorch, L2 regularization (aka, "weight decay") is a function of the optimizer. See the following code for a proper example, which doesn't add L2-reg/weight decay to biases or norm layer weights:

https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_clm_no_trainer.py#L417-L428

2. How to add dropout?

Dropout can be added by MLP(..., norm_layer=nn.Dropout(0.1)).

[jaried]

Thanks for your answer. This works great for me.

[jaried]

I got a reply from someone today that RL doesn't need to prevent overfitting, because RL requires accuracy.