What are the key hyper-parameters to tune

[RaptorMai]

Hi,

We are trying to apply CN-DPM on another dataset called CORE50 and the result is quite bad. Since CNDPM has so many hyper-parameters, except for log-alpha and classifier-chill, what hyper-parameters do you think are crucial to tune? Thank you so much and I look forward to hearing back from you.

[soochan-lee]

Sorry for the late reply.
Here are some tips for finding a good set of hyperparameters.

First, check the task inference accuracy under a task-based setting.
There is an option called send_to_stm_always in the configuration files to simulate a task-based setting. If you set this to true, every training example is sent to the STM. Then set the stm_capacity to the number of examples in a task so that every example is trained in the sleep phase. You can now tune the VAEs in complete isolation. Even in this configuration, however, the VAEs would not be good at recognizing tasks.

Second, tune the training-time task inference.
You want the nl_cond_dist graph of each expert to be high only during its responsible task. Adjust classifier_chill to have a sufficient gap. Then set the log_alpha such that it sits between high nl_cond_dists and low nl_cond_dists. Note that the graphs can be noisy due to the wrong classifications and poor VAE accuracy.