Why don't you split the adnet into the two nets: policy net and confience net?

[deeplearnerJHB]

Thank you for your team's such meaningful work! I have a question, why don't you split the adnet into the two nets: policy net and confience net to train repectively? After the sl step, you use the rl to train the fc1-fc6 except for the fc7 and in online adaptation step, you use the fc7's parameter directly. Don't you worry about that the parameters' change in rl step may lead the confidence branch of the net less accurate?

[aseuteurideu]

I am not the author, but I guess it is also about the speed. Having separate net may lead to higher accuracy (or may be not as the confidence and actions are kinda share same knowledge in low level. So I think it is better to share some low level layers) but it definitely will cost double of the computation. Anyway, the confidence branch and the layers from fc4 to fc6 are fine tuned again in the online adaptation part, so it will be adjusted with the previous layer again.

But if you want to experiment this thing, I would suggest you to also try having the branch start from middle (e.g. from fc4) in additional of two separate nets.