Softplus activation

[eduardramon]

Hey there,

This is not an issue, just a question. Using softplus activation is significantly slower than using for instance ReLU. Is there any technical (theoretical or practical) reason why you chose softplus in front of other more efficient activations?

Thank you!

[lioryariv]

Hi,
The theoretical reason we are using softplus rather than relu is because we enforcing the Eikonal regularization (equation 11 in the paper), hence we need a differentiable activation (relu is not differential at 0).
Note the for the rendere network we do use relu, since we don’t have such restriction.

Best
Lior

[eduardramon]

Thanks for the fast answer @lioryariv !