Encoder + MLP combo #2063
Encoder + MLP combo #2063
Conversation
| @@ -102,38 +103,41 @@ def __init__( | |||
| self.build_nn_modules() | |||
| elif implementation == "tcnn" and not TCNN_EXISTS: | |||
| print_tcnn_speed_warning("MLP") | |||
| self.build_nn_modules() | |||
There was a problem hiding this comment.
I fixed this in this PR too!
|
@ethanweber How come this way its faster? |
nerfstudio/fields/nerfacto_field.py
Outdated
| "num_layers": num_layers, | ||
| "layer_width": hidden_dim, | ||
| "out_dim": 1 + self.geo_feat_dim, | ||
| "activation": nn.ReLU(), |
There was a problem hiding this comment.
it seems like this should be passing nn.ReLU instead of an instance, is there a reason it needs to call the constructor here?
|
I cleaned up the code after seeing issue #2348. I tested with the dozer scene from the Nerfstudio Dataset. Running nerfacto, I get these memories with the combined MLP and hash encoder vs. sequential.
@tancik, I went with your idea of combining the encoder and MLP into a Python class with typed arguments. |
|
Nice that it's supposed to be faster! Just note that it breaks older checkpoints, and the default unstrict loading causes silent skipping of parts of the model, leading to poor renders that are hard to explain. |
In PR #1936, I removed the functionality of
tcnn.NetworkWithInputEncodingand usedtorch.nn.Sequentialinstead. Here I revert that with a new class calledEncoderAndMLP. This should be faster when using tccn with nerfacto. See "combo" below where I show some runs, though it doesn't show a significant (if any) improvement.