This is a PyTorch implementation of the original paper on Neural Radiance Fields (NeRF), "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" by Mildenhall et al. This project synthesizes novel views of a complex 3D scene from a collection of 2D images with known camera poses.
Here is a 360-degree render of hte Lego bulldozer scene generated by this model after training
Achieved a PSNR of 28.5 after 8 hours of training on an RTX 2060
Here is a depth map calculated by taking the expectation over opacity along each ray
This implementation includes the core components of the original NeRF paper:
- MLP Architecture: A simple MLP to map 5D coordinates (x,y,z and viewing direction) to volume density and color.
- Positional Encoding: Sine-cosine positional encoding is used to enable the model to learn high-frequency details.
- Volumetric Rendering: A fully differentiable ray-marching algorithm to render the color for each pixel.
- Hierarchical Sampling: A coarse-to-fine strategy that samples more points in important regions of the scene to improve detail.
- Training & Rendering: Includes scripts for training a model from scratch, validating with PSNR, and rendering a smooth orbital video from a trained checkpoint.