Version 0.7.6

This version supports PyTorch 2.2.0

New features

• We can now read UV textures from a PLY file #1100
• Align_corners switch in Volumes 94da884
• submeshing now supports TexturesUV and TexturesAtlas 8a27590 e46ab49
• Colors in cubify ae9d878

Improvements

• PointsRenderer documentation 3b4f8a4
• Made TexturesUVs more efficient 06cdc31
• Numerical stability improved in so3_log_map / so3_exp_map 292acc7
• get_rgbd_point_cloud takes any number of channels, not just RGB 799c1cd

Fixes

• Fix to C++ marching cubes c292c71
• Fix to cuda marching cubes (which was broken in 0.7.5) f613682 7606854
• Remove unused argument mask_points from get_rgbd_point_cloud and fix get_implicitron_sequence_pointcloud, which assumed it was used f4f2209
• Standardize matrix_to_quaternion output 3087ab7

Version 0.7.5

Brings PyTorch 2.1 support

New feature

• Implicitron: Integral Position Encoding, blurpool; based on MIP-NeRF ccf860f 29b8ebd 5910d81 3d011a9 9446d91
• Chamfer distance: single-direction option, absolute option 5ffeb4d
• Chamfer without point reduction d84f274
• VolumeSampler: expose padding_mode for inside out rendering b7f4ba0

Implicitron improvements

• Softly deprecate the get_str=False flag in stats.print. d08fe6d
• FrameDataBuilder is more extensible b046259
• SQLDataset improvements d2119c2 09a99f2 cd5db07
• OverfitModel’s fine and coarse implicit functions have the right names 35badc0
• Read depth maps from OpenXR files using OpenCV d851bc3

Minor

• matplotlib compatibility 928efdd 5592d25 8842985
• Implicitron: fg_mask/fg_probability typing fix 8164ac4

Version 0.7.4

A small release. It also has builds for PyTorch 2.0.1

Minor new features

• Saving normals to OBJ 092400f
• Saving TexturesVertex to GLB 178a777
• Implicitron can now read from a new dataset format which stores metadata in sqlite files. This requires the sqlalchemy library. 32e1992

Fixes

• Error in example in cameras.md f5a117c
• Build fix with newer CUB b921efa
• Warnings when we don’t obey user’s request of torch.use_deterministic_algorithms. Some cuda implementations in pytorch3d are non-deterministic. ef5f620
• Implicitron evaluation still works when ground truth depth map is not supplied 0e3138e
• Loading bounding boxes in Co3D-type datasets in implicitron was failing when masks were not requested 7aeedd1

Version 0.7.3

A small release. We now support PyTorch 2.0.

Minor new features

• join_pointclouds_as_scene a123815
• OverfitModel for NeRF-style training with overfitting to one scene, a specific case of GenericModel 813e941

Improvements

• No relative imports in GenericModel, making the file easier to copy into a user’s project. d388881
• Implicitron classes which are both nn.Modules and configurables now have simpler initialization, where user does not need to worry about calling nn.Module’s __init__. 9540c29 c8af1c4
• Windows compilation fix 3388d3f
• doc improvements: opencv cameras 18c38ad
• Camera objects can now be iterated 84851c8
• Fixes for RayBundle plotting 9dc28f5
• Fixing bug in rasterizer.py caused by duplicate line (#1421) b95535c
• Doc and fixes for get_rgbd_point_cloud, including new euclidean option a12612a
• Implicitron improvements to Co3D-style dataset and evaluation including to render_flyaround: 1de2d0c d561f19 54eb76d 3239594 11959e0 a7256e4
• Version incrementing for inplace tensor modifications 7d8b029
• Implicitron’s components now ensure the registry is prefilled using a new extension point pre_expand. This is for compatibility with lazy import systems like cinder. c759fc5
• Big refactor of Implicitron’s json_index_dataset, with much loading logic moved to a new FrameDataBuilder concept. ebdbfde

Version 0.7.2

A small release. Python 3.7 is no longer supported. We support from PyTorch 1.9.0 to PyTorch 1.13.0. [EDIT: post-release, builds for 1.13.1 have also been added.]

New feature

• Marching cubes now has an efficient CUDA implementation. 8b82918 . Improvements to the cpu code too 1706eb8

Minor new features

• The experimental glTF support now allows writing meshes to GLB files, including with TexturesUV textures. cc2840e
• Specification of param groups in implicitron made more flexible 7be49bf
• Implicitron’s loader for Co3D-style datasets can load multiple categories e4a3298
• Implicitron can be used without the lpips library being present. 322c8f2

Fixes

• Implicitron visualization c3a6ab0 94f321f
• Implicitron’s rasterize_mc 1706eb8
• Implicitron’s voxel grid, especially its scaffold a1f2ded bea84a6 f711c4b
• raysampling: The origins in a raybundle (which downstream code typically doesn’t use) were wrong when unit_directions=True eff0aad

Documentation

• Fixes to readmes and readthedocs

Version 0.7.1

This release brings a lot of fixes and improvements to implicitron.

PyTorch 1.8 is no longer supported. We support from PyTorch 1.9.0 to PyTorch 1.12.1.

New features

• Marching cubes fixes to existing python implementation, and fast C++ implementation for when data is on CPU 0d8608b 850efdf
• New fisheye camera object. This involves a change to the API to allow such “nonlinear” cameras, i.e. those whose projection is not a projective transformation. In particular, get_projection_transform can now fail. b0515e1 d19e624 84fa966 8a96770

Minor new features

• get_se3_log function on Transform3D 9a0f9ae
• get_rotation_to_best_fit_xy in circle_fitting 74bbd6f
• Saving a mesh via IO to OBJ now includes TexturesUV texture 6ae6ff9
• Checkerboard mesh utility ce3fce4
• Camera batches can now be indexed by bool tensors b7c826b
• Implicitron: You can just use a Configurable class straight away, you no longer need to call expand_args_fields or get_default_args first. d6a197b
• Implicitron: In the newest pytorch versions, you can now use the faster foreach versions of optimizers 209c160
• Implicitron: full_image variants of psnr, l1 and lpips losses (ignoring mask). The masked image ones are renamed to make this clear. 7b98570 d35781f
• Implicitron: Enable additional test-time source views for json dataset provider v2 2ff2c7c
• Implicitron: Option for filtering outlier input cameras in trajectory estimation d281f8e
• Implicitron: yaml configs to train selected methods on CO3Dv2 1163eaa
• Implicitron raymarcher replicate_last_interval option. 611aba9
• Mixed Frame Raysampling. You can now sample rays iid from a group of cameras, instead of a fixed number from each camera. This needed a new RayBundleHeterogeneous alternative to RayBundle. Implicitron now has its own raybundle object. c311a4c ad8907d 6ae863f

Major bug fix

• Implicitron: stratified_sampling no longer disabled 438c194 df36223

Bug fixes

• Cloned camera objects can now be saved with pickle and torch.save efea540
• Fix implicitron iteration time measurement dd58ded
• IOU3d fix for numerical stability 1bfe6bf
• Update SMPL url 5e7707b
• Implicitron doc fixes fdaaa29 4711d12
• Implicitron reduction feature aggregation broadcasting fix 73ba66e
• Implicitron Fix for loading Co3Dv2’s point clouds 95a2acf

Performance improvements

• Omit _check_valid_rotation_matrix by default 46cb5aa
• _compute_vertex_normals does less work a65928d
• Pulsar logging removed for performance gain 7623457
• VolumeSampler memory saving 4c8338b
• Implicitron: Avoid extra CPU->GPU communication in raysampling 305cf32
• Mesh rasterization on CPU now uses multiple threads 6471893

Minor

• In implicitron, there are components in progress for voxel-grid-based implicit functions: decoders, voxel grids, param groups edee25a e7c609f fe5bdb2 24f5f4a
• Implicitron config system fixes 4d9215b da7fe28 8339cf2
• Implicitron error improvements a607dd0
• Implicitron JsonIndexDataset fix for numpy.split warning 0b5def5
• Doc fix for IOU a2659e1 6080897
• Doc fix for Implicitron base renderer 4711d12
• pytorch3d.ops doc fixes 6653f44
• Lengths validation in chamfer and farthest_points cb7bd33
• Implicitron visualize_reconstruction and render_flyaround improvements and fixes 6e25fe8 3b3306f f6d43ea c79c954
• Implicitron compatibility with hydra 1.2 90b758f
• Implicitron SimpleDataLoaderMapProvider sample batches without replacement 73ea418
• Implicitron: visdom is now optional ff933ab

Internal

• packed_to_padded now accepts all sizes f34da3d
• test_opengl_utils unfriendly test fixed 9a1213e
• Implicitron: raybundle input to ImplicitFunctions has been made more consistent 72c3a0e
• location logic factored out of Volumes into a new VolumeLocator class so it can be reused in other objects. f825f7e
• Sample_farthest_points uses CUB instead of a handwritten block reduction 8ea4da2
• Implicitron: MLP and Transformer have moved as they are no-longer only used by nerf. 898ba5c
• Cameras benchmarking 03562d8
• Removed wrappers for eigh, solve, lstsq, and qr because we now rely on PyTorch 1.9 or greater d4a1051

Version 0.7.0

This release brings Implicitron and MeshRasterizerOpenGL. There are builds for PyTorch 1.12.0, but no longer 1.7.x.

Large New Features

• Implicitron, a framework for new-view synthesis via modeling neural-network based representations. See its README for an introduction.
• MeshRasterizerOpenGL, a faster alternative to MeshRasterizer described in Cole et al.. The speedup, which grows with mesh size, is about 20x on a 2M-faces-large mesh. This rasterizer is designed to work with SplatterPhongShader, a new per pixel lighting model. To use it, you must have pycuda with the gl extension enabled, as well as pyopengl.

New features

• You can now choose to ignore the XY flip in cameras’ transform_points_screen commit
• Fragments is now a dataclass with docstring and detach() commit
• SoftDepthShader and HardDepthShader for rendering depth maps commit and commit
• AmbientLights can now be used to render an arbitrary number of channels (not just RGB) commit

Bug fixes

• Fix for loading PLY files with heterogenous faces (e.g. mixture of triangles and quadrilaterals commit
• Pointclouds.num_points_per_cloud now always returns a tensor commit
• Chamfer fix for empty pointclouds commit

For developers

• We are now using black 22.3 or later
• Tests should now be run from the pytorch3d root directory not the tests directory.

Minor

• In blending, pull common functionality into get_background_color commit
• raising translated errors with from None so a simpler traceback is printed 8e0c82b
• PnP test fix commit
• fix test which had cuda indexes of a cpu tensor commit

Version 0.6.2

This release brings new targets and many improvements.

Builds for PyTorch 1.10.1, 1.10.2, 1.11.0, but no longer 1.6.0. Builds for Python 3.10 but no longer 3.6.

New Features

• MultinomialRaysampler and NDCMultinomialRaysampler replace GridRaysampler and NDCGridRaysampler commit which can sample using the new n_rays input and also bring stratified sampling along rays and direction normalization (also commit
• Function join_cameras_as_batch commit
• Function join_pointclouds_as_batch commit
• Camera batches can be indexed commit and commit
• Meshes.submesh function to take a set of faces from a mesh. commit and commit
• L1 support for KNN and chamfer commit

Bug fixes

• Rasterizer.to broken without cameras commit
• Joining batches of meshes with TexturesAtlas and TexturesUV broke first input meshes’ texture commit
• Transform3D.stack was entirely broken. commit which also added typing.
• The function cameras_from_opencv_projection always created on CPU #1021 commit
• Batching didn’t work for AmbientLights #1043 commit
• Pointclouds.subsample failed on windows #1015 commit
• Fit_textured_meshes tutorial now turns off perspective_correct in the final optimization, to avoid nans. commit

Improvements

• Points_normals much faster through use of symeig workaround commit
• A warning is now printed to console on rasterizer bin overflow commit

Small improvements

• MeshRendererWithFragments and matrix_to_axis_angle made more importable
• FacePointDistance, FacePointDistance and point_mesh_face_distance get min_triangle_area argument commit - default also changed in commit
• Fix for small faces in IoU3D commit
• Lower the epsilon value in the IoU3D commit
• Flexible background color for point compositing commit and commit
• In points_to_volumes, the rescaling of features is now optional. commit
• LinearWithRepeat layer clarified and moved inside PyTorch3D from the NeRF project commit
• HarmonicEmbedding moved inside pytorch3d from projects/nerf. commit and commit
• Invalid default values in Meshes.init removed commit
• Support obj files missing usemtl declarations commit
• CamerasBase.unproject_points typing commit
• Fix for Pointclouds.init in certain cases with None in a list commit
• Consistency validation for sampling_mode in joining TexturesUV commit
• Doc fix for camera space fix from v0.6.1 commit
• Doc fix for renderer pictures commit
• New ndc_grid_sample to handle grid-sampling using our NDC conventions commit
• Fix scatter_ error in cubify on CPU commit
• In Transform3D, fix some dtype inconsistencies commit
• Fixes to make windows builds happier commit

Internal

• Rename pytorch3d.common.types to pytorch3d.common.datatypes commit
• Cleanup in ply_io commit
• A major new feature, a framework for new-view synthesis via modeling neural-network based representations, called implicitron, is work in progress.
• We now use usort not isort for sorting imports.
• Future proofing calls to torch.meshgrid commit and symeig commit also commit. Also future proofing tests for a change to autograd behaviour that inplace operations don’t need to return the same object commit
• Future proofing for the loss of TH headers in pytorch (needed for PyTorch 1.11) commit and commit
• Base class for shaders introduced commit

Version 0.6.1

This release brings PyTorch 1.10 builds and numerical fixes and improvements

Large fixes

• Raysampling now works with non-square image conventions commit
• Perspective_correct mesh rasterization calculation is protected against divide-by-zero. This fixes quite a number of bug reports, e.g. #561. commit

Breaking changes

• This commit makes camera code more consistent in behaving like align_corners=False everywhere, by removing some extra -1’s in the code for screen space conversion. (1) If you create a camera in screen coordinates, i.e. setting in_ndc=False, then anything you do with the camera which touches NDC space may be affected, including trying to use renderers. The transform_points_screen function will not be affected. (2) If you call the function “transform_points_screen” on a camera defined in NDC space then results will be different.

Bug fixes

• Raysampling now works with cameras defined in screen space. commit.
• Pointclouds.inside_box is now properly reduced rather than returning separate results per axis. commit
• PulsarPointsRenderer fixed with non-square cameras commit
• Functions clone and detach on TexturesUV properly propagate align_corners and padding_mode options. commit
• Fixing default arguments of add_points_features_to_volume_densities_features commit
• Fix opencv camera conversion for non-square images (affects pulsar) commit

Small improvements

• Some matrix conversions are now traceable with torch.jit. commit
• Fixes to compiled code to make windows builds happier commit commit
• A new set of tests, test_camera_pixels, illustrates the precise mapping of pixels for cameras. commit
• Numerical improvements in IOU calculation commit and commit
• New option on TexturesUV to choose sampling_mode. commit
• The plotly visualization will show face colors from a TexturesAtlas with K=1 commit

Internal

• Benchmarks have been moved to tests/benchmarks commit
• Spelling commit
• Some more type annotations for Pointclouds commit
• Test fixes commit
• Special implementation of eigenvalue calculation for 3x3 matrices which can be faster and more reliable than native PyTorch. Currently not used commit

Version 0.6.0

This release contains several new optimized operations related to point clouds.

New features

• Farthest point sampling here, here and here
• Ball query operation. commit
• Sample_pdf importance sampling operation, implemented with a CUDA kernel. Previously in Python in the NeRF project. here and here
• Fast accurate calculation of Intersection over union for 3D boxes. See note. here, here and here
• Subsample method for Pointclouds commit
• Adding point clouds to volumes is now implemented in C++/CUDA, which means it can always operate inplace here, here and here

Breaking changes

• This commit removes _xy_grid from the saved state of the GridRaySampler module, if the PyTorch version is 1.6.0 or higher. This means that you will get errors if you try to load a model saved with such a module by an older version of PyTorch3D (unless strict==False). Similarly the NeRF project’s HarmonicEmbedding no longer stores _frequencies.
• PyTorch 1.5.0 or above is now required.

Bug fixes

• Fix duplicate arguments errors when cameras are specified in certain places commit
• Fix to join_scene for meshes using TexturesUV, which picked the wrong verts_uvs in certain cases commit
• Fix to edge cases when creating TexturesAtlas. commit
• Points to volumes fix when the grid_sizes are not specified. commit

Small improvements

• Making the rasterizer deterministic if there are ties between faces commit
• The function so3_log_map is now torchscriptable commit
• The GridRaySampler change means it can be reused at different image sizes. commit
• More documentation in the renderer, with RasterizationSettings and PointRasterizationSettings being dataclasses commit and here
• Ability to save colors as 8bit (i.e. uint8) when writing data to PLY files commit

Internal

• Coarse rasterization code has been reorganized here, here and here