.ply point cloud export

[cpkcpkcpk]

Hey,

Is there any possibility of adding .ply point cloud export of the NeRF as it is shown in the viewport, without meshing?

Thanks

[Tom94]

Hi! You can use the Python bindings to get just the vertices of the mesh (without adjacency data) and then use a Python .ply module to export that. Cheers!

[bonfry]

I found this method for getting vertices:testbed.aabb.get_vertices() but how can | export all vertices from Bounding box. I tried this:

import numpy as np

args = np.array([testbed.aabb.min,testbed.aabb.max])
testbed.aabb.get_vertices(args)

but I receive this error:

TypeError: get_vertices(): incompatible function arguments. The following argument types are supported:
    1. (self: pyngp.BoundingBox, arg0: numpy.ndarray[numpy.float32[3, 1]]) -> None

Invoked with: <pyngp.BoundingBox object at 0x7f8ff5af3230>, array([[-1.5, -1.5, -1.5],
       [ 2.5,  2.5,  2.5]], dtype=float32)

[satyajit-ink]

@bonfry i got it working

pip install plyfile

import pyngp as ngp
import numpy as np
from plyfile import PlyData, PlyElement

mode = ngp.TestbedMode.Nerf
testbed = ngp.Testbed(mode)
testbed.load_snapshot("../data/room_dslr/base.msgpack")
mc = testbed.compute_marching_cubes_mesh()

vertex = np.array(list(zip(*mc["V"].T)), dtype=[('x', 'f4'), ('y', 'f4'), ('z', 'f4')])
vertex_color = np.array(list(zip(*((mc["C"] * 255).T))), dtype=[('red', 'u1'), ('green', 'u1'), ('blue', 'u1')])

n = len(vertex)
assert len(vertex_color) == n

vertex_all = np.empty(n, vertex.dtype.descr + vertex_color.dtype.descr)

for prop in vertex.dtype.names:
    vertex_all[prop] = vertex[prop]

for prop in vertex_color.dtype.names:
    vertex_all[prop] = vertex_color[prop]

ply = PlyData([PlyElement.describe(vertex_all, 'vertex')], text=False)

ply.write('nerf_pc.ply')

now you can open it with something like meshlab

[qungvinh1280]

@bonfrytôi đã làm cho nó hoạt động

pip install plyfile

import pyngp as ngp
import numpy as np
from plyfile import PlyData, PlyElement

mode = ngp.TestbedMode.Nerf
testbed = ngp.Testbed(mode)
testbed.load_snapshot("../data/room_dslr/base.msgpack")
mc = testbed.compute_marching_cubes_mesh()

vertex = np.array(list(zip(*mc["V"].T)), dtype=[('x', 'f4'), ('y', 'f4'), ('z', 'f4')])
vertex_color = np.array(list(zip(*((mc["C"] * 255).T))), dtype=[('red', 'u1'), ('green', 'u1'), ('blue', 'u1')])

n = len(vertex)
assert len(vertex_color) == n

vertex_all = np.empty(n, vertex.dtype.descr + vertex_color.dtype.descr)

for prop in vertex.dtype.names:
    vertex_all[prop] = vertex[prop]

for prop in vertex_color.dtype.names:
    vertex_all[prop] = vertex_color[prop]

ply = PlyData([PlyElement.describe(vertex_all, 'vertex')], text=False)

ply.write('nerf_pc.ply')

bây giờ bạn có thể mở nó bằng một cái gì đó nhưmeshlab

I am trying to run your file but it shows error:

[satyajit-ink]

Update:

the resolution can be computed like this

def round_up_to_base(x, base=10):
    return x + (base - x) % base

def get_marching_cubes_res(res_1d: int, aabb:  ngp.BoundingBox ) -> np.ndarray:
	scale = res_1d / (aabb.max - aabb.min).max()
	res3d = (aabb.max - aabb.min) * scale + 0.5
	res3d = round_up_to_base(res3d.astype(np.int32), 16)
	return res3d

mc = testbed.compute_marching_cubes_mesh(resolution=get_marching_cubes_res(512, testbed.aabb), aabb=testbed.aabb, thresh=2)

[satyajit-ink]

also how would i make the resolution higher in the point cloud? what numbers would i change in the above code?

mc = testbed.compute_marching_cubes_mesh(resolution=get_marching_cubes_res(512, testbed.aabb), aabb=testbed.aabb, thresh=2)

here change the 512 to whatever you like.

[Daphoeno]

I tried playing around with this code to compmute the marching cubes mesh for multiple different chunks of the NeRF data by changing the aabb bounds value and whilst it seemed to crop the mesh correctly the result is a bit messed up both with the point positions/resolution and the vertex colours

do you have any suggestion on how I could export the point cloud for a given region in the testbed bounds by changing the above code?

[Mehi44]

Thanks for great solution ! it worked for me too, but when I try to change the resolution I get this :

import pyngp as ngp
import numpy as np
from plyfile import PlyData, PlyElement

mode = ngp.TestbedMode.Nerf
testbed = ngp.Testbed(mode)
testbed.load_snapshot("/home/mehi/Nerfs/instant-ngp/saved/office.msgpack")

def round_up_to_base(x, base=10):
    return x + (base - x) % base

def get_marching_cubes_res(res_1d: int, aabb:  ngp.BoundingBox ) -> np.ndarray:
	scale = res_1d / (aabb.max - aabb.min).max()
	res3d = (aabb.max - aabb.min) * scale + 0.5
	res3d = round_up_to_base(res3d.astype(np.int32), 16)
	return res3d

mc = testbed.compute_marching_cubes_mesh(resolution=get_marching_cubes_res(512, testbed.aabb), aabb=testbed.aabb, thresh=2)

09:50:35 INFO #vertices=14423155 #triangles=25967014

RuntimeError Traceback (most recent call last)
Cell In[13], line 10
7 res3d = round_up_to_base(res3d.astype(np.int32), 16)
8 return res3d
---> 10 mc = testbed.compute_marching_cubes_mesh(resolution=get_marching_cubes_res(512, testbed.aabb), aabb=testbed.aabb, thresh=2)

RuntimeError: /home/mehi/Nerfs/instant-ngp/dependencies/tiny-cuda-nn/include/tiny-cuda-nn/cutlass_matmul.h:332 status failed with error Error Internal

[Fred3D-tech]

I tried playing around with this code to compmute the marching cubes mesh for multiple different chunks of the NeRF data by changing the aabb bounds value and whilst it seemed to crop the mesh correctly the result is a bit messed up both with the point positions/resolution and the vertex colours

do you have any suggestion on how I could export the point cloud for a given region in the testbed bounds by changing the above code?

I have the same question. How to change the testbed bounds without messing everything ? would it be possible to generate a high res point cloud by generating smaller blocks/parts ?
By the way, thanks a lot for the code.

[kafei123456]

Excuse me, have you solved the problems? : " generate a high res point cloud"