Can I export the mesh?

[BoghosDavtyan]

Hi, thanks for your amazing work!

The project was pretty easy to install because of the provided script, thank you for that!

And I successfully ran the "generate_face.sh" script and got this output:

But I didn't get a .OBJ file or any mesh file, or at least I couldn't find one, does the script suppose to export the mesh? if yes where does he export the mesh? and if the script does not suppose to export the mesh, can it be done by editing the scripts?

[xiyichen]

Hello,
We currently rely on external multi-view mesh reconstruction pipelines (NeuS/NeuS2) to get the mesh. I was planning to add some instructions about mesh reconstruction later.
For now, you could run the following code in the root directory of this repo to get the preprocessed data in NeuS2 convention:

import numpy as np
import cv2
import os
import json

camera_dict = np.load('./assets/facescape_test_traj.pkl', allow_pickle=True)
batch = cv2.imread('./output/out.png')[:,:,::-1][:, 256:, :] # path to the generated multi-view image
neus2_data_root = '../NeuS2/data/test_output' # path to store the preprocessed NeuS2 data
os.makedirs(os.path.join(neus2_data_root, 'images'), exist_ok=True)
d = {}
d['w'] = 256
d['h'] = 256
d['aabb_scale'] = 1.0
d['scale'] = 1.0
d['offset'] = [0.5,0.5,0.5]
d['frames'] = []
for k in range(16):
    img = batch[:, k*256:(k+1)*256, :]
    alpha_channel = (~(np.all(img > 240, axis=-1))).astype(np.int8)*255
    E = np.eye(4)
    E[:3,:4] = np.array(camera_dict['extrinsics'][k])
    c2w = np.linalg.inv(E)
    c2w[:,1] *= -1
    c2w[:,2] *= -1
    intrinsics = np.array(camera_dict['intrinsics'][k])[:3,:3]
    d_curr = {}
    d_curr['file_path'] = f'images/{str(k).zfill(2)}.png'
    d_curr['transform_matrix'] = c2w.tolist()
    d_curr['intrinsic_matrix'] = intrinsics.tolist()
    d['frames'].append(d_curr)
    img_bgra = np.zeros((256,256,4))
    img_bgra[:,:,:3] = img[:,:,::-1]
    img_bgra[:,:,-1] = alpha_channel
    cv2.imwrite(os.path.join(neus2_data_root, f'images/{str(k).zfill(2)}.png'), img_bgra)
with open(os.path.join(neus2_data_root, f'transform.json'), 'w') as f:
    json.dump(d, f, indent=4)

Then, you could go to NeuS2 directory and run the following:
python scripts/run.py --scene ./data/test_output/transform.json --name test_output --network base.json --n_steps 5000
It takes about 3 minutes to reconstruct the mesh on a single 2080 Ti GPU.

[BoghosDavtyan]

Thank you!