the preprocessed data does’nt has attribute [pos_normals] or [neg_normals]

[QtEngineer]

I tried to set [use_precomputed]=false but error still happened cause of ：

if sdf_samples_with_normals is None:
    if use_precomputed_init:
        return torch.from_numpy(np.load(initialization_file))  
    else:
        npz = np.load(sdf_filename)
        pos_tensor = deep_sdf.data.remove_nans(torch.from_numpy(np.concatenate([npz["pos"], npz["pos_normals"]], axis=1))).numpy()
        neg_tensor = deep_sdf.data.remove_nans(torch.from_numpy(np.concatenate([npz["neg"], npz["neg_normals"]], axis=1))).numpy()

#----------

import numpy as np
cat=np.load('/user-data/patchnets-master/code/data/SdfSamples/ShapeNetV2/02691156/1a29042e20ab6f005e9e2656aff7dd5b.npz')
cat.files
['pos', 'neg']

[edgar-tr]

Hi, are you using the version of src/PreprocessMesh.cpp that is included in this repo or the one from DeepSDF? This repo contains a modified version that also writes out normals: https://github.com/edgar-tr/patchnets/blob/1d20bfd1ec74f8c6687939c6092642685fdeee94/src/PreprocessMesh.cpp#L258 On the other hand, DeepSDF only writes out pos and neg (as in your case, apparently): https://github.com/facebookresearch/DeepSDF/blob/main/src/PreprocessMesh.cpp#L220 It is best to first install DeepSDF and to then overwrite all files in that installation with the files included in this (PatchNets) repo.

[QtEngineer]

Thank you so much! Now I can successfully generate correct data .
However, I noticed that some npz files have suffix of "rejected" because of :

if (wrong_ratio > rejection_criteria_obs || bad_tri_ratio > rejection_criteria_tri) {
std::cout << "mesh rejected" << std::endl;
npyFileName = npyFileName + "____rejected.npz";
//return 1;
}
(in ProprocessMesh.cpp )
It will not happen when using cpp files included in DeepSDF.

I'm confused about it and I want to know whether the npz files were generated correctly.
In addition ，if there is no errors ，should I use these rejected.npz files to train or test ， or delete them？

[edgar-tr]

I believe I added that merely as a marker for debugging (I did not really clean up the code before releasing it) but use them as normal shapes everywhere. The original DeepSDF code similarly does not do anything with the information that some point clouds are not perfectly suitable: https://github.com/facebookresearch/DeepSDF/blob/main/src/PreprocessMesh.cpp#L509 In the folder that I can still find, there appear to be only five shapes in the 13 Shapenet categories that we use that are marked as "rejected". So either way it won't make any noticeable difference.

[edgar-tr]

Let me correct that, when loading the shapes, this line here skips npz files that don't fit the standard (non-rejected) naming scheme: https://github.com/edgar-tr/patchnets/blob/master/code/deep_sdf/data.py#L20 Note that the train/test/val splits are generated using the .obj files, before PreprocessMesh.cpp is ever called: https://github.com/edgar-tr/patchnets/blob/master/code/preprocess_data.py#L238 So the splits assume that the rejected shapes exist (although the data.py code mentioned in the previous paragraph will skip them during training when loading the shapes in the split). That's important because it means that the .obj's of the rejected npz's should not be deleted. If they are deleted, the splits will get messed up.

[QtEngineer]

Thanks a lot! Up till now I have generated 1110 npz files used sv2_lamps_json and there is 6 rejected mesh. And I generated a part of sofa meshes (about 643 with 13 were rejected).I think it will not make a big difference?

Besides ,I met a problem when I tried to train with some specific npz files(not rejected mesh) ,There is a 【 RuntimeWarning: invalid value encountered in arcsin 】because of ：
def _get_euler_angles_from_rotation_matrix(rotation_matrix):
if np.abs(rotation_matrix[2,0]) != 1.:
beta = -np.arcsin(rotation_matrix[2,0])
cosBeta = np.cos(beta)
return np.array([np.arctan2(rotation_matrix[2,1], rotation_matrix[2,2]), beta, np.arctan2(rotation_matrix[1,0], rotation_matrix[0,0])])

I guess it is because the value of rotation_matrix[2,0] is outside the range of [- 1, 1] and it cause an error:
def remove_nans(tensor):
tensor_nan = torch.isnan(tensor[:, 3])
tensor_nan_fixed = torch.isnan(tensor).any(-1)
if (tensor_nan != tensor_nan_fixed).any():
import ipdb; ipdb.set_trace()
return tensor[~tensor_nan, :]
The program paused because 【tensor_nan != tensor_nan_fixed】( for Ipython Debugger ). How can I fix it?

[edgar-tr]

I'm not following. The _get_euler_angles_from_rotation_matrix can return NaNs, yes, but they are removed immediately afterwards: https://github.com/edgar-tr/patchnets/blob/1d20bfd1ec74f8c6687939c6092642685fdeee94/code/train_deep_sdf.py#L374 remove_nans gets only called with input "tensor"s that contain positions or normals, but not with rotation matrices: https://github.com/edgar-tr/patchnets/blob/1d20bfd1ec74f8c6687939c6092642685fdeee94/code/deep_sdf/data.py#L58 You could replace the ipdb line with tensor_nan = torch.logical_or(tensor_nan, tensor_nan_fixed), I suppose. I don't remember this part well.

[QtEngineer]

Thank you so much! I replaced ipdb line and trained model successfully. I trained about 900 ShpenetV2obj with 300 epoch , but after reconstructing and evaluating，the result was not so good, （The average chamfer distance is as more than 1.0）
I noticed that while reconstructing ,the code does not use specs and sdffile:
(in reconstruct.py)
(Referrence:)
err, latent = reconstruct(
decoder,
int(args.iterations),
mixture_latent_size,
data_sdf,
0.01, # [emp_mean,emp_var],
0.1,
num_samples=8000,
lr=5e-3,
l2reg=True,
)
(Define:)
def reconstruct(
...,
specs=None,
sdf_filename=None,):

I am not sure whether it will cause error.

[edgar-tr]

The reconstruct.py is not compatible with PatchNets. It's a bit hidden in the readme, but "|useful_scripts.py| also contains code to extract meshes." See here: https://github.com/edgar-tr/patchnets/blob/master/useful_scripts.py#L1705

[edgar-tr]

For fitting to test data, see https://github.com/edgar-tr/patchnets#evaluation

[QtEngineer]

Thank you so much! I managed to convert latent codes into meshes by using functioon visualize_parts_individually (patches),which generated .obj followed train_split.json.
But when I tried to run useful_scripts.py to Evaluation, an error occured ：
error:
......，
File "useful_scripts.py", line 105, in visualize_mixture
_, latent_code = sdf_to_latent(sdf, full_loss=True, latent_init=latent_code, num_samples=3072, num_iterations=1000, sdf_filename=sdf_filename, lr=0.001, adjust_lr_every=200, decrease_lr_by=2)
UnboundLocalError: local variable 'latent_code' referenced before assignment

The function evaluate_patch_network_metrics() includs visualize_mixture()（ # full object）, in visualize_mixture：(Line 105)
train_latent_codes = torch.load(results_folder + "LatentCodes/" + checkpoint + ".pth")
for i, mesh in enumerate(mesh_files):
if mesh in train_split:
train_id = train_split.index(mesh)
latent_code = train_latent_codes["latent_codes"][train_id,0,:].unsqueeze(0).clone().cuda()
else:
sdf_filename = data_source + mesh + ".npz"
sdf = np.load(sdf_filename)
sdf = (torch.from_numpy(sdf["pos"]), torch.from_numpy(sdf["neg"]))
_, latent_code = sdf_to_latent(sdf, full_loss=True, latent_init=latent_code, num_samples=3072, num_iterations=1000, sdf_filename=sdf_filename, lr=0.001, adjust_lr_every=200, decrease_lr_by=2)

It seems that the variable [latent_code] does not declared at (latent_init=latent_code) in the else branch. How can I fix it?

[edgar-tr]

There should never be a case where the code lands in the else branch. If you follow the readme for test-set reconstruction (see my earlier comment), the test set will technically be treated as a training set by the code (which is why it's important to keep the network weights fixed, for example). Therefore, the if branch should always get chosen by the code.

[QtEngineer]

thanks a lot ！