Bug with compute_iss_keypoints (failing randomly)

[genevieve-le-houx]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for master branch).

Describe the issue

I'm trying to extract the keypoints from a point cloud with the method open3d.geometry.keypoint.compute_iss_keypoints(). I'm looping thru a lot of files in directory and sub-directories. It looks like :

``

import open3d as o3d
from pathlib import Path

directory_pcd = Path("directory_to_pcd_files")
for file in directory_pcd.iterdir():
    pcd = o3d.io.read_point_cloud(str(file))
    print(f"Compute keypoints of {file}")
    keypoint = o3d.geometry.keypoint.compute_iss_keypoints(
        pcd, gamma_21=10, gamma_32=10, min_neighbors=1
    )
    print(f"Finished computing keypoints of {file}")

``

The problem is the compute_iss_keypoints fails randomly. Like I can see with my print that's exactly at this line that it fails, but it will never fails at the same file, never files after processing the same number of files. Yes, the method .iterdir() process the files in the same order each time.

I've tried this with Python 3.8.7, 3.9.10. open3d version 0.14 and version 0.15. It still fails.

Steps to reproduce the bug

1. Take a directory with some .pcd files in it and replace the first line of code with the valid directory
2. Run and look at the print statements. It will never fails at the same file (while still looping them in the same order each time)

Error message

Process finished with exit code -1073741819 (0xC0000005)

Error in the console from PyCharm

Expected behavior

The method should work for each file and not fail randomly

Open3D, Python and System information

- Operating system: Windows 10 64-bit
- Python version: Python 3.9.10
- Open3D version: output from python: 0.15.1
- System architecture: x64
- Is this a remote workstation?: no
- How did you install Open3D?: From Poetry
- Compiler version (if built from source): not applicable

Additional information

No response

[Redsparkfish]

Have you solved it? I am encountered with the same problem.

[genevieve-le-houx]

Have you solved it? I am encountered with the same problem.

@Redsparkfish I'm still having this issue

[genevieve-le-houx]

I'm still having this issue with open3d version 0.16.0

I'm working on Windows11 with Python 3.8.7

[Redsparkfish]

That is sad. I have to write a ISS keypoints computation function manually.

[genevieve-le-houx]

That is sad. I have to write a ISS keypoints computation function manually.

@Redsparkfish Do you mind sharing your implementation? This would really help me. Thanks!

[Redsparkfish]

import os
import time
import trimesh
import numpy as np
import open3d as o3d
from sklearn.neighbors import NearestNeighbors

def getNeighbors(pcd: o3d.geometry.PointCloud(), radius):
points = np.asarray(pcd.points)
nn = NearestNeighbors(radius=radius)
nn.fit(points)
neighbor_indices = nn.radius_neighbors(return_distance=False)
return neighbor_indices

def computeISS(pcd: o3d.geometry.PointCloud(), t1=0.5, t2=0.5, radius=10):
point = np.asarray(pcd.points)
point_size = point.shape[0]
ISS = np.array([0, 0, 0])
weights = np.zeros(point_size)
neighborSet = getNeighbors(pcd, radius=radius)
for i in range(point_size):
if neighborSet[i].shape[0] == 0:
continue
weights[i] = 1 / len(neighborSet[i])
for i in range(point_size):
if neighborSet[i].shape[0] == 0:
continue
d_vectors = point[i] - point[neighborSet[i]]
X = np.multiply(weights[neighborSet[i]].reshape(d_vectors.shape[0], 1), d_vectors).T
Y = d_vectors
cov = np.dot(X, Y) / np.sum(weights[neighborSet[i]])
eigvalue, eigvector = np.linalg.eig(cov)
eigvalue = np.sort(eigvalue)
if t1 * eigvalue[1] > eigvalue[0] and t2 * eigvalue[2] > eigvalue[1]:
ISS = np.vstack((ISS, point[i]))
iss_pcd = o3d.geometry.PointCloud()
if ISS.shape[0] < 200:
iss_pcd.points = iss_pcd.points = o3d.utility.Vector3dVector(point)
else:
iss_pcd.points = o3d.utility.Vector3dVector(ISS[1:])
return iss_pcd

[DNKpp]

I've started using the ISS Keypoints myself and encountered some weird crashes, too. I debuged the resulting kp_indices and detected some very high indice values (far exceeding the actual point count), which is usually a very good indicator for a race condition.
I'm not too familiar with omp, but line 122 (the emplace_back line) seems very suspicious to me.. Marking it with #pragma omp critical (and thus guard it with a mutex) actually solved that crash for me.

[Hall-jFalise]

Update: Issue is solved, thanks to converting my voxel back as a pointcloud.
Hi,

Could we please open back this thread?
I seem to be encountering this bug in version 0.17, though I cannot get an error message (could give one, if somebody would help me with that).

In my application, I call the keypoint.compute_iss_keypoints(voxel) in a loop, where each time I feed a different point cloud (a croped section from a bigger point cloud). It crashes at random position, meaning its not the same input crashing the method.
I also tried to convert back the voxel object in a point cloud, but still crashes all the same, or maybe a bit less.

Also, I would appreciate if we could extend the input type to the voxels objects. I have solved this issue like so:

try:
      pcd_tmp = o3d.geometry.PointCloud()
      pcd_tmp.points = o3d.utility.Vector3dVector(self.voxel_pcl.points)
      keypoints = o3d.geometry.keypoint.compute_iss_keypoints(pcd_tmp)
  except Exception as e:
      print(f'Keypoint computing failed- Error: {e}')

System information

- Operating system: Windows 10 64-bit
- Python version: Python 3.10.13
- Open3D version: output from python: 0.17
- System architecture: x64
- Is this a remote workstation?: no
- How did you install Open3D?: From Pip
- Compiler version (if built from source): not applicable

[ssheorey]

Hi @Hall-jFalise the fix was merged after v0.17.

Please try with the latest development Python wheel from here:
http://www.open3d.org/docs/latest/getting_started.html#development-version-pip
and let us know if you still have this issue.

[Hall-jFalise]

Hi @Hall-jFalise the fix was merged after v0.17.

Please try with the latest development Python wheel from here: http://www.open3d.org/docs/latest/getting_started.html#development-version-pip and let us know if you still have this issue.

Thanks for the quick answer, it seems to be fixed now.
Could we extend the keypoint function to work with voxels? It seems that the instability came from there.

This is my fix:

pcd_tmp = o3d.geometry.PointCloud()
pcd_tmp.points = o3d.utility.Vector3dVector(self.voxel.points)
keypoints = o3d.geometry.keypoint.compute_iss_keypoints(pcd_tmp)