SegmentByGraphCut not Modifying faceBitSet

[emil-peters]

This morning, I experimented with the segmentByGraphCut function via the Python API as part of my segmentation pipeline.
The objective was to refine the segmentation by aligning the blue and purple teeth closer to the gingiva's curvature (see the attached image). However, after running the algorithm, I noticed no difference between the mesh before and after execution.
The output faceBitSet contained the same number of faces as the input faceBitSet, indicating that segmentByGraphCut did not modify the segmentation as anticipated.

I've provided a zip file containing the mesh, segmentation, and the code used for reproducing the issue

(it should work after installing requirements.txt and running segment_by_graphcut_minimal_example.py). Any insights on what might be causing this would be greatly appreciated. Thank you in advance!

segment_by_graphcut_example.zip

[emil-peters]

If you don't want to download the zip the code can be found here:

import json
import meshlib.mrmeshpy as mrmesh
import meshlib.mrmeshnumpy as mrmeshnumpy
import numpy as np
import trimesh
# tool link: https://www.youtube.com/watch?v=klJhdY0RYb0

def load_mesh_from_json(path):
    with open(path, "r") as f:
        data = json.load(f)
    mesh = trimesh.Trimesh(vertices=data["points"], faces=data["faces"])
    face_labels = np.array(data["faceLabels"]).astype(int)
    point_labels = np.array(data["pointLabels"]).astype(int)
    return mesh,face_labels,point_labels

def visualize_mesh_with_face_labels(mesh, face_labels):
    # Ensure numpy and trimesh are imported
    np.random.seed(0)  # Keep colors consistent
    
    # Create a unique color for each label
    unique_labels = np.unique(face_labels)
    color_map = np.random.rand(len(unique_labels), 3)
    
    # Create a map from label to color
    label_to_color = {label: color_map[i] for i, label in enumerate(unique_labels)}
    
    # Assign colors to faces based on their labels
    face_colors = np.array([label_to_color[label] for label in face_labels])
    
    # Update mesh with the new colors
    mesh.visual.face_colors = (face_colors * 255).astype(np.uint8)
    
    # Show the mesh
    mesh.show()
mesh, face_labels, point_labels = load_mesh_from_json("crown_mesh.json")
visualize_mesh_with_face_labels(mesh, face_labels)


mesh_labels = np.unique(face_labels[face_labels!=0])

mr_mesh = mrmeshnumpy.meshFromFacesVerts(mesh.faces, mesh.vertices)
new_face_labels_dict = {}
for i in mesh_labels:
    # example meshlib docs:
    
    # source = mrmesh.FaceBitSet()
    # sink = mrmesh.FaceBitSet()

    # # For mesh segmentation, one has to specify source and sink faces for it.
    # source.resize(mesh.topology.getValidFaces().size(),False)
    # sink.resize(mesh.topology.getValidFaces().size(),False)

    # source.set(mrmesh.FaceId(0),True)
    # sink.set(mrmesh.FaceId(23),True)
    
    
    crown_faces = (face_labels==i)
    print(i, "  ", np.sum(crown_faces))
    metric = mrmesh.edgeCurvMetric(mr_mesh)
    # metric = mrmesh.edgeLengthMetric(mr_mesh)
    source = mrmeshnumpy.faceBitSetFromBools(crown_faces)
    sink = mrmeshnumpy.faceBitSetFromBools(~crown_faces)
    


    res = mrmesh.segmentByGraphCut(mr_mesh.topology,source,sink,metric)
    face_bitset = res
    
    new_face_labels = mrmeshnumpy.getNumpyBitSet(res)
    print("new_face_labels length: ", np.sum(new_face_labels))
    new_face_labels_dict[i] = new_face_labels
    # mr_mesh.pack()
    
    # mrmesh.saveMesh(mr_mesh, mrmesh.Path("mesh_segmented.stl"))
    

new_face_labels = np.zeros_like(face_labels)
for i in new_face_labels_dict:
    print(i, "  ", np.sum(new_face_labels_dict[i]))
    new_face_labels[new_face_labels_dict[i]] = i
visualize_mesh_with_face_labels(mesh, new_face_labels)

[Grantim]

Hello!

    source = mrmeshnumpy.faceBitSetFromBools(crown_faces)
    sink = mrmeshnumpy.faceBitSetFromBools(~crown_faces)

If I got it correctly source + sink here covers all faces of the mesh, that's why segmentByGraphCut does not change anything. Its purpose is to find best boundary between source and sink (source and sink can only be expanded, so user input does not change labels) but in this example there is no free space to determine new boundary.

I suggest you such patch

    source = mrmeshnumpy.faceBitSetFromBools(crown_faces)
    sink = mrmeshnumpy.faceBitSetFromBools(~crown_faces)
+   mrmesh.shrink(mr_mesh.topology,source,3) # allow 3 faces lines free space (inside crown_faces) for boundary clarification
+   mrmesh.shrink(mr_mesh.topology,sink,3) # allow 3 faces lines free space (inside non crown_faces) for boundary clarification

[emil-peters]

Thanks works as expected now!