Testing booleans of meshes

[starseeker]

Hi! Apologies if this is just me messing up, but I'm trying to wire manifold into my application's CSG boolean evaluation tree and I'm not sure exactly what I'm hitting. I'm feeding in two triangle meshes, creating manifolds, calling boolean, and then getting a mesh back from the boolean result to return to my program. However, I'm hitting a crasher with a subtraction of two cylinders: https://brlcad.org/~starseeker/b1.obj and https://brlcad.org/~starseeker/b2.obj (the crash is in my app - these are obj exports of our internal data structures.) The error is "terminate called after throwing an instance of 'std::out_of_range' what(): Vec out of range"

#9  0x00007fffee0ebd00 in manifold::Manifold::Impl::UpdateVert(int, int, int) [clone .cold] ()
    at /home/user/manifold-install/lib/libmanifold.so
#10 0x00007fffee12b91b in manifold::Manifold::Impl::FormLoop(int, int) ()
    at /home/user/manifold-install/lib/libmanifold.so
#11 0x00007fffee12c9c6 in manifold::Manifold::Impl::CollapseEdge(int, std::vector<int, std::allocator<int> >&) ()
    at /home/user/manifold-install/lib/libmanifold.so
#12 0x00007fffee12e57e in manifold::Manifold::Impl::SimplifyTopology() ()
    at /home/user/manifold-install/lib/libmanifold.so
#13 0x00007fffee10c08f in manifold::Boolean3::Result(manifold::OpType) const ()
    at /home/user/manifold-install/lib/libmanifold.so
#14 0x00007fffee1257e4 in manifold::CsgOpNode::ToLeafNode() const () at /home/user/manifold-install/lib/libmanifold.so
#15 0x00007fffee1455ec in manifold::Manifold::GetCsgLeafNode() const ()
    at /home/user/manifold-install/lib/libmanifold.so
#16 0x00007fffee14591e in manifold::Manifold::Status() const () at /home/user/manifold-install/lib/libmanifold.so
#17 0x00007fffee187557 in bool_meshes(double**, int*, unsigned int**, int*, manifold::OpType, double*, int, unsigned int*, int, double*, int, unsigned int*, int)

bool_meshes is my function that does the manifold steps and data translation:

  long
  bool_meshes(
          double **o_coords, int *o_ccnt, unsigned int **o_tris, int *o_tricnt,
          manifold::OpType b_op,
          double *a_coords, int a_ccnt, unsigned int *a_tris, int a_tricnt,
          double *b_coords, int b_ccnt, unsigned int *b_tris, int b_tricnt)
  {
      if (!o_coords || !o_ccnt || !o_tris || !o_tricnt)
          return 0;
  
      manifold::Mesh a_mesh;
      for (int i = 0; i < a_ccnt; i++)
          a_mesh.vertPos.push_back(glm::vec3(a_coords[3*i], a_coords[3*i+1], a_coords[3*i+2]));
      for (int i = 0; i < a_tricnt; i++)
          a_mesh.triVerts.push_back(glm::vec3(a_tris[3*i], a_tris[3*i+1], a_tris[3*i+2]));
      manifold::Mesh b_mesh;
      for (int i = 0; i < b_ccnt; i++)
          b_mesh.vertPos.push_back(glm::vec3(b_coords[3*i], b_coords[3*i+1], b_coords[3*i+2]));
      for (int i = 0; i < b_tricnt; i++)
          b_mesh.triVerts.push_back(glm::vec3(b_tris[3*i], b_tris[3*i+1], b_tris[3*i+2]));
  
      manifold::Manifold a_manifold(a_mesh);
      if (a_manifold.Status() != manifold::Manifold::Error::NoError) {
          bu_log("Error - a invalid\n");
          return 0;
      }
      manifold::Manifold b_manifold(b_mesh);
      if (b_manifold.Status() != manifold::Manifold::Error::NoError) {
          bu_log("Error - b invalid\n");
          return 0;
      }
  
      manifold::Manifold result = a_manifold.Boolean(b_manifold, b_op);
      if (result.Status() != manifold::Manifold::Error::NoError) {
          bu_log("Error - bool result invalid\n");
          return 0;
      }
      manifold::Mesh rmesh = result.GetMesh();
  
      (*o_coords) = (double *)calloc(rmesh.vertPos.size()*3, sizeof(double));
      (*o_tris) = (unsigned int *)calloc(rmesh.triVerts.size()*3, sizeof(unsigned int));
      for (size_t i = 0; i < rmesh.vertPos.size(); i++) {
          (*o_coords)[3*i] = rmesh.vertPos[i].x;
          (*o_coords)[3*i+1] = rmesh.vertPos[i].y;
          (*o_coords)[3*i+2] = rmesh.vertPos[i].z;
      }
      for (size_t i = 0; i < rmesh.triVerts.size(); i++) {
          (*o_tris)[3*i] = rmesh.triVerts[i].x;
          (*o_tris)[3*i+1] = rmesh.triVerts[i].y;
          (*o_tris)[3*i+2] = rmesh.triVerts[i].z;
      }
      *o_ccnt = (int)rmesh.vertPos.size();
      *o_tricnt = (int)rmesh.triVerts.size();
  
      return rmesh.triVerts.size();
  }

I seem to get a lot of successful evaluations before hitting this failure, so I thought I had the basic steps correct, but perhaps I need something else for improved robustness? Or perhaps I'm not using the Manifold setup correctly to identify invalid inputs? Any insight appreciated - thanks!

[pca006132]

You hit #535, which causes non-manifold output, which break our decimator as well.

@elalish is working on the triangulator fix, so you probably have to wait a bit. Another way would be to revert #528, which may give you overlapping triangulation (invalid geometry) but that will not cause invalid topology.

[pca006132]

Closing this as this is a duplicate of #535.