Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Shape detection with region growing #3864

Conversation

danston
Copy link
Contributor

@danston danston commented Apr 18, 2019

Shape detection with generic region growing - version 1.0

This submission proposes a major update to the Point_set_shape_detection_3 package by extending it with the generic region growing algorithm that works on a set of arbitrary items. We contribute the generic version of the region growing algorithm and its three particular instances: region growing on points in 2D and 3D, and region growing on a polygon mesh.

Summary of Changes

  • Renaming the package;
  • Refactoring the docs;
  • Removing current version of region growing on 3D points;
  • Adding a generic region growing algorithm;
  • Adding three instances of region growing: on 2D points, on 3D points, on a polygon mesh;
  • Adding all related region growing examples, tests, and benchmarks;
  • Cleaning all related RANSAC files.

Status

  • Proposed for acceptance by reviewers;
  • Developed and locally tested (MacOS/AppleClang).

Release Management

  • Affected package(s): Point_set_shape_detection_3
  • Feature: Generalized Region Growing Proposed for acceptance. --Andreas Fabri 16:20, 20 March 2019 (CET)
  • Link to compiled documentation (obligatory for small feature) link
  • License and copyright ownership: GPL (v3 or later)

TODO:

  • After merge, redirection should be put in place
  • News entry for cgal.org

@sgiraudot
Copy link
Contributor

@maxGimeno Can you help Dmitry on this PR? Travis fails, but I'm pretty sure it's something not code-related (but more of a config problem): this PR renames the package Point_set_shape_detection_3 as Shape_detection_3 and adds a bunch of new files. If I remember correctly, there is something to do to make Travis take it into account. Do you remember how?

@sgiraudot sgiraudot changed the title Shape detection with region growing [Feature] Shape detection with region growing Apr 19, 2019
@maxGimeno
Copy link
Contributor

Yes. You need to go to the cgal_git/.travis directory, and from there run

bash ./generate_travis.sh 

It will update the .travis.yml file, and then you can commit.

@danston
Copy link
Contributor Author

danston commented Apr 19, 2019

I see. Thanks!

Copy link
Member

@sloriot sloriot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cpp11 should be replaced by std and BOOST_FOREACH by c++11 for range loop.

@danston
Copy link
Contributor Author

danston commented Apr 24, 2019

@maxGimeno Could you please have a look at this pull again? I generated travis as you said, but it anyways failed. I then also modified a few doc references of the packages which depend on my new package and they also started failing. What do I do wrong? Is there any tutorial on github how to do a pull request correctly? Thanks!

@maxGimeno
Copy link
Contributor

Actually, reading the travis reports, i see that you include files that do not exist.
Travis is not just a bunch of green check marks and red crosses, you can see the log of the compilation and analyse the errors. The only tricky one is the CHECK test that explains what is wrong and how to fix it in echo calls, most of the time.
From what I understand, you are including Shape_detection_3.h when the file is actually Shape_detection.h.

/home/travis/build/CGAL/cgal/Advancing_front_surface_reconstruction/examples/Advancing_front_surface_reconstruction/reconstruction_structured.cpp:7:10: fatal error:
'CGAL/Shape_detection_3.h' file not found
#include <CGAL/Shape_detection_3.h>

/home/travis/build/CGAL/cgal/Classification/examples/Classification/example_cluster_classification.cpp:16:10: fatal error:
'CGAL/Shape_detection_3.h' file not found
#include <CGAL/Shape_detection_3.h>

/home/travis/build/CGAL/cgal/Point_set_3/examples/Point_set_3/point_set_algo.cpp:7:10: fatal error:
'CGAL/Shape_detection_3.h' file not found
#include <CGAL/Shape_detection_3.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~

/usr/local/include/CGAL/Shape_detection/Efficient_RANSAC/Efficient_RANSAC.h:29:10: fatal error:
'CGAL/function.h' file not found
#include <CGAL/function.h>

And the CHECK test indicates that you failed to update travis.yml.

@maxGimeno
Copy link
Contributor

When I try from your branch, I get this diff:

     - PACKAGE='AABB_tree Advancing_front_surface_reconstruction Algebraic_foundations '
     - PACKAGE='Algebraic_kernel_d Algebraic_kernel_for_circles Algebraic_kernel_for_spheres '
     - PACKAGE='Alpha_shapes_2 Alpha_shapes_3 Apollonius_graph_2 '
-    - PACKAGE='Arithmetic_kernel Arrangement_on_surface_2 BGL '
-    - PACKAGE='Barycentric_coordinates_2 Boolean_set_operations_2 Bounding_volumes '
-    - PACKAGE='Box_intersection_d CGAL_Core CGAL_ImageIO '
-    - PACKAGE='CGAL_ipelets Cartesian_kernel Circular_kernel_2 '
+    - PACKAGE='Arithmetic_kernel Arrangement_on_surface_2 Barycentric_coordinates_2 '
+    - PACKAGE='BGL Boolean_set_operations_2 Bounding_volumes '
+    - PACKAGE='Box_intersection_d Cartesian_kernel CGAL_Core '
+    - PACKAGE='CGAL_ImageIO CGAL_ipelets Circular_kernel_2 '
     - PACKAGE='Circular_kernel_3 Circulator Classification '
     - PACKAGE='Combinatorial_map Cone_spanners_2 Convex_decomposition_3 '
     - PACKAGE='Convex_hull_2 Convex_hull_3 Convex_hull_d '
@@ -38,16 +38,16 @@ env:
     - PACKAGE='Polyhedron_IO Polyline_simplification_2 Polynomial '
     - PACKAGE='Polytope_distance_d Principal_component_analysis Principal_component_analysis_LGPL '
     - PACKAGE='Profiling_tools Property_map QP_solver '
-    - PACKAGE='Random_numbers Ridges_3 STL_Extension '
-    - PACKAGE='Scale_space_reconstruction_3 Scripts SearchStructures '
-    - PACKAGE='Segment_Delaunay_graph_2 Segment_Delaunay_graph_Linf_2 Set_movable_separability_2 '
-    - PACKAGE='Shape_detection Skin_surface_3 Snap_rounding_2 '
-    - PACKAGE='Solver_interface Spatial_searching Spatial_sorting '
+    - PACKAGE='Random_numbers Ridges_3 Scale_space_reconstruction_3 '
+    - PACKAGE='Scripts SearchStructures Segment_Delaunay_graph_2 '
+    - PACKAGE='Segment_Delaunay_graph_Linf_2 Set_movable_separability_2 Shape_detection '
+    - PACKAGE='Skin_surface_3 Snap_rounding_2 Solver_interface '
+    - PACKAGE='Spatial_searching Spatial_sorting STL_Extension '
     - PACKAGE='Straight_skeleton_2 Stream_lines_2 Stream_support '
     - PACKAGE='Subdivision_method_3 Surface_mesh Surface_mesh_approximation '
-    - PACKAGE='Surface_mesh_deformation Surface_mesh_parameterization Surface_mesh_segmentation '
-    - PACKAGE='Surface_mesh_shortest_path Surface_mesh_simplification Surface_mesh_skeletonization '
-    - PACKAGE='Surface_mesher Surface_sweep_2 TDS_2 '
+    - PACKAGE='Surface_mesh_deformation Surface_mesher Surface_mesh_parameterization '
+    - PACKAGE='Surface_mesh_segmentation Surface_mesh_shortest_path Surface_mesh_simplification '
+    - PACKAGE='Surface_mesh_skeletonization Surface_sweep_2 TDS_2 '
     - PACKAGE='TDS_3 Testsuite Three '
     - PACKAGE='Triangulation Triangulation_2 Triangulation_3 '
     - PACKAGE='Union_find Visibility_2 Voronoi_diagram_2 '

@danston
Copy link
Contributor Author

danston commented Apr 24, 2019

@maxGimeno Aaah, ok, sorry I missed some logs, true. Thanks for pointing out. However, the PACKAGE check is still weird for me because I used the script generate_travis as you said and then committed it so I am still wondering why travis.yml is not updated.

@maxGimeno
Copy link
Contributor

@danston I think it is updated, just the order of the packages is different for some reason. It should be alphabetical but I see it is not. Maybe it is platform related, and bash doesn't act the same way on macOS or something. Anyways, I pushed the fix .

@danston
Copy link
Contributor Author

danston commented Apr 24, 2019

@maxGimeno Ok, thanks a lot. The order difference is strange. I will try to have a look at my system and see what could have gone wrong.

…_growing-danston' of github.com:danston/cgal into Point_set_shape_detection_3-make_it_general_with_region_growing-danston
@maxGimeno
Copy link
Contributor

Travis reports:

Differences in Shape_detection: Point_set_processing_3 have disappeared.
 Differences in Shape_detection: Arithmetic_kernel
Cartesian_kernel
Filtered_kernel
HalfedgeDS
Hash_map
Homogeneous_kernel
Intersections_2
Intersections_3
Modifier
Polyhedron
Polyhedron_IO
Shape_detection
Surface_mesh are new and not committed.

It means that when it compares the dependencies of the package in that branches to the ones reported in package)info, there are differences.
in CGAL//Shape_detection/package_info/Shape_detection/dependencies:
Point_set_processing_3 should be removed,
Arithmetic_kernel, Cartesian_kernel, Filtered_kernel, HalfedgeDS, Hash_map, Homogeneous_kernel, Intersections_2, Intersections_3, Modifier, Polyhedron, Polyhedron_IO, Shape_detection and Surface_mesh should be added. Unless some are unwanted, in which case you will have to track the headers that add those dependencies.

@maxGimeno
Copy link
Contributor

BGL is a new dependency misisng in the file. Is it wanted ?

@danston
Copy link
Contributor Author

danston commented Apr 26, 2019

Yes, I just did not have time to fix it yesterday.

@lrineau lrineau merged commit 5a4fa9e into CGAL:master Jun 5, 2019
lrineau added a commit that referenced this pull request Jun 5, 2019
…e_it_general_with_region_growing-danston

[Feature] Shape detection with region growing
@lrineau lrineau removed the rm only: ready for master For the release team only: that indicates that a PR is about to be merged in 'master' label Jun 5, 2019
@danston
Copy link
Contributor Author

danston commented Jun 5, 2019

Ok, I will prepare such a news entry and let you know.

@afabri
Copy link
Member

afabri commented Jun 12, 2019

The testsuite Ic-57 for VC++ fails.

Here

you have to replace CGAL::max(..) with (CGAL::max)(..)

@sloriot @sgiraudot : Shall one use CGAL::max or std::max ?

@sloriot
Copy link
Member

sloriot commented Jun 12, 2019

CGAL::max is the safest option IMO

@danston
Copy link
Contributor Author

danston commented Jun 12, 2019

I changed it in my fork, but the pull request is already closed. How should I do it now? Thanks.

@lrineau
Copy link
Member

lrineau commented Jun 12, 2019

@maxGimeno commented on Jun 3, 2019, 10:15 AM GMT+2:

Successfully tested in https://cgal.geometryfactory.com/CGAL/testsuite/results-5.0-Ic-49.shtml

@maxGimeno Please be careful when you mark pull-requests as tested. The results for CGAL-5.0-Ic-49 were incomplete: with any tests on Windows.


@danston commented on Jun 12, 2019, 11:00 AM GMT+2:

I changed it in my fork, but the pull request is already closed. How should I do it now? Thanks.

Dmitry, you will have to open a new pull-request. It can use the same branch you already have.

@danston
Copy link
Contributor Author

danston commented Jun 12, 2019

@lrineau Ok, clear. I will open a new one. But how come the test suite is incomplete? I checked it and everything was green. Do I miss smth?

@afabri
Copy link
Member

afabri commented Jun 12, 2019

Can Maxime not do that directly in master?

@maxGimeno
Copy link
Contributor

No. I can't push in master.

@danston
Copy link
Contributor Author

danston commented Jun 12, 2019

Here is the new pull request: #3994

@lrineau
Copy link
Member

lrineau commented Jun 12, 2019

@danston commented on Jun 12, 2019, 11:47 AM GMT+2:

@lrineau Ok, clear. I will open a new one. But how come the test suite is incomplete? I checked it and everything was green. Do I miss smth?

It was incomplete because that day some of the test platforms had network or power issues, and had not sent their results.

@danston
Copy link
Contributor Author

danston commented Jun 12, 2019

@lrineau Ok, I see. Next time, I will wait longer before reporting my results. Thanks for clarifying.

@maxGimeno
Copy link
Contributor

You don't have to report your results, I check the testsuite every morning and report the points of interest. We were just unlucky to not notice the missing platforms this time.

@MaelRL
Copy link
Member

MaelRL commented Jun 27, 2019

As it is a feature / major update, it would be nice to have a small text with a couple images for a news entry on cgal.org (and a neat, large image for the front page?).

Bump! :)

@danston
Copy link
Contributor Author

danston commented Jul 5, 2019

I remember about the news entry. I still need to finish a few other things and then I will immediately prepare a picture and a short description, just the picture I want to do requires a bit of coding :) Thanks for the patience.

@MaelRL
Copy link
Member

MaelRL commented Aug 6, 2019

- `RegionType::update()` - updates the internal flag from the default `false` to `true`.

We also define a `SeedMap`, such that the second object is handled first, while the first object follows.
Moreover, the last object is always skipped. Notice that in this example, the container with objects is `std::list`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice that in this example, the container with objects is std::list

That's not the case region_growing_with_custom_classes.cpp uses a vector: std::vector<std::size_t> neighbors;

Yet the code traverses the vector as if it was a linked list:

    void operator()(
      const std::size_t query_index, 
      std::vector<std::size_t>& neighbors) const {
      
      std::size_t i = 0;
      for (const auto& object : m_objects) {
        if (i == query_index) {
          
          neighbors = object.neighbors;
          return;
        } ++i;
      }
    }

That is rather confusing.

Perhaps it once was a list but then the type was changed to a vector before the PR was submitted/merged, without updating the algorithm and docs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nh2 Hi. You are completely right! Thank you for this remark! It is my fault. Indeed, I first used std::list and then updated to std::vector, because some reviewers did not like using the list, but I forgot to update the docs for some reason. I will fix it. However, note that RG can still work with std::list. In general, std::vector is not required, it is used only because it is more efficient.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danston Makes sense!

Another quick question:

From the added docs, on region growing, it doesn't seem to become quite clear how max_accepted_angle works. The docs say:

angle_threshold - the maximum accepted angle between the normal associated with a point and the normal of a line/plane;

This suggests that the angle is indeed between each individual face and the fitted plane. However, I have observed the algorithm to produce two things that seem impossible under that interpretation, using face-based plane fitting:

  • With an angle_threshold of 90: When I project associated faces onto the fitted plane, some project normals have opposite signum to the plane's normal
  • With an angle_threshold of 70: Two noisy parallel planes facing each other being detected as part of the same region (shouldn't that be impossible, given that their face normals pretty much look at each other?)

I haven't fully read all implementation yet; maybe you can give me a quick hint at whether that's intended, and how we could make the documentation best reflect what's happening?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I found the reason for the above:

In

Vector_3 face_normal;
get_face_normal(face, face_normal);
const FT distance_to_fitted_plane =
get_max_face_distance(face);
const FT cos_value =
CGAL::abs(m_scalar_product_3(face_normal, m_normal_of_best_fit));

we have abs() which throws away the sign of the normal, thus also permitting faces that are within 180 <= here < 180-angle_threshold degrees!

E.g. assume your fit plane's normal points exactly upwards. If you set the angle_threshold to, say, 5 degrees, this will not permit angles that are "pretty much exactly upwards" but also "pretty much exactly downwards".

Is that intended?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed another problem: The plane's normal is not estimated correctly (it is arbitrary, likely depending on the position of the plane relative to the origin).

I've put a fix for the above 2 issues into PR #4563.

nh2 added a commit to nh2/cgal that referenced this pull request Mar 6, 2020
In CGAL#3864 region growing was introduced, but the
`linear_least_squares_fitting_3()` used in `update()` cannot
estimate a correct face normal direction from just unoriented vertex points
(the plane normal's orientation signum may be flipped).

This commit flips the estimated plane normal to face into the
same direction as the majority of the faces it is estimated from.

With the estimated planes now being "the right way around",
this commit subsequently also fixes the incorrect use of `abs()`
in `is_part_of_region()` when comparing a face's normal to the
estimated plane normal (direction matters here).

This fixes parallelly-opposing planar regions (means: with opposite normals),
connected with some triangles in between, like this:

      | ->      <- |
      | ->      <- |      Legend:
      | ->      <- |        face:   |
      | ->      <- |        normal: ->
      | ->      <- |
       \__________/

being detected as 1 region.

This should have been impossible according to the docs
("... and if the angle between its normal and the plane's normal is within
the `angle_threshold`"), given the documented requirement
("\pre `angle_threshold >= 0 && angle_threshold <= 90`").
Now it is impossible.
nh2 added a commit to nh2/cgal that referenced this pull request Mar 6, 2020
In CGAL#3864 region growing was introduced, but the
`linear_least_squares_fitting_3()` used in `update()` cannot
estimate a correct face normal direction from just unoriented vertex points
(the plane normal's orientation signum may be flipped).

This commit flips the estimated plane normal to face into the
same direction as the majority of the faces it is estimated from.

With the estimated planes now being "the right way around",
this commit subsequently also fixes the incorrect use of `abs()`
in `is_part_of_region()` when comparing a face's normal to the
estimated plane normal (direction matters here).

This fixes parallelly-opposing planar regions (means: with opposite normals),
connected with some triangles in between, like this:

      | ->      <- |
      | ->      <- |      Legend:
      | ->      <- |        face:   |
      | ->      <- |        normal: ->
      | ->      <- |
       \__________/

being detected as 1 region.

This should have been impossible according to the docs
("... and if the angle between its normal and the plane's normal is within
the `angle_threshold`"), given the documented requirement
("\pre `angle_threshold >= 0 && angle_threshold <= 90`").
Now it is impossible.
nh2 added a commit to nh2/cgal that referenced this pull request Mar 6, 2020
In CGAL#3864 region growing was introduced, but the
`linear_least_squares_fitting_3()` used in `update()` cannot
estimate a correct face normal direction from just unoriented vertex points
(the plane normal's orientation signum may be flipped).

This commit flips the estimated plane normal to face into the
same direction as the majority of the faces it is estimated from.

With the estimated planes now being "the right way around",
this commit subsequently also fixes the incorrect use of `abs()`
in `is_part_of_region()` when comparing a face's normal to the
estimated plane normal (direction matters here).

This fixes parallelly-opposing planar regions (means: with opposite normals),
connected with some triangles in between, like this:

      | ->      <- |
      | ->      <- |      Legend:
      | ->      <- |        face:   |
      | ->      <- |        normal: ->
      | ->      <- |
       \__________/

being detected as 1 region.

This should have been impossible according to the docs
("... and if the angle between its normal and the plane's normal is within
the `angle_threshold`"), given the documented requirement
("\pre `angle_threshold >= 0 && angle_threshold <= 90`").
Now it is impossible.
danston added a commit to danston/cgal that referenced this pull request Mar 11, 2020
danston added a commit to danston/cgal that referenced this pull request Mar 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants