Image masking #715

monsieurgustav · 2019-11-20T08:46:30Z

Description

Support image masking to control 3D reconstruction.

Features list

Add imageMasking app implementing an HSV based keying
Update featureExtraction to filter features based on image masks
Update prepareDenseScene to output undistorted images with alpha channel based on masks

Implementation remarks

…eigen

[doc] INSTALL.md: add boost-timer and lz4 to vcpkg install command

As they can be included in the SfMData

…ow a warning instead of an error

…lder names

* always use the file extension in the UID * use file system date/time if there is no date/time metadata

…rting with valid poses When starting with existing poses: 1/ If we don't have any landmark, we need to triangulate them from the known poses. 2/ But even if we already have landmarks, we need to try to triangulate new points with the current set of parameters.

Workaround for a build incompatibility between prettyprint and eigen

Add Oneplus 5T sensor width to the database

[sensorDB] add sony IMX

Added working infos in LG G6 camera sensors

…NbCamerasForCalibration

…romSingleCamera

…v/improveRigSupport

…improveRigSupport

…olume If we have a mask in input to filter the values, all the similarity voxels will have exactly the worst value 255. So the output of retrieveBestZ should be "no valid depth" instead of returning the last plane of the similarity volume.

…v/improveRigSupport

fabiencastan · 2019-11-20T10:02:53Z

It works fine to mask features in FeatureExtraction, but I get troubles when using it in input of PrepareDenseScene:
PrepareDenseScene gives me: "Invalid image mask size: mask is ignored." probably due to image orientation.

The surprising thing is that the orientation of the mask is the same when used on FeatureExtraction where there is no problem...

- move to c++14 to use auto in function return type without explicit information. - additional typename/using needed to resolve template type value

monsieurgustav · 2019-11-21T09:15:05Z

The issue in PrepareDenseScene is fixed

simogasp · 2019-11-21T09:48:29Z

src/aliceVision/imageMasking/eigen2cvHelpers.hpp

+template<> class DataType<aliceVision::image::RGBColor>
+{
+public:
+    typedef uchar       value_type;


prefer using value_type = uchar; rather than typedef

It's an OpenCV template specialization.
Other specializations of this template in OpenCV are written using typedef, so I prefer to use the same syntax.

semantically they are the same thing, it's just a raccomended way to use using
https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#Rt-using

I think it's better to be consistent in our code, maybe at some point they will change that in opencv too

simogasp · 2019-11-21T09:51:49Z

src/aliceVision/imageMasking/imageMasking.cpp

+namespace imageMasking{
+
+namespace{
+  float clamp(float value)


do not indent for namespace

I did indent anonymous namespace to make obvious that included functions are local. I feel it's useful and does not hurt readability too much because only few functions are defined in anonymous namespace.

for completely new file I'd prefer to stick with the new formatting rules, we need to start at some point. But it\s true that we need to do that in batch at some point.

src/aliceVision/imageMasking/imageMasking.cpp

simogasp · 2019-11-21T09:56:04Z

src/aliceVision/imageMasking/imageMasking.hpp

+/**
+ * @brief Create a mask based on the hue of each pixel.
+ *
+ * @param[in] result The resulting binary mask.


simogasp · 2019-11-21T09:57:14Z

src/aliceVision/imageMasking/imageMasking.hpp

+ */
+void hsv(OutImage& result, const InImagePath& inputPath, float hue, float hueRange, float minSaturation, float maxSaturation, float minValue, float maxValue);
+
+void postprocess_invert(OutImage& result);


please add the doxygen if these are meant to be exposed

simogasp · 2019-11-21T10:07:05Z

src/CMakeLists.txt

-endif(NOT CXX11_COMPILER)
-set(CMAKE_CXX_STANDARD 11)
-set(CMAKE_CXX_STANDARD_REQUIRED ON)
+## ==============================================================================


@fabiencastan ?

simogasp · 2019-11-21T10:10:04Z

src/CMakeLists.txt

-# C++11
-set(CMAKE_CXX_STANDARD 11)
+# C++14
+set(CMAKE_CXX_STANDARD 14)


if we go with this, please update INSTALL.md, at the beginning

C/C++ compiler (gcc or visual studio or clang) with C++14 support (i.e. gcc >= 5, clang >= 3.4, msvc >=19).

@fabiencastan ?

Yes, good point. I will fix that.

Just checking, I guess it does not affect MeshroomMaya, if i remember correctly the gcc version used to compile maya plugins tended to be very conservative in the past. Dunno if it is still the case, just wanted to remind that in case.

For maya 2019
https://around-the-corner.typepad.com/adn/2019/01/maya-2019-api-updates-guide.html
Seems to be fine actually, maybe to check windows because they require Visual Studio 2015 Update 3 that sports msvc 19.00 while the full c++14 comes with later versions. I don't know if cmake check the compiler must fully support the standard or just some of it

for previous versions i guess it depends again on how strict is cmake and if the needed feature was already implemented
https://around-the-corner.typepad.com/adn/2018/02/compiler-versions-for-maya-2018.html

simogasp · 2019-11-21T12:00:11Z

src/cmake/Helpers.cmake

@@ -162,26 +162,29 @@ function(alicevision_add_software software_name)
    list(APPEND SOFTWARE_SOURCE "${CMAKE_CURRENT_BINARY_DIR}/${software_name}_version.rc")
  endif()

-  add_executable(${software_name} ${SOFTWARE_SOURCE})
+  add_executable(${software_name}_exe ${SOFTWARE_SOURCE})


not a big fan of this. I imagine it's because of name conflicts.
I think it makes the cmake code less readable, as one expect that the target name given is used to define the target with that name. And then in the code if u need to do something with the target you need to know that a suffix is added.

Maybe instead of masking we can use segmentation, as in this case we are more generating the mask by processing the image rather than performing actions with a mask. That does not solve the name conflict but maybe we give the library a different, more general, broader meaning (eg imageProcessing on the same line as imgproc of opencv)

I don't like the idea of having to play with name variantes, just to avoid stupid cmake conflicts.
I agree that the exe suffix is hidden, but we cannot make the alias explicit as the output will get this name. It looks to me as the least worst solution, but I'm open to other suggestions if you have better idea... other than playing with name variantes which is a pain we have for too long.

i was not suggesting to use alias or variants, i was just arguing that in cv masking and segmentation are different, and here we are doing segmentation (ie generating the mask) rather than masking (performing operations with existing masks, as eg in feature extraction). Nothing to do with the cmake conflict. That being said, the suggestion of having a lib of broader spectrum like imageProcessing (other algo may coming up in the future) would help to avoid the conflict.

to at least preserve the encapsulation (ie not see the use of suffix _exe outside the call to add_software) we can do this for the targets with optional dependencies:

set(<targetname>_LINKS aliceVision_sfmData aliceVision_sfmDataIO etc etc) if(ALICEVISION_HAVE_CCTAG) list(APPEND <targetname>_LINKS CCTag::CCTag) endif() alicevision_add_software(<targetname> SOURCE main_<targetname>.cpp FOLDER ${FOLDER_SOFTWARE_PIPELINE} LINKS <targetname>_LINKS ...

at least everything stays in alicevision_add_software and we have not to go back to the target and use target_link_libraries. A bit more verbose but it seems cleaner to me. :-)

…n/AliceVision into imageMasking

…aleThreshold - option to load another image than the original for creating the mask like an input depth map provided by a capture device. In that case, rescale the output mask to the original image size. - new algorithm autoGrayscaleThreshold using Otsu binarization - add to_lower in algorithm stringToEnum conversion

…patibility with g++-4.8

Federica Mauriello and others added 30 commits October 4, 2019 10:39

added correct infos on LG g6 camera sensors

874b831

[doc] INSTALL.md: add boost-timer and lz4 to vcpkg install command

0747f01

minor workaround for a build incompatibility between prettyprint and …

de49312

…eigen

Merge pull request #704 from alicevision/doc/installVcpkgArgs

58e0763

[doc] INSTALL.md: add boost-timer and lz4 to vcpkg install command

[sfm] typo fixes

9cc67bd

[software] SfM: features/matches folders are now optional

79cbe6e

As they can be included in the SfMData

[software] CameraInit: having different number of poses in a RIG is n…

ca6022b

…ow a warning instead of an error

[software] CameraInit: improve extraction of rig numbers from file/fo…

2715dbf

…lder names

[software] sfmTransfer: improve rig transfer

8c90106

[sfmDataIO] fix import of SfMData in Alembic containing rigs

732adf7

[sfmData] check if pose and intrinsics is defined with a rig

0a465dc

[sfmData] improve UID computation

0213d7d

* always use the file extension in the UID * use file system date/time if there is no date/time metadata

Add Oneplus 5T sensor width to the database

3e9a09b

[sensorDB] add sony IMX

a9501ff

Merge pull request #705 from alicevision/fix/buildConflictPrettyprint

c04ef2d

Workaround for a build incompatibility between prettyprint and eigen

Merge pull request #706 from labellson/develop

d335224

Add Oneplus 5T sensor width to the database

Merge pull request #707 from alicevision/sensordb-sony-imx

7937641

[sensorDB] add sony IMX

Merge pull request #699 from eadensplace/add-lg-g6-camera-infos

5bdfe01

Added working infos in LG G6 camera sensors

[sfm] fix sfm transform when using camera rigs

792b0f9

[sfm] expose parameters minNbCamerasToRefinePrincipalPoint and rigMin…

2187d4a

…NbCamerasForCalibration

[sfm] enable usage of simplified regex in computeNewCoordinateSystemF…

c0e1dab

…romSingleCamera

[sfm] fix sfmTransform for cameras rig

2e8b6f6

[software] new utility to log distances between markers or cameras

4040d37

[software] sfmTransform: fix translate if rotation is disabled

66e258c

Merge remote-tracking branch 'origin/wip-devDepthListUnified' into de…

4faaa71

…v/improveRigSupport

Merge remote-tracking branch 'origin/dev/improveRigSupport' into dev/…

c5680fb

…improveRigSupport

Merge remote-tracking branch 'origin/develop' into dev/improveRigSupport

86cff57

[masking] add imageMasking lib & exe

73b903e

[masking] implement HSV masking

afec38e

fabiencastan added 2 commits November 19, 2019 19:48

Merge remote-tracking branch 'origin/wip-devDepthListUnified' into de…

8a40ecc

…v/improveRigSupport

fabiencastan added 2 commits November 20, 2019 16:22

[cmake] fix softwares target names

51203f8

Build fix for g++: move to c++14 and typename resolution

a837e02

- move to c++14 to use auto in function return type without explicit information. - additional typename/using needed to resolve template type value

natowi added the feature label Nov 20, 2019

[prepareDenseScene] fix mask is always empty

d183cf9

simogasp requested changes Nov 21, 2019

View reviewed changes

simogasp reviewed Nov 21, 2019

View reviewed changes

Guillaume Buisson and others added 3 commits November 21, 2019 14:19

[imageMasking] add document, fix indent, use using

656a2fe

[doc] update INSTALL about C++14 requirement

8a2b762

Merge branch 'dev/improveRigSupport' of https://github.com/alicevisio…

f0fd0c8

…n/AliceVision into imageMasking

fabiencastan mentioned this pull request Nov 26, 2019

[request] Image Mask bulk generation alicevision/Meshroom#713

Open

fabiencastan added 2 commits December 17, 2019 12:15

[depthMap] update error message

2a50b04

[software] sfmDistance: fix build without cctag

63b480d

fabiencastan mentioned this pull request Dec 17, 2019

Image masking alicevision/Meshroom#188

Open

fabiencastan added 3 commits December 17, 2019 22:31

[build] rollback to c++11

ae3aed9

[software] cameraInit: switch from std::regex to boost::regex for com…

9112e1b

…patibility with g++-4.8

ChemicalXandco mentioned this pull request Jan 2, 2020

WIP Masking interface alicevision/Meshroom#750

Closed

fabiencastan added this to the 2020.2.0 milestone Jan 20, 2020

fabiencastan merged commit 4c8da93 into wip-devDepthListUnified Jan 20, 2020

fabiencastan deleted the imageMasking branch January 20, 2020 10:37

fabiencastan mentioned this pull request Jan 20, 2020

Depth map refactoring #619

Merged

8 tasks

This was referenced Jan 20, 2020

Trying to get a scan of upper body, but getting hole where person should be alicevision/Meshroom#760

Closed

[request] BoundingBox filter node alicevision/Meshroom#744

Closed

monsieurgustav mentioned this pull request Jan 21, 2020

Image masking alicevision/Meshroom#708

Merged

1 task

elishahung mentioned this pull request Aug 28, 2020

綠幕去背 MoonShineVFX/4drec#4

Closed

3 tasks

fabiencastan removed this from the 2.4.0 milestone Feb 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image masking #715

Image masking #715

monsieurgustav commented Nov 20, 2019 •

edited

Loading

fabiencastan commented Nov 20, 2019

monsieurgustav commented Nov 21, 2019

simogasp Nov 21, 2019

monsieurgustav Nov 21, 2019

simogasp Nov 21, 2019

simogasp Nov 21, 2019

monsieurgustav Nov 21, 2019

simogasp Nov 21, 2019

simogasp Nov 21, 2019

simogasp Nov 21, 2019

simogasp Nov 21, 2019

monsieurgustav Nov 21, 2019

simogasp Nov 21, 2019

monsieurgustav Nov 21, 2019

fabiencastan Nov 21, 2019

simogasp Nov 21, 2019

simogasp Nov 22, 2019

simogasp Nov 21, 2019

fabiencastan Nov 21, 2019 •

edited

Loading

simogasp Nov 21, 2019

Image masking #715

Image masking #715

Conversation

monsieurgustav commented Nov 20, 2019 • edited Loading

Description

Features list

Implementation remarks

fabiencastan commented Nov 20, 2019

monsieurgustav commented Nov 21, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabiencastan Nov 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

monsieurgustav commented Nov 20, 2019 •

edited

Loading

fabiencastan Nov 21, 2019 •

edited

Loading