New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image masking #188

Open
robertguetzkow opened this Issue Aug 13, 2018 · 18 comments

Comments

Projects
None yet
9 participants
@robertguetzkow
Copy link

robertguetzkow commented Aug 13, 2018

I'd like to suggest image masking as a feature for a future version of Meshroom. In case this would be implemented, masked areas should not be used for keypoint detection nor for depth maps and texture generation. A simple import of masks in the form of binary images would be enough, a build in editor is not necessary.

Thank you for developing this great photogrammetry software!

@finnbear

This comment has been minimized.

Copy link

finnbear commented Aug 26, 2018

@robertguetzkow Can I ask how you would create your binary masks? I don't know how I would do that right now so I would want a built in editor.

@octupe

This comment has been minimized.

Copy link

octupe commented Aug 27, 2018

You could just paint in some rough mask per image in GIMP or photoshop and save it as a PNG using the image name it applies to with a predetermined extension or suffix.

@robertguetzkow

This comment has been minimized.

Copy link
Author

robertguetzkow commented Sep 16, 2018

@finnbear exactly as @octupe described. Sorry been on vacation, that's why I didn't respond sooner.
GIMP, Krita, Affinity Photo, Photoshop or any other image editing software that allows to use layers.

@fabiencastan

This comment has been minimized.

Copy link
Member

fabiencastan commented Sep 16, 2018

I agree that this would be a good feature.

In the meantime, a simple workaround is to apply your mask on the input pixel (RGB=0). There is no drawback in doing that. You just need to be careful to keep your image metadata (if your image editing software doesn't preserve metadata, you can transfer metadata from one image to another with exiftool).

@robertguetzkow

This comment has been minimized.

Copy link
Author

robertguetzkow commented Sep 16, 2018

@fabiencastan wouldn't that workaround select edges in the RGB=0 area as features, therefore possibly mismatching images when masking multiple objects/areas?

Yes, metadata needs to be preserved for focal length calculation. Exiftool is very useful if you need to manually add, edit or copy exif data. I definitely recommend it.

@fabiencastan

This comment has been minimized.

Copy link
Member

fabiencastan commented Sep 16, 2018

Yes sure but in practice it works quite fine.
I have always used soft-edges on my masks.

@natowi

This comment has been minimized.

Copy link

natowi commented Oct 26, 2018

I think a masking node could be based on G'MIC.

@ChemicalXandco

This comment has been minimized.

Copy link

ChemicalXandco commented Oct 26, 2018

If there was to be built-in masking, a hierarchical agglomerative clustering procedure that expands the background and foreground regions from the initial strokes seems to be the best solution.

@zfarm

This comment has been minimized.

Copy link

zfarm commented Nov 3, 2018

Sorry Im a bit new. Does this entail pulling each photo into an editor and painting out what isnt necessary for the scan?
I agree a bounding volume you could define once the point cloud was generated would save a lot of time .

@robertguetzkow

This comment has been minimized.

Copy link
Author

robertguetzkow commented Nov 4, 2018

@zfarm yes that was my idea. It has the benefit that there are already plenty image editors out there that are stable and it would greatly reduce the necessary work to integrate this feature. Integrating a mask editor into Meshroom seems overkill to me.

@natowi

This comment has been minimized.

Copy link

natowi commented Nov 4, 2018

@robertguetzkow Of course you can apply masks to a few images or paint out unwanted areas by hand - but what about datasets with 100+ images? GMIC can extract the foreground and create reject masks - but not in bulk.
I was thinking more of Image Segmentation like this to automatically identify and select the object in multiple images and create masks.

@zfarm Yes, a bounding volume/Mask (as a tool in the 3D Viewer?) for the point cloud would be nice.
Maybe PCL (->github) could be useful here.

@robertguetzkow

This comment has been minimized.

Copy link
Author

robertguetzkow commented Nov 4, 2018

@natowi sure I can think of several more automated and sophisticated approaches, but somebody has to implement it as well. I was just being pragmatic. Having any masking at all would be an improvement.
In case somebody wants to try your segmentation approach, I'd highly suggest to not rely on automated processing alone. There should always be a way to make manual adjustments to the masks.

@natowi

This comment has been minimized.

Copy link

natowi commented Nov 5, 2018

I'd highly suggest to not rely on automated processing alone

@robertguetzkow I agree
When I have some time, I´ll look for a suitable library or tool we could use for this task and think about a way to integrate it in the Meshroom workflow. (This is a good starting point)

@jumpjack

This comment has been minimized.

Copy link

jumpjack commented Jan 19, 2019

Have a look to 3df Zephyr "Masquerade" plugin, which you can start from inside the program right after importing images at a click o f abutton: it makes you draw on first image which parts to take into account and which to ignore, and then you can propagate same mask to all images with one click.
Very useful if you have fixed obstructed area on your images.

@natowi

This comment has been minimized.

Copy link

natowi commented Jan 19, 2019

@jumpjack I am looking for suitable open-source projects. Here is a list I started.

@jumpjack

This comment has been minimized.

Copy link

jumpjack commented Jan 21, 2019

I was talking about the idea: an "image mask editor" embedded in Meshroom rather than an external program.

@natowi

This comment has been minimized.

Copy link

natowi commented Jan 21, 2019

@jumpjack Yes, I know. That´s why I am looking for an open-source project with a compatible license and good usability/results which could be merged with Meshroom, as the main devs have enough to do (I think) with working on Meshroom core features.

@Misterdudeman

This comment has been minimized.

Copy link

Misterdudeman commented Jan 29, 2019

I'd be interested in something that could accept an alpha channel from a png or tiff as a form of mask input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment