Problem with image generation

[juvinski]

Hi,

I'm using ODM and after some map generation, there is an issue with the final products. On the final image - orthomosaic - there are lines like that:


How can I debug or try to see what is been done by mvs-texturing to figure out what is the cause of this error - the source images doesn't have this line BTW.

Best regards

[nmoehrle]

This looks to me like an artifact of the local seam leveling. Could you show me the wireframe of at one of these edges?

Best

[juvinski]

Hi @nmoehrle thanks for your reply.
I'm pretty new on MVS, how can I get this wireframes ?

Best

[pierotofy]

Open the textured mesh (.obj) from the odm_texturing folder in MeshLab, there’s an option to enable wireframe view there.

I’ve also seen these artifacts before, would love to learn more about possible causes and how to prevent them.

[juvinski]

Ok, I will try, thanks a lot @pierotofy

[juvinski]

Hi,

I made the procedure to access the wireframe

The 3d image showing a problem area and the wireframe.
I coudn't see anything but if you can I put all texturing objects at this drive's folder
https://drive.google.com/open?id=1_JB-ofIcUSW2lS_qdERF6A9gGmHpmmeT

[pierotofy]

@juvinski for the future, use "flat lines" and toggle the lights.

Here's a good screenshot.

A few other artifacts from the same dataset.

[juvinski]

Really nice @pierotofy
thanks a lot for the information. I'm looking more this meshlab tool and is really nice :)

[pierotofy]

Just encountered yet another dataset with the artifacts:

[nmoehrle]

I don't have an explanation for these artifacts, my first guess was an issue with the local seam leveling but those artifact would appear along mesh edges. Nothing within the algorithm should change the input images in such a way. I hope that I find the time to investigate this during the holidays.

[pierotofy]

Please let me know if I can provide you with data or ways to reproduce it. Happy holidays!

[juvinski]

Me too, I have three or four datasets with same problem

[nmoehrle]

Is one of these datasets small enough that you can bisect the issue? Was there a version of texrecon that did not produce these issues?

[pierotofy]

I remember first seeing this problem about 8 months ago. It's possible that it worked before, but I wouldn't be able to tell. @juvinski ?

Tomorrow I'll try to put together a reproducible dataset.

[pierotofy]

@juvinski would you be willing to share one of your datasets' images on google drive?

[juvinski]

hi @pierotofy and @nmoehrle ,

I don't remember any dataset with this problems since the datasets O processed since November.
I'm running some datasets taked around this year to see if I can see the problem happening with this older datasets.
I have tested in two versions of texrecon - one built in March and other in November - both generated the model with the fail.
This is a dataset - around 760 Mb - If you need something bigger or smallest please let me know.
https://drive.google.com/open?id=1m9QtEJkjk6CvmWDJ3BfoIhipIC_gN_kE

Best regards

[pierotofy]

Ok here's a full, reproducible test case:

Texrecon version is from the OpenDroneMap fork (https://github.com/OpenDroneMap/mvs-texturing/) which is almost on par with the current master.

Dataset with mesh, images and visualsfm file: https://drive.google.com/open?id=12rrV51VSwXvQgXf-TbZBubwN9i-YMZ8w

texrecon /subset/reconstruction.nvm /subset/odm_mesh.ply /subset/output -d gmi -o gauss_clamping -t none

[juvinski]

Btw,
the -d area flag produces the same result.

[pierotofy]

Found a smaller reproducible dataset (18 images) which might be easier to work with.

https://drive.google.com/open?id=1KUPVMyGwStRirXJmoC6BNoiwFp2pGLTF

# texrecon reconstruction.nvm odm_25dmesh.ply odm_textured_model -d gmi -o gauss_clamping -t none

Top right corner shows:

[nmoehrle]

I finally managed to look into this and the artifact is introduced during the global seam leveling, if the global seam leveling is disabled (--skip_global_seam_leveling) the artifact is gone. However, I have yet to track down why it is introduced. There is a lot of overlapping geometry in the mesh as well as gigantic triangles, but the area around the artifact looks clean as far as I can tell.

[nmoehrle]

I just had a result where everything was pink like yours. I initially thought that was the original color you fed into the algorithm, did the algorithm distort the color towards pink?

[pierotofy]

You mean the "subset" dataset? The images are already pinkish due to the camera filter that was used to take the images.

[nmoehrle]

Ah ok no I just go a completely corrupt result when retexturing with -L and thought you might have experienced the same...

[pierotofy]

Oh I see, no I've never seen that type of pink output. ^_^

[juvinski]

Hi,
I didn’t notice problems until last October.
I will try to strip some big dataset. I notice my new camera - nir + green + blue - all datasets have problems instead my rgb.

[juvinski]

Hi,

this dataset can be used.
If you want another one or a little one please let me know.

https://drive.google.com/drive/folders/1DYESz3VnH95kPolA-Qhm1rx4OaZfCJhk?usp=sharing

Best regards.

[nmoehrle]

Ok so the issue arises due to the jpeg compression of the undistored images.

The current implementation assumes a pitch black (RGB(0,0,0)) border within undistorted images and calculates a validity mask from it. In a lossy compressed format like jpeg the border is not pitch black and thus regions that should be invalid might be classified as valid and used in the reconstruction. The artifacts we see are the undistortion borders of images.

My previous assessment that the the problem is caused by the global seam leveling was wrong -- there was a race condition when using data costs or the labeling of a previous run (-L, -D) which was fixed with 20bf2a8.

I am reluctant to increase the threshold although this would fix this particular problem because it might cause other issues.

Is it possible for you to write out undistorted images in a lossless format?

[pierotofy]

Thanks for taking the time to look into this @nmoehrle! I think we can export the images in a lossless format, I will discuss it with the OpenDroneMap team.

[juvinski]

Hi @nmoehrle,

I'm generating undistorted images in png format( and testing compression from 0 to 9).
This change give me better results, for instance, with jpg I was having 65 slashes in the result, now using png I'm having 5. Do you have any clue about how get this result better ?

[nmoehrle]

Oh that's surprising - can you provide the dataset that gives you five slashes?

[juvinski]

in true is the same dataset - I was comparing the image convertion to see what results or differences I will get.
There are datasets I didn't have any slashes with png convertion, but there are I have this diminution.
Previous - with jpg:

After with png:

This is the dataset. https://drive.google.com/open?id=1y2G1Et72H4zsVU0EVNvypZVAtk88Y-Yw
If you wish I can give the model and textures too.

[juvinski]

Hi @nmoehrle , I discovered something more about this issue.
I have 2 cameras - all the same except the lens - one is with 3.97 mm and other is 4.35mm.
The 3.97 I didn't have problems, just on the 4,95 lens. How the focal can interfere with the that ?

Best regards.

[pierotofy]

@juvinski you should try to provide a reproducible dataset (images, nvm file, etc.) so that we can try to identify the problem further. Without it it's a bit difficult 👍

[dbadrian]

Has anyone ever looked deeper into this and found out something new? Using some lossless compression like png fixed a lot of issues, but still some remain as for @juvinski .

Edit: Unlike mentioned above, I could resolve quite a few/all of the left-over artifacts by using area instead of gmi as data term. Why exactly this caused them I can't tell (yet), but I intuitively I'm not surprised that it might cause some local favoring of strong gradients (of maybe a slightly misaligned camera view?)