Stitching equirectangular images with Neural Style Transfer applied.

[StijnWillemen]

Hi!

I have been using your stitcher for blending equirectangular images with a self-created overlap (by pasting a part of the picture on the opposite side).

Then applying a Neural Style Transfer which uses an input style image and original image to restyle the original image to the input style image as if it was created in that style.

Afterwards I split the image down the middle and swap the pieces so it can be blended.
Using the original unstyled image to find key-features but applying all transformations/blending to the styled image this
generates a very good result, as there is no visible line in the output image on the blended part.

But this creates a new stitch line on the opposite side of the image (in 360° view).
It seems this stitch is created because one of the two images slightly shifted vertically and there also seem to be a kind of fading color effect on the sides of the image which is visible when connecting the sides of the image.
(very bad quality because this screenshot is taken in a 360° web viewer)

Are there any parameters I could change to combat this effect?

Thanks in advance.

[lukasalexanderweber]

I don't know if I followed correctly. I'll try to explain your workflow in my own words:

1. You stitch some images together
2. You apply NST on the result
3. You now want to distribute the image content differently (e.g. put the person from center to left), so you

Afterwards I split the image down the middle and swap the pieces so it can be blended.

meaning to me:

you perform another stitch just to move the content.

---

A better sollution to just to move the content could be the following:

• Decide on which x value you want to swap the content
• Move the right part of your NST to the left and the left part to the right of this x

old_right_new_left = img[:, x:]
old_left_new_right = img[:, :x]
result = np.concatenate((old_right_new_left , old_left_new_right), axis=1)

The code is not tested and I'm not sure if I understood you correctly. Here are my sources:

https://stackoverflow.com/questions/15589517/how-to-crop-an-image-in-opencv-using-python
https://stackoverflow.com/questions/7589012/combining-two-images-with-opencv

[StijnWillemen]

Thanks for the quick response.

The position of elements does not matter as this is for 360° view in Virtual Reality but I split the images and blend them because there is a stitching line where the outer left and outer right side of the image meet when viewed in VR.

Here is a github docs page with images in steps to show the process.
https://modernwayz.github.io/AI-Project/docs/Algemeen.html#sift-advanced

And I use two images the original and the NST image where I use the original image as input for mapping key features and then transform the NST image. I do this because barely any key features can be found on a NST image.

But in the stitching process the camera angle is shifted slightly (I think) and there is some sort of color saturation which causes a new stitching line when bringing the left and right side together.

Sorry for the unclear explanation in my first post.
Here is my forked repository with the changes to use two different input imagesets and example images/experiments.
https://github.com/StijnWillemen/stitching

[lukasalexanderweber]

To me it seems the different color saturation comes from the NST. What about doing the stitch first and NST later?

[lukasalexanderweber]

But you can also play around with blending methods and blend widths. Also make sure that exposure error compensation and seam carving is run on the NST Images and not on the original.